A data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, electronic equipment and storage medium, the method comprising: obtaining online behavior data of a user, the online behavior data comprising data generated by the user interacting with a plurality of first objects online; calculating the online behavior data based on a Bayesian personalized ranking algorithm to determine prediction information corresponding to the first objects, the prediction information being used to describe a probability of the user generating positive feedback for the first objects; selecting online behavior data corresponding to a second object from the online behavior data according to the prediction information; and inputting the online behavior data corresponding to the second object into a pre-trained prediction model to determine a target probability of the user generating positive feedback for the second object. According to embodiments of the present application, the prediction efficiency can be improved.

Description

Technical Field

[0001] This application belongs to the field of Internet technology, and in particular relates to a data processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the development of information technology, research on consumer behavior analysis is on the rise. In many applications, explicit feedback in the form of ratings is often difficult to collect, resulting in a limited amount of data. Therefore, utilizing implicit feedback data for user preference learning and personalized recommendations becomes crucial. However, implicit feedback typically does not represent a user's absolute preferences.

[0003] Therefore, current recommendation algorithms cannot accurately and efficiently determine a user's liking or satisfaction with an item. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, and storage medium that can solve the problem of low prediction accuracy in current recommendation algorithms.

[0005] In a first aspect, embodiments of this application provide a data processing method, the method comprising:

[0006] Acquire users' online behavior data, which includes data generated from users' online interactions with multiple primary objects;

[0007] The Bayesian personalized ranking algorithm is used to calculate online behavior data to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will give positive feedback to the first object.

[0008] Based on the prediction information, the online behavior data corresponding to the second object is filtered out from the online behavior data;

[0009] The online behavior data corresponding to the second object is input into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object.

[0010] Secondly, embodiments of this application provide a data processing apparatus, which includes:

[0011] The first conversion module is used to acquire the user's online behavior data, which includes data generated by the user's online interaction with multiple first objects;

[0012] The Bayesian personalized ranking algorithm is used to calculate online behavior data to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will give positive feedback to the first object.

[0013] Based on the prediction information, the online behavior data corresponding to the second object is filtered out from the online behavior data;

[0014] The online behavior data corresponding to the second object is input into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object.

[0015] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the method as described in the first aspect or any possible implementation of the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium storing computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any possible implementation thereof.

[0017] In this embodiment, by acquiring user online behavior data, including data generated from online interactions between the user and multiple first objects, and calculating the online behavior data based on a Bayesian personalized ranking algorithm, prediction information corresponding to the first objects is determined. This prediction information describes the probability of the user generating positive feedback to the first objects, thus initially predicting the degree of positive feedback. Based on the prediction information, online behavior data corresponding to second objects is filtered from the online behavior data, enabling data cleaning, reducing the amount of data requiring subsequent prediction processing, lowering prediction costs, and improving prediction efficiency. Finally, the online behavior data corresponding to the second objects is input into a pre-trained prediction model to determine the target probability of the user generating positive feedback to the second objects, enabling rapid and accurate prediction of the target probability of the user generating positive feedback to the second objects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a data processing method provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] The features and exemplary embodiments of various aspects of this application will now be described in detail. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain this application and are not configured to limit this application. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0024] The following is a brief introduction to the technical terms used in this application.

[0025] Implicit feedback is often referred to as one-class feedback; therefore, recommendation problems based on implicit feedback are also known as one-class collaborative filtering (OCCF) problems.

[0026] Bayesian Personalized Ranking (BPR) is a ranking algorithm that uses implicit feedback, such as click behavior information and collection behavior information. It ranks objects by obtaining the maximum posterior probability through Bayesian analysis of the question, and then generates recommendations.

[0027] Bayesian statistics, unlike general statistical methods, not only utilizes model and data information but also makes full use of prior information. Through empirical analysis, the prediction results of the Bayesian prediction model are compared with those of the ordinary regression prediction model, demonstrating the significant superiority of the Bayesian prediction model. The BPR algorithm processes user ratings of items (explicit feedback "1", implicit feedback "0") into a set of pairs.<i,j> Let i represent items with a rating of 1 and j represent items with a rating of 0. Assuming a user has M ratings of "1" and N ratings of "0", then this user has M*N pairs. Thus, the dataset consists of triples.<u,i,j> This indicates that the physical meaning of the triplet is: user "u" prefers item "i" to item "j".

[0028] The data processing method provided in this application embodiment can be applied to at least the following application scenarios, which will be described below.

[0029] In recent years, with the rise of big data, research on consumer behavior analysis has flourished, and scholars from many fields such as databases and data mining, information systems and information management, image processing and computer vision, social network analysis, and e-commerce have joined the ranks of consumer behavior researchers.

[0030] Meanwhile, this research field has also attracted significant attention from companies operating in the digital economy, including e-commerce and social networks. In these emerging fields, consumer behavior research is known as consumer profiling and holds an important position within research areas such as social computing.

[0031] Traditional matrix factorization uses explicit feedback to predict user scores for unrated items by decomposing the user's rating matrix for items, and then makes recommendations based on these scores. While explicit feedback has high accuracy in practice, it is often difficult to collect, and sometimes implicit feedback must be used, which can be easily obtained from log files.

[0032] In many applications, explicit feedback in the form of ratings is often difficult to collect, resulting in a limited amount of data. Therefore, leveraging implicit feedback data, such as purchase behavior, for user preference learning and personalized recommendations becomes crucial. Implicit feedback typically does not represent a user's absolute preferences; for example, a user not purchasing an item does not necessarily mean that the user dislikes that item.

[0033] For example, for a user, some samples might focus on what videos they watched. Beyond those videos, it's unknown whether they watched the rest. Not knowing doesn't mean the user didn't watch them; perhaps they watched them on other websites, or perhaps they watched them on television.

[0034] Therefore, current recommendation algorithms cannot accurately and efficiently determine a user's level of liking or satisfaction with an item.

[0035] Figure 1 This is a flowchart of a data processing method provided in an embodiment of this application.

[0036] like Figure 1 As shown, the data processing method may include steps 110-140, and the method is applied to a data processing device, as detailed below:

[0037] Step 110: Obtain the user's online behavior data, which includes data generated from the user's online interactions with multiple first objects.

[0038] Step 120: Calculate the online behavior data based on the Bayesian personalized ranking algorithm to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will give positive feedback to the first object.

[0039] Step 130: Based on the prediction information, filter out the online behavior data corresponding to the second object from the online behavior data.

[0040] Step 140: Input the online behavior data corresponding to the second object into the pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object.

[0041] In this embodiment, by acquiring user online behavior data, including data generated from online interactions between the user and multiple first objects, and calculating the online behavior data based on a Bayesian personalized ranking algorithm, prediction information corresponding to the first objects is determined. This prediction information describes the probability of the user generating positive feedback to the first objects, thus initially predicting the degree of positive feedback. Based on the prediction information, online behavior data corresponding to second objects is filtered from the online behavior data, enabling data cleaning, reducing the amount of data requiring subsequent prediction processing, lowering prediction costs, and improving prediction efficiency. Finally, the online behavior data corresponding to the second objects is input into a pre-trained prediction model to determine the target probability of the user generating positive feedback to the second objects, enabling rapid and accurate prediction of the target probability of the user generating positive feedback to the second objects.

[0042] The following describes the contents of steps 110-140 respectively:

[0043] Step 110 is involved.

[0044] Acquire users' online behavior data, which includes data generated from users' online interactions with multiple primary objects.

[0045] Online behavioral data includes data generated from users' online interactions with multiple first objects, specifically including: data generated from browsing first objects, data generated from commenting on first objects, data generated from collecting first objects, and data generated from purchasing first objects.

[0046] Step 120 is involved.

[0047] The Bayesian personalized ranking algorithm is used to calculate online behavioral data and determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will give positive feedback to the first object.

[0048] Among these methods, a matrix factorization-based approach, also known as the Bayesian personalized ranking algorithm, is used to predict the probability of a user's preference for an unknown item, i.e., the first item. Simultaneously, an optimized BPR algorithm employs a Bayesian prediction model to predict triplet parameters instead of randomly sampled parameters.

[0049] In one possible embodiment, step 120 may specifically include the following steps:

[0050] Based on the Bayesian personalized ranking algorithm, online behavioral data is calculated to determine the first probability that the category of the first object is consistent with the category of the preset object;

[0051] Based on the Bayesian personalized ranking algorithm, online behavioral data is calculated to determine the second probability that the category of the first object is inconsistent with the category of the preset object;

[0052] The prediction information is determined based on the first probability and the second probability.

[0053] The preset object can be an object preferred by the user. The second probability that the category of the first object is the same as the category of the preset object is R1, that is, the probability that the user prefers the first object is R1. The second probability that the category of the first object is different from the category of the preset object is R2, that is, the probability that the user does not prefer the first object is R2.

[0054] The extracted monitoring indicator feature values ​​are represented as: T = {t1, t2, t3, ...}, representing user U's rating data for uninteracted item j. According to the conditional probability formula, under the condition of satisfying feature value T, the first probability that the category of the first object matches the category of the preset object is:

[0055]

[0056] Where P(T|R1) represents the probability of feature value T appearing in the preference subset of the sample set, and P(T|R1) represents the proportion of the user preference subset in the entire sample set.

[0057] Similarly, the probability that a suspected preference is judged as R² under the condition of the occurrence of feature value T, i.e., the second probability that the category of the first object is inconsistent with the category of the preset object, can be expressed as:

[0058]

[0059] Where P(T|R2) represents the probability of feature value T appearing in the normal subset of the sample set, and P(T|R2) represents the proportion of the user-disliked subset in the entire sample set.

[0060] Finally, the prediction information is determined based on the first probability P(R1|T) and the second probability P(R2|T).

[0061] Step 130 is involved.

[0062] Based on the prediction information, the online behavior data corresponding to the second object is filtered out from the online behavior data.

[0063] In one possible embodiment, step 130 may specifically include the following steps:

[0064] If the first probability is greater than the second probability, the online behavior data is determined to be the online behavior data corresponding to the second object.

[0065] Due to the Bayesian approach, each feature value is independent of the others. After calculating the probabilities of the suspected preference as the first and second probabilities based on the relatively independent event probabilities, the two probabilities are compared. If P(R1|T) is greater than P(R2|T), it is considered that the analyzed user U has a preference for the rating data of the uninterrupted item j. That is, if the first probability is greater than the second probability, the online behavior data is determined to be the online behavior data corresponding to the second object.

[0066] If P(R1|T) is not greater than P(R2|T), then the analyzed user U is considered to have no preference for the rating data of the uninterrupted item j. That is, if the first probability is not greater than the second probability, it can be inferred that the user has no preference for the second object.

[0067] That is, if the first probability is greater than the second probability, it can be inferred that the user prefers the second object; if the first probability is not greater than the second probability, it can be inferred that the user does not prefer the second object.

[0068] Therefore, the online behavior data of the second object with a higher probability than the second object can be filtered out, thus achieving the function of data cleaning. The online behavior data of objects that are unlikely to be favored by users can be filtered out, reducing the amount of data for subsequent prediction processing, reducing prediction costs, and improving prediction efficiency.

[0069] Step 140 is involved.

[0070] The online behavior data corresponding to the second object is input into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object.

[0071] Before inputting the online behavior data corresponding to the second object into the pre-trained prediction model, a prediction model can be built first. The specific steps for building the prediction model are as follows:

[0072] First, the preference hypothesis and prediction formula are constructed to provide the initial parameter relationship for the triple array of the BPR algorithm.

[0073] The formula incorporates pairwise preference data defined in the database for two second objects belonging to the same user:

[0074]

[0075] Where, if r ui >r uj This means that user u is very likely to like an item i∈τ that they have interacted with. u It is better than an item j∈τ\τ that it has never interacted with. u Compared to the absolute preferences in the pointwise preference hypothesis, the relative preferences in the pairwise preference hypothesis are generally more reasonable. This pairwise preference hypothesis often produces better recommendations than the pointwise preference hypothesis.

[0076] Secondly, the likelihood and optimization objective of pairwise preferences are used to refine and optimize the triplet objective function of the BPR algorithm. The optimization objective of the BPR algorithm is expressed as:

[0077]

[0078] Among them, f uij This represents the objective function (including the loss function and regularization term) defined on a triple (u, i, j);

[0079] Next, the gradient is calculated, and the formula and algorithm flow are updated to provide the necessary operational parameters for the triple array of the BPR algorithm.

[0080]

[0081]

[0082]

[0083]

[0084]

[0085] in,

[0086] With the gradients above, the parameters can be updated according to the following rules:

[0087]

[0088] The BPR algorithm process may specifically include: randomly selecting a pair (u, i) ∈ R, and randomly selecting from j\j u Select an item j from the list, replace the random selection j with the rating preference j obtained from the prediction model, calculate the gradient according to Equation 5, and update the model parameters according to Equation 6.

[0089] Finally, parameter initialization and hyperparameter settings are performed.

[0090] Specifically, parameters can be initialized based on the statistical characteristics of the data:

[0091]

[0092] V ik =(r-0.5)×0.01,k=1,2,…,d

[0093] U uk =(r-0.5)×0.01,k=1,2,…,d

[0094] Where r is a random number, 0 <= r < 1.

[0095] In one possible embodiment, the pre-trained prediction model is set on a preset server, and before step 140, the following steps may also be included:

[0096] Get the saturation of worker threads, the saturation of computing tasks, and the growth rate of the computing task buffer queue; worker threads are multiple threads between the first device and the preset server, and the computing tasks include online behavioral data corresponding to at least one second object;

[0097] Based on preset weighting coefficients, the saturation of worker threads, the saturation of computing tasks, and the growth rate of the buffer queue are weighted and calculated to obtain the thread pool load.

[0098] If the thread pool load exceeds the preset load, adjust the working status information of the worker threads.

[0099] Specifically, based on preset weighting coefficients, the saturation of worker threads, the saturation of computational tasks, and the growth rate of the buffer queue are weighted to obtain the thread pool load. The specific load can be determined as follows:

[0100]

[0101] Where N is the number of worker threads in the thread pool at runtime, N max This is the maximum number of threads set. Describes the saturation of worker threads; T cur T is the number of tasks in the current data collection time window. pre Q is the number of tasks in the previous data collection window, and Q is the size of the task buffer queue. Describe the current task saturation. Describes the growth rate of the task buffer queue; ξ is a preset weighting coefficient.

[0102] By comparing the calculated thread pool load with the preset load ω', if the thread pool load is greater than the preset load, the working status information of the worker threads is adjusted, for example, by distributing unloaded computing tasks to other threads. If the thread pool load is not greater than the preset load, the current collection time window is skipped, meaning no adjustment to the worker threads is needed.

[0103] In one possible embodiment, high-frequency and working threads can be matched for the computation task based on the prediction information corresponding to the first object, wherein the prediction information is used to describe the probability that the user will generate positive feedback to the first object.

[0104] Here, worker threads are allocated based on the preference probability of unpurchased items, i.e., the prediction information corresponding to the first object, from high to low, and the calculated order of more to fewer idle threads. These worker threads are used to pass the prediction information corresponding to the first object to the prediction model for training.

[0105] Here, objects with a high probability of user preference can be trained first to improve training efficiency.

[0106] In one possible embodiment, the preset server includes multiple computing nodes, and before step 140, the following steps may also be included:

[0107] Obtain computing power and latency information from multiple computing nodes;

[0108] Based on prediction information, computing power information, and latency information, computing nodes are matched for computing tasks.

[0109] Here, by using the prediction information corresponding to the first object, the computing power information and latency information of the computing nodes, computing nodes are matched for computing tasks. This allows for matching computing nodes with different idle computing power based on the probability of user preference for unknown items as indicated by the prediction information. This optimizes the slow convergence speed of existing BPR algorithms and prevents the computing power consumed by tasks from exceeding the idle computing power of the computing nodes. Furthermore, computing tasks corresponding to objects with high user preference probabilities can be trained first to improve training efficiency.

[0110] The aforementioned acquisition of computing power and latency information from multiple computing nodes includes:

[0111] Obtain the redundant computing power, mapping ratio coefficient, and mapping function of the computing node;

[0112] The computing power information is calculated based on the redundancy computing power, the mapping ratio coefficient, and the mapping function.

[0113] The computing power information is calculated based on the redundancy computing power, the mapping ratio coefficient, and the mapping function. Specifically, it can be determined in the following way:

[0114]

[0115] In the formula, C br y represents the total computing power requirement, i.e., computing power information; f(x) is the mapping function; a, B, and y are mapping ratio coefficients; q represents redundant computing power.

[0116] Taking parallel computing capability as an example, assuming there are three different types of parallel computing chip resources: b1, b2, and b3, then f(b j ) represents the mapping function of the parallel computing capability that the j-th parallel computing chip b can provide, and q2 represents the redundant computing power of parallel computing.

[0117] Here, the computing power information of each computing node can be calculated separately, so as to allocate computing nodes reasonably for computing tasks.

[0118] The multiple computing nodes include edge computing nodes and cloud data center computing nodes. The acquisition of computing power and latency information for these multiple computing nodes, as mentioned above, includes:

[0119] The first transmission latency from the first device to the edge computing node, the processing latency of the edge computing node, the second transmission latency between the edge computing node and the cloud data center computing node, and the computing latency of the cloud data center computing node are obtained.

[0120] Calculate the delay information based on the first transmission delay, processing delay, second transmission delay, and calculation delay.

[0121] The total latency of a computing task includes the first transmission latency from the first device to the edge computing node, the processing latency of the edge computing node, the second transmission latency between the edge computing node and the cloud data center computing node, and the computing latency of the cloud data center computing node. In other words, the total latency of the computing task in the system is:

[0122]

[0123] The following will explain each point:

[0124] This involves the first transmission delay from the first device to the edge computing node.

[0125] Assume a user accesses an edge computing node via wireless communication. Let the data transmission bandwidth of the wireless communication link be B. According to Shannon's theorem, in a channel environment with limited bandwidth and noise interference, the transmission delay from user i to the edge computing node is:

[0126]

[0127] Where, p i h is the transmission power of the i-th device; k，i σ is the channel gain from the i-th user terminal to the k-th edge node, which is a random, independent, and identically distributed variable; 2 It is the power of additive white Gaussian noise.

[0128] This involves the processing latency of edge computing nodes.

[0129] User tasks can be scheduled based on different needs for computing and network resources; that is, some tasks can be performed on edge computing nodes, while others can be offloaded to cloud data centers for computation. i λ represents the proportion of the computing tasks assigned to the i-th user by its corresponding edge computing node. i [0,1]. Then 1﹣λ i A proportion of computational tasks are offloaded to cloud data centers, using This represents the computing resources allocated to user i by the k-th edge computing node.

[0130] Therefore, the edge computing latency of task i is:

[0131]

[0132] The second transmission latency involves edge computing nodes and cloud data center computing nodes.

[0133] Assume the cloud data center provides bandwidth W for task i at the k-th edge node. k，i For a connection service with a speed of (bit / s), the transmission latency from the edge node to the cloud data center can be expressed as:

[0134]

[0135] This involves the computation latency of cloud data center computing nodes.

[0136] Data center allocation f i c If the computing resources are used to compute the computing task of the i-th user, then the computing latency of the cloud data center is expressed as:

[0137]

[0138] In the wireless time domain, latency is a crucial characteristic for evaluating system performance. It can be measured by the sum of the task queue lengths across the cloud, network, edge, and endpoint segments. Considering the dynamic queue characteristics of edge nodes and cloud data center nodes, the average system latency can be expressed as:

[0139]

[0140] Among them, S K (t) represents the queue of computing tasks that are unloaded to the cloud data center server at any given time.

[0141] Q K (t) The task queue existing on the edge computing node. t represents the t-th decision time.

[0142] Calculate the latency from the network node containing the current model to the network node containing the computing power, and allocate the model computation to the node with the shortest latency and the most idle computing power. This prevents the computing power consumed by the task from exceeding the idle computing power of the computing node.

[0143] The data processing method provided in this application acquires users' online behavior data, including data generated from online interactions between users and multiple first objects. Based on a Bayesian personalized ranking algorithm, the online behavior data is calculated to determine the prediction information corresponding to the first objects. This prediction information describes the probability of the user generating positive feedback to the first objects, thus initially predicting the degree of positive feedback. Based on the prediction information, online behavior data corresponding to second objects is filtered from the online behavior data, enabling data cleaning, reducing the amount of data requiring subsequent prediction processing, lowering prediction costs, and improving prediction efficiency. Finally, the online behavior data corresponding to the second objects is input into a pre-trained prediction model to determine the target probability of the user generating positive feedback to the second objects, enabling rapid and accurate prediction of the target probability of the user generating positive feedback to the second objects.

[0144] Based on the above Figure 1 The data processing method shown in this application also includes a data processing apparatus, such as... Figure 2 As shown, the data processing device 200 may include:

[0145] The acquisition module 210 is used to acquire the user's online behavior data, which includes data generated by the user's online interaction with multiple first objects;

[0146] The first determining module 220 is used to calculate online behavior data based on the Bayesian personalized ranking algorithm to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will generate positive feedback to the first object.

[0147] The filtering module 230 is used to filter out the online behavior data corresponding to the second object from the online behavior data based on the prediction information.

[0148] The second determining module 240 is used to input the online behavior data corresponding to the second object into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object.

[0149] In one possible embodiment, the first determining module 220 is specifically used for:

[0150] Based on the Bayesian personalized ranking algorithm, online behavioral data is calculated to determine the first probability that the category of the first object is consistent with the category of the preset object;

[0151] Based on the Bayesian personalized ranking algorithm, online behavioral data is calculated to determine the second probability that the category of the first object is inconsistent with the category of the preset object;

[0152] The prediction information is determined based on the first probability and the second probability.

[0153] In one possible embodiment, the filtering module 230 is specifically used for:

[0154] If the first probability is greater than the second probability, the online behavior data is determined to be the online behavior data corresponding to the second object.

[0155] In one possible embodiment, the pre-trained prediction model is set on a preset server, and the data processing device 200 may further include:

[0156] The first acquisition module is used to acquire the saturation of worker threads, the saturation of computing tasks, and the growth rate of the buffer queue of computing tasks; the worker threads are multiple threads between the first devices and the preset server, and the computing tasks include online behavioral data corresponding to at least one second object;

[0157] The weighting module is used to calculate the thread pool load by weighting the saturation of worker threads, the saturation of computing tasks, and the growth rate of the buffer queue based on preset weight coefficients.

[0158] The adjustment module is used to adjust the working status information of worker threads when the thread pool load exceeds the preset load.

[0159] In one possible embodiment, the preset server includes multiple computing nodes, and the data processing device 200 may further include:

[0160] The second acquisition module is used to acquire computing power information and latency information of multiple computing nodes;

[0161] The matching module is used to match computing nodes for computing tasks based on prediction information, computing power information, and latency information.

[0162] In one possible embodiment, the second acquisition module is specifically used for:

[0163] Obtain the redundant computing power, mapping ratio coefficient, and mapping function of the computing node;

[0164] The computing power information is calculated based on the redundancy computing power, the mapping ratio coefficient, and the mapping function.

[0165] In one possible embodiment, the multiple computing nodes include: edge computing nodes and cloud data center computing nodes; the second acquisition module is specifically used for:

[0166] The first transmission latency from the first device to the edge computing node, the processing latency of the edge computing node, the second transmission latency between the edge computing node and the cloud data center computing node, and the computing latency of the cloud data center computing node are obtained.

[0167] Calculate the delay information based on the first transmission delay, processing delay, second transmission delay, and calculation delay.

[0168] In this embodiment, by acquiring user online behavior data, including data generated from online interactions between the user and multiple first objects, a Bayesian personalized ranking algorithm is used to calculate the predictive information corresponding to the first objects. This predictive information describes the probability of the user providing positive feedback to the first objects, thus initially predicting the degree of positive feedback. Based on the predictive information, online behavior data corresponding to second objects is filtered from the online behavior data. This process enables data cleaning, reduces the amount of data requiring subsequent predictive processing, lowers prediction costs, and improves prediction efficiency. Finally, the online behavior data corresponding to the second objects is input into a pre-trained prediction model to determine the target probability of the user providing positive feedback to the second objects, enabling rapid and accurate prediction of the target probability of the user providing positive feedback to the second objects.

[0169] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown.

[0170] An electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0171] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0172] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory. In a particular embodiment, memory 302 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0173] The processor 301 implements any of the data processing methods in the embodiment shown in the figure by reading and executing computer program instructions stored in the memory 302.

[0174] In one example, the electronic device may also include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0175] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0176] Bus 310 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0177] The electronic device can execute the data processing method described in the embodiments of this application, thereby achieving the combination Figure 2 Describe the data processing methods.

[0178] Furthermore, in conjunction with the data processing methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; these computer program instructions are implemented when executed by a processor. Figure 1 Data processing methods.

[0179] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0180] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0181] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0182] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data processing method, characterized in that, Applied to a first device, the method includes: Acquire user's online behavior data, which includes data generated from the user's online interactions with multiple first objects; the first objects are items with unknown user preferences; the online behavior data includes: data generated from browsing the first objects, data generated from commenting on the first objects, data generated from collecting the first objects, and data generated from purchasing the first objects; The online behavior data is calculated based on the Bayesian personalized ranking algorithm to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will give positive feedback to the first object. Based on the prediction information, online behavior data corresponding to a second object is filtered out from the online behavior data; the second object is the user preference object filtered out from the first object; The online behavior data corresponding to the second object is input into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object. The process of building a pre-trained prediction model includes: The preference hypothesis and prediction formula provide the initial parameter relationship for the triple array of the BPR algorithm; The formula incorporates pairwise preference data defined in the database for two second objects belonging to the same user: Among them, if If user u likes an item they have interacted with, then... Better than an item that the user has never interacted with. ; The likelihood of pairwise preferences and the optimization objective are used to optimize the triplet objective function of the BPR algorithm, where the BPR algorithm optimization objective is expressed as: in, Let represent the objective function defined on a triple (u, i, j).

2. The method according to claim 1, characterized in that, The calculation of the online behavior data based on the Bayesian personalized ranking algorithm to determine the prediction information corresponding to the first object includes: The online behavior data is calculated based on the Bayesian personalized ranking algorithm to determine the first probability that the category of the first object is consistent with the category of a preset object; the preset object is the object preferred by the user. The online behavior data is calculated based on the Bayesian personalized ranking algorithm to determine a second probability that the category of the first object is inconsistent with the category of the preset object; The prediction information is determined based on the first probability and the second probability.

3. The method according to claim 2, characterized in that, The step of filtering online behavior data corresponding to the second object from the online behavior data based on the prediction information includes: If the first probability is greater than the second probability, the online behavior data is determined to be the online behavior data corresponding to the second object.

4. The method according to claim 1, characterized in that, The pre-trained prediction model is set on a preset server. Before inputting the online behavior data corresponding to the second object into the pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object, the method further includes: The saturation of worker threads, the saturation of computing tasks, and the growth rate of the buffer queue of the computing tasks are obtained; the worker threads are multiple threads between the first device and the preset server, and the computing tasks include at least one online behavior data corresponding to the second object; Based on preset weighting coefficients, the saturation of the worker threads, the saturation of the computing tasks, and the growth rate of the buffer queue are weighted and calculated to obtain the thread pool load. If the thread pool load exceeds the preset load, adjust the working status information of the worker threads.

5. The method according to claim 4, characterized in that, The preset server includes multiple computing nodes. Before inputting the online behavior data corresponding to the second object into the pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object, the method further includes: Obtain computing power and latency information of the multiple computing nodes; Based on the prediction information, the computing power information, and the latency information, a computing node is matched for the computing task.

6. The method according to claim 5, characterized in that, The acquisition of computing power and latency information from multiple computing nodes includes: Obtain the redundant computing power, mapping ratio coefficient, and mapping function of the computing node; The computing power information is calculated based on the redundant computing power, the mapping ratio coefficient, and the mapping function.

7. The method according to claim 5, characterized in that, The plurality of computing nodes includes: edge computing nodes and cloud data center computing nodes. The acquisition of computing power and latency information of the plurality of computing nodes includes: The first transmission latency from the first device to the edge computing node, the processing latency of the edge computing node, the second transmission latency between the edge computing node and the cloud data center computing node, and the computing latency of the cloud data center computing node are obtained. The delay information is calculated based on the first transmission delay, the processing delay, the second transmission delay, and the calculation delay.

8. A data processing apparatus, characterized in that, Applied to the first device, the data processing apparatus includes: The acquisition module is used to acquire the user's online behavior data, which includes data generated by the user's online interaction with multiple first objects; the first objects are items with unknown user preferences; the online behavior data includes: data generated by browsing the first objects, data generated by commenting on the first objects, data generated by collecting the first objects, and data generated by purchasing the first objects; The first determining module is used to calculate the online behavior data based on the Bayesian personalized ranking algorithm to determine the prediction information corresponding to the first object. The prediction information is used to describe the probability that the user will generate positive feedback to the first object. The filtering module is used to filter out the online behavior data corresponding to the second object from the online behavior data based on the prediction information; the second object is the user preference object filtered from the first object; The second determining module is used to input the online behavior data corresponding to the second object into a pre-trained prediction model to determine the target probability that the user will generate positive feedback to the second object. The process of building a pre-trained prediction model includes: The preference hypothesis and prediction formula provide the initial parameter relationship for the triple array of the BPR algorithm; The formula incorporates pairwise preference data defined in the database for two second objects belonging to the same user: Among them, if If user u likes an item they have interacted with, then... Better than an item that the user has never interacted with. ; The likelihood of pairwise preferences and the optimization objective are used to optimize the triplet objective function of the BPR algorithm, where the BPR algorithm optimization objective is expressed as: in, Let represent the objective function defined on a triple (u, i, j).

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the data processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the data processing method as described in any one of claims 1-7.

Images