Object behavior data storage and object behavior prediction method, device and system

Abstract

The application discloses an object behavior data storage and object behavior estimation method, device and system. The estimation method comprises the following steps: obtaining behavior memory data of an object to be estimated recorded in memory slots of each level in a memory network model; performing weighted calculation on the behavior memory data of the object obtained from the memory slots of different levels to obtain long-term behavior memory data of the object; taking feature data of display information, static feature data of the object and the long-term behavior memory data of the object as inputs of a click estimation model to obtain an object behavior probability estimation result of the object to the display information. When the object behavior probability estimation is performed, long-term and / or short-term object behavior data can be considered according to needs, the estimation result is more accurate, and the object can be recommended with display information more in line with needs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network, in particular to an object behavior data storage and object behavior estimation method, device and system. BACKGROUND

[0002] With the development of network, the speed of information dissemination and updating is getting faster and faster, which makes the behavior of the object show short-term, variability and other characteristics. For example, the same object (such as a user) may show different behaviors in different scenarios and at different time points, and the reason behind it may be the change of the role of the object, the migration of the interest of the object, etc. In the case of increasing factors leading to uncertain object behavior, how to accurately estimate the change of object behavior has become a difficult problem in the field of big data analysis and mining.

[0003] The present application inventors found that the short-term behavior of the object cannot well reflect the change of the preference of the object over time, and therefore, the estimation of the future behavior of the object based on the short-term behavior of the object will result in inaccurate estimation results. SUMMARY

[0004] In view of the above problems, the present application is proposed in order to provide an object behavior data storage and object behavior estimation method, device and system which can overcome the above problems or at least partially solve the above problems.

[0005] The embodiment of the present application provides an object behavior data storage method, comprising:

[0006] obtaining a current operation behavior of an object to display information;

[0007] when the current operation behavior is the Nth operation behavior made by the object cumulatively, determining the level of the memory slot in the memory network model which needs to be updated according to the value of N and a preset memory slot update rule of the memory network model, the memory network model being used for recording behavior memory data of the object;

[0008] updating the behavior memory data of the object recorded in the memory slot corresponding to the determined level with the behavior data generated by the current operation behavior.

[0009] In some optional embodiments, the determining the level of the memory slot in the memory network model which needs to be updated according to the value of N and a preset memory slot update rule of the memory network model comprises:

[0010] performing remainder calculation on the update period corresponding to the N, and each level of the memory slot of the memory network model corresponds to a preset update period;

[0011] The level of the memory slot with the remainder calculation result of 0 is determined as the level of the memory slot to be updated.

[0012] The embodiment of the present application provides a subject behavior estimation method, comprising:

[0013] Obtaining behavior memory data of a subject recorded in memory slots at different levels in a memory network model;

[0014] Performing weighted calculation on the behavior memory data of the subject obtained from the memory slots at different levels to obtain long-term behavior memory data of the subject;

[0015] Taking feature data of the display information, static feature data of the subject and the long-term behavior memory data of the subject as inputs of a click estimation model to obtain a behavior probability estimation result of the subject to the display information.

[0016] In some optional embodiments, the weighted calculation on the behavior memory data of the subject obtained from the memory slots at different levels to obtain the long-term behavior memory data of the subject comprises:

[0017] Determining the weight of the memory slot at each level of the memory network model according to a preset energy model;

[0018] Weighting the behavior memory data in the memory slot at each level by using the weight of the memory slot at the level, summing the weighted results of the memory slots at different levels to obtain the long-term behavior memory data of the subject.

[0019] In some optional embodiments, the method further comprises:

[0020] Obtaining short-term behavior memory data representing short-term behavior of the subject from a preset short-term data memory model;

[0021] Performing weighted summation on the long-term behavior memory data and the short-term behavior memory data to obtain long-term behavior memory data of the subject fused with short-term behavior.

[0022] In some optional embodiments, the above method further comprises:

[0023] Obtaining at least one of the following feature data of the display information: display information ID, display information location and display information description information;

[0024] Processing the feature data of the display information by using a preset network model to obtain feature data of the display information for inputting the click estimation model.

[0025] In some optional embodiments, the above method further comprises:

[0026] acquire static feature data of the object, the static feature data being data corresponding to features that do not change due to the object operating the display information;

[0027] process the static feature data using a preset network model to obtain static feature data for inputting into a click estimation model.

[0028] In some optional embodiments, the method further includes:

[0029] acquiring a current operation behavior of the object with respect to the display information;

[0030] when the current operation behavior is an Nth operation behavior made by the object cumulatively, determining a level of a memory slot in a memory network model that needs to be updated according to a value of the N and a memory slot update rule of the memory network model, the memory network model being used to record behavior memory data of the object;

[0031] updating the behavior memory data of the object recorded in the memory slot corresponding to the determined level with behavior data generated by the current operation behavior.

[0032] Embodiments of the present application also provide an object behavior data storage device, including:

[0033] a behavior acquisition module, configured to acquire a current operation behavior of an object with respect to display information;

[0034] an update module, configured to, when the current operation behavior is an Nth operation behavior made by the object cumulatively, determine a level of a memory slot in a memory network model that needs to be updated according to a value of the N and a memory slot update rule of the memory network model, the memory network model being used to record behavior memory data of the object, and update the behavior memory data of the object recorded in the memory slot corresponding to the determined level with behavior data generated by the current operation behavior.

[0035] Embodiments of the present application also provide an object behavior estimation device, including:

[0036] a behavior data acquisition module, configured to acquire behavior memory data of an object to be estimated recorded in memory slots at different levels in a memory network model, and perform weighted calculation on the behavior memory data of the object acquired from the memory slots at different levels to obtain long-term behavior memory data of the object;

[0037] an estimation module, configured to take feature data of display information, static feature data of the object and the long-term behavior memory data of the object as input of a click estimation model to obtain an object behavior probability estimation result of the object with respect to the display information.

[0038] The embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to realize the object behavior data storage method and / or the object behavior estimation method.

[0039] The embodiment of the present application further provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor realizes the object behavior data storage method and / or the object behavior estimation method when executing the program.

[0040] The embodiment of the present application further provides an object behavior estimation system, which comprises a server and a client, wherein:

[0041] The server is provided with the object behavior data storage device, which is used for updating the behavior memory data of the object recorded in the memory network model according to the behavior data of the object provided by the client and directed to the display information.

[0042] The server is provided with the object behavior estimation device, which is used for obtaining the object behavior probability estimation result of the object to the display information according to the behavior memory data of the object recorded in the memory network model, and determining the display information recommended to the object according to the object behavior probability estimation result.

[0043] The client is used for collecting the behavior data of the object when the object operates the display information and sending the behavior data to the server, and recommending the display information to the object according to the display information recommended to the object by the server.

[0044] The embodiment of the present application provides the above technical solutions, and the beneficial effects at least include:

[0045] The operation behavior of the object to the display information is obtained, the memory slot level to be updated is determined according to the current operation behavior and the memory slot update rule of the preset memory network model, the behavior memory data of the object in the memory slot of the corresponding level of the memory network model is updated by using the behavior data generated by the current operation behavior, the behavior memory data in the memory slots of different levels of the memory network model is updated at different frequencies, the data in some memory slots is updated faster, and the data in some memory slots is updated slower, so that the data in different memory slots can reflect the object behavior tendency in different periods.

[0046] In the object behavior estimation for the display information, the long-term behavior memory data of the object is obtained through the weighted calculation based on the behavior memory data of the object recorded in the memory slots of different levels in the memory network model, the long-term behavior memory data for estimation is obtained by comprehensively considering the object behaviors in different periods through different weighting of the memory data reflecting the behavior tendency of the object in different periods, so that the change of the object interest preference with time can be reflected in the object behavior estimation; meanwhile, in the estimation, the static characteristics of the object, the characteristics of the display information and other information are combined, the effect of the collaborative modeling by using the object behavior and the attribute state change of the display information is realized, the heat trend of the display information side and the audience change are considered, the object behavior estimation is better realized, the object behavior probability of the estimation is more accurate, the estimation error rate is significantly reduced, so that the display information more meeting the public demand can be recommended for the object according to the estimation result.

[0047] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by means of the instrumentalities particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0048] The technical solutions of the present application will be further described in detail below by means of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the application. In the drawings:

[0050] Figure 1 The flow chart of the object behavior data storage method in the embodiment one of the present application;

[0051] Figure 2 The flow chart of the object behavior estimation method in the embodiment one of the present application;

[0052] Figure 3 The schematic diagram of the hierarchical architecture of the memory network model in the embodiment two of the present application;

[0053] Figure 4 The flow chart of the object behavior estimation method in the embodiment three of the present application;

[0054] Figure 5 The principle block diagram of the implementation of the object behavior estimation method in the embodiment three of the present application;

[0055] Figure 6 The modeling process example diagram of the object behavior estimation in the embodiment four of the present application;

[0056] Figure 7 A model example diagram for object behavior estimation in the fourth embodiment of the present application;

[0057] Figure 8 A modeling process example diagram for object behavior estimation in the fifth embodiment of the present application;

[0058] Figure 9 A structure schematic diagram of an object behavior estimation system in the embodiment of the present application;

[0059] Figure 10 A structure schematic diagram of an object behavior data storage device in the embodiment of the present application;

[0060] Figure 11 A structure schematic diagram of an object behavior estimation device in the embodiment of the present application. DETAILED DESCRIPTION

[0061] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and so that the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0062] In order to solve the problem of inaccurate or low-accuracy prediction of object future behavior (especially click behavior) in the prior art, an object behavior estimation method is provided in the embodiments of the present application, which is based on a memory network model to model super-long time series data, so as to estimate the probability of object behavior of an object to information before the information is displayed or recommended, and the object behavior can be clicking, collecting, purchasing, etc.

[0063] Embodiment I:

[0064] Embodiment I of the present application provides an object behavior estimation method, which realizes estimation of object behavior probability based on object behavior data stored by a memory network model.

[0065] First, a method for storing object behavior data based on a memory network model is introduced, and the flow thereof is as shown in Figure 1 The method comprises the following steps:

[0066] S11: Obtain a current operation behavior of an object to display information;

[0067] S12: When the current operation behavior is the Nth operation behavior made by the object cumulatively, determine the level of the memory slot to be updated in the memory network model according to the value of N and a preset memory slot update rule of the memory network model, wherein the memory network model is used to record behavior memory data of the object;

[0068] S13: updating the behavior memory data of the object recorded in the memory slot corresponding to the determined level with the behavior data generated by the current operation behavior.

[0069] According to the value of N and the memory slot update rule of the preset memory network model, the level of the memory slot that needs to be updated in the memory network model is determined, including: performing remainder calculation on the update period of each memory slot level with N respectively, and determining the level of the memory slot with a remainder calculation result of 0 as the level of the memory slot that needs to be updated in the memory network model. Wherein, the memory slot of each level of the memory network model corresponds to a preset update period.

[0070] The flow of the object behavior estimation method provided by the embodiment of the application is shown in Figure 2 The flow of the object behavior estimation method provided by the embodiment of the application is shown in

[0071] S21: obtaining the behavior memory data of the object to be estimated recorded in the memory slot of each level in the memory network model;

[0072] S22: performing weighted calculation on the behavior memory data of the object obtained from the memory slot of different levels to obtain the long-term behavior memory data of the object;

[0073] In this case, the weight of the memory slot of each level of the memory network model is determined according to the preset energy model; the behavior memory data in the memory slot of each level is weighted with the weight of the memory slot of each level; and the weighted results of the memory slots of each level are summed to obtain the long-term behavior memory data of the object.

[0074] Optionally, the comprehensive behavior memory data of the object can also be obtained based on the combination of the behavior memory data recorded in the memory slot of each level of the memory network model and the short-term behavior memory data memorized by the short-term network memory model. In this case, the weight of the memory slot of each level of the memory network model is determined according to the preset energy model; the behavior memory data in the memory slot of each level is weighted with the weight of the memory slot of each level; the weighted results of the memory slots of each level are summed to obtain the long-term behavior memory data of the object; the short-term behavior memory data representing the short-term behavior of the object is obtained from the preset short-term data memory model; and the long-term behavior memory data and the short-term behavior memory data are weighted and summed to obtain the long-term behavior memory data of the object fused with the short-term behavior.

[0075] The behavior memory data is obtained based on the behavior memory data that is continuously updated, so it reflects the behavior of the object at different periods and is updated in real time as the behavior of the object occurs.

[0076] S23: input the feature data of the display information, the static feature data of the object, and the long-term behavior memory data of the object into the click estimation model to obtain an object behavior probability estimation result of the object on the display information.

[0077] Before inputting the feature data of the display information, the static feature data of the object, and the long-term behavior memory data of the object into the click estimation model, a process of processing the data can be further included.

[0078] For example, at least one of the following feature data of the display information is obtained: a display information ID, a display information location, and display information description information; and the feature data of the display information is processed using a preset network model to obtain feature data of the display information for inputting into the click estimation model.

[0079] For another example, the static feature data of the object is obtained, wherein the static feature data is data corresponding to a feature that does not change due to the operation of the object on the display information; and the static feature data is processed using a preset network model to obtain static feature data for inputting into the click estimation model.

[0080] The specific implementation process of the above object behavior estimation method and object behavior data storage method will be described in detail below through specific embodiments.

[0081] Embodiment Two

[0082] Embodiment Two of the present application provides a specific memory network model for implementing the above object behavior data storage method, for example, a hierarchical periodic memory neural network (HPMN), as shown in Figure 3 The memory network is a 4-layer network architecture, for example, and can maintain the object behavior data of a single object. The memory network includes 4 memory slots (D=4), and each memory slot can be set with an update period, for example: the jth memory slot corresponding to the jth layer maintains the object behavior data with an update period of t j The update periods of the memory slots of different levels follow an exponential distribution, for example, V1-V8 in the figure represent a complete update period, and V9-V16 represent a complete update period.m j The content maintained in the memory slot is a vector representation of the object behavior feature, and each layer of the memory network maintains the content m j in the specific memory slot, and the behavior memory data output by the jth layer at the ith step corresponds to the vector representation of the behavior memory data obtained after the ith operation behavior of the object. Not only can it be input into the memory slot for storage for subsequent reading or rewriting, but also can be input into the next layer of the current step.

[0083] When the object generates an object behavior at the i-th step, the updating rule of the memory slot can be: if i mod t j is 0, that is, imodt j = 0, the content of the current layer in the current memory slot is updated based on the content in the previous memory slot of the same layer and the content of the next layer in the current memory slot, and if the remainder is not 0, the content of the current layer in the current memory slot is updated with the content in the previous memory slot of the same layer, that is, the content in the memory slot with the remainder not being 0 is not updated. That is, the memory slot content maintained by the j-th layer of the i-th behavior in the memory network satisfies the following formula:

[0084]

[0085] As shown in Figure 3 , the memory updating process from v1 to v16 steps is shown, the content of the first layer is updated at v1 to v16 steps, the content of the second layer needs to be updated at v2, v4, v6, v8, v10, v12, …, v16 steps, the content of the third layer needs to be updated at v4, v8, v12, v16 steps, the content of the fourth layer needs to be updated at v8, v16 steps, and so on.

[0086] In general, the HPMN maintains the content m j of the specific memory slot in each layer, and the behavior memory data of the j-th layer at the i-th behavior is not only stored in the memory slot for subsequent reading or rewriting, but also input into the next layer memory slot of the step. The updating of the memory slot is periodic, and the content of each layer memory slot is updated according to the current behavior step i and the updating period t j of each layer.

[0087] Among them, the updating period of each level of memory slot can be set as needed, and setting different updating periods can make the updating speed of the memory slot different, so that the user behavior memory data recorded in the memory slot with fast updating speed can better reflect the user's short-term behavior, and the data in the memory slot with slow updating speed can better reflect the user's long-term behavior, so that the user's long-term behavior and short-term behavior can be considered when obtaining user behavior data, thereby taking into account the user's behavior characteristics at different periods and better reflecting the change of user preference.

[0088] Embodiment Three

[0089] Embodiment three of the present application provides an object behavior estimation method, which realizes object behavior estimation based on the object behavior memory data recorded in the HPMN model, and the flow is as shown in Figure 4 , and the principle is as shown in Figure 5 , and the method comprises the following steps:​

[0090] Step S101: generating a feature vector of the display information according to feature data of the display information.

[0091] Referring to Figure 5 As shown in the display information related feature part, the feature vector of the display information is obtained based on the display information data. Each display information has some attribute information that is different from other display information, such as the name of the display information, the geographical location where the display information is located, some description of the display information, and the like. Therefore, the feature data of the display information is obtained, including at least one of the following display information features: display information ID, display information location, and display information description information; and the obtained feature data of the display information is processed using a preset network model to obtain the feature vector of the display information.

[0092] Step S102: generating a static feature vector of the object according to static feature data of the object.

[0093] Referring to Figure 5 As shown in the object side feature part, the static feature vector of the object is obtained based on the object static data. Each object has its own static feature to distinguish from other objects, such as some attribute information of the object, device information used, geographical location where the object is located, and the like. That is, the static feature data is the data corresponding to the feature that does not change due to the operation of the object on the display information, and therefore, the static feature data of the object is obtained, including at least one of the following static features: object ID, object location, system version used by the object, and object IP; and the obtained static feature data is processed using a preset network model to obtain the static feature vector of the object.

[0094] Step S103: obtaining behavior memory data of the object recorded in the memory slots of each level in the memory network model; and performing weighted calculation on the behavior memory data of the object obtained from the memory slots of different levels to obtain a long-term behavior feature vector of the object.

[0095] The object behavior data occurring in the network within a period of time can be obtained as needed, such as the time when the object browses or collects a certain display information, browses or clicks a certain advertisement, or purchases a certain display information, and the like. The object behavior of an object can form a serialized object behavior data according to the time sequence of the occurrence of the object behavior. Figure 5 As shown in the object side feature part, the long-term behavior feature vector of the object is obtained based on the object behavior, which is used as comprehensive behavior memory data input into the click estimation model.

[0096] According to the behavior memory data of the object recorded in the memory slots of each level of the memory network model, the acquired behavior memory data of the object is weighted and calculated to obtain long-term behavior memory data of the object, i.e., a long-term behavior feature vector of the object, which can have two different ways:

[0097] Method one: input the object behavior data into the memory network model introduced above for maintenance, and obtain a comprehensive long-term behavior feature vector of the object according to the behavior memory data of the object maintained by the memory network model.

[0098] The object behavior data is processed using the memory network model, and the object behavior memory data in the memory network model is updated when new object behavior occurs, and the object behavior memory data recorded in each memory slot is updated. When the object is estimated, different weights can be given to the object behavior memory data in different memory slots. When estimating, the recorded behavior memory data needs to be read from the memory slots of each level of the memory network model, and the weight value of each level of the memory slot is determined according to the preset energy model. The behavior memory data in the memory slot of the corresponding level is weighted and summed according to the weight value to obtain a long-term behavior feature vector of the object.

[0099] Method two: use a short-term data memory model (such as a basic model) and a memory network model to process short-term and long-term object behavior data respectively, and combine the long-term and short-term object behavior data to obtain a long-term behavior feature vector that fuses short-term behavior.

[0100] The long-term object behavior data in the acquired object behavior data that occurs before the set time is input into the memory network model, and the object behavior memory data in the memory network model is updated when new object behavior occurs, and the object behavior memory data recorded in each memory slot is updated. When the object is estimated, different weights can be given to the object behavior memory data in different memory slots. When estimating, the recorded behavior memory data needs to be read from the memory slots of each level of the memory network model, and the weight value of each level of the memory slot is determined according to the preset energy model. The behavior memory data in the memory slot of the corresponding level is weighted and summed according to the weight value to obtain a long-term behavior memory data of the object;

[0101] The short-term object behavior data in the acquired object behavior data that occurs after the set time is input into the basic data model, and the short-term behavior memory data representing the short-term behavior of the object is obtained from the preset short-term data memory model.

[0102] The long-term behavior memory data and the short-term behavior memory data are weighted and summed to obtain long-term behavior memory data that fuses short-term behavior, i.e., a long-term behavior feature vector.

[0103] The attention mechanism can be used to assign weights to the behavior memory data of the objects in the different levels of memory slots. The reason for using the attention mechanism is that the weights of the object behavior can be adjusted according to the time characteristics of the object behavior. For example, when the long-term object behavior is used for prediction, the behavior memory data that reflects the long-term behavior of the object is given a higher weight. When the short-term object behavior is used for prediction, the behavior memory data that reflects the short-term behavior of the object is given a higher weight. A long-term memory network model can be used to maintain the behavior feature data of a single object. For example, in the hierarchical memory model of the long-term memory network, the memory pool of the memory slot of different levels is used to maintain the object behavior data occurring at different times. When a new object behavior occurs, the content in the memory slot can be updated.

[0104] The execution order of the above steps S101, S102 and S103 is not limited and they can be executed simultaneously.

[0105] Step S104: input the display information feature vector, the static feature vector of the object and the long-term behavior feature vector of the object into the pre-established prediction model to obtain the object behavior probability prediction result of the object to the display information.

[0106] Referring to Figure 5 As shown in the figure, after obtaining the long-term behavior feature vector of the object, the static feature vector of the object and the display information feature vector, the pre-established prediction model is input for prediction to obtain the possibility of the object behavior of the given display information, i.e. the object behavior probability, to realize the object feedback prediction.

[0107] In the above method of the embodiment, when the object behavior prediction is performed on the display information, the long-term behavior memory data of the object is obtained by weighting calculation based on the behavior memory data of the object recorded in the memory slots of different levels in the memory network model. By different weighting of the memory data reflecting the object behavior tendency in different periods, the long-term behavior memory data used for prediction is obtained by considering the short-term behavior of the object and the long-term behavior of the object, and by comprehensively considering the object behavior in different periods, so that the change of the object interest preference with time can be reflected in the object behavior prediction. At the same time, the static features of the object, the features of the display information and other information are combined for prediction, so as to realize the effect of collaborative modeling of the object behavior and the attribute state change of the display information, and to consider the heat trend of the display information side and the change of the audience. The object behavior prediction is better realized, the object behavior probability of the prediction is more accurate, and the prediction error rate is significantly reduced, so that the display information that meets the needs of the public can be recommended to the object according to the prediction result.

[0108] Embodiment Four

[0109] Embodiment four of the present application provides a specific implementation process of the object behavior prediction method, which uses a memory network model to realize feedback prediction of the behavior of the object based on the historical object behavior. The modeling process of the object behavior feedback prediction is as shown in Figure 6

[0110] The main goal of the above object behavior prediction modeling is to capture the hidden object sequence pattern in the object behavior sequence, so a recurrent neural network is used for modeling, as shown in Figure 6 Based on the object behavior data, the memory network model is input, which can be a hierarchical periodic memory network (HPMN) model. In the memory network model, the behavior memory data corresponding to the object recorded in the memory slot of different levels in the memory model is updated according to the number of occurrences of the object behavior. The memory network model can maintain the data of a single object respectively and write the data into the memory unit, which can be a lifelong personalized memory (LSM) unit. The corresponding data can be read from the LSM unit when needed.

[0111] When object behavior prediction is needed, the behavior memory data of the object recorded in the memory slot of each level in the memory network model is read, and the weight of the memory slot of each level is determined according to the preset energy model; the behavior memory data in the memory slot of each level is weighted and summed according to the weight, to obtain the long-term behavior feature vector of the object, as shown in the comprehensive vector (comprehensive representation) in the figure. And form the corresponding query vector (Query Vector) according to the prediction requirement and input it into the HPMN. The prediction requirement can include the target object and the target display information, that is, generate the static feature vector of the object according to the static feature data of the object, and generate the feature vector of the display information according to the feature data of the display information.

[0112] The HPMN reads the corresponding object behavior data according to the prediction requirement, obtains the long-term behavior feature vector of the object according to the read serialized object behavior feature vector, and performs object behavior prediction on the display information according to the prediction requirement and the long-term behavior feature vector of the object, and outputs the probability of the object operating the display information, that is, the comprehensive prediction result, such as the probability of the object A clicking, collecting or purchasing the goods B displayed by the display information.

[0113] Figure 6 In the modeling process shown, considering the rapid growth of the interaction between the object and the display information, the model and the test cannot implement the browsing of the complete object behavior history, so only the latest object memory is maintained, and a real-time incremental updating method is used, that is, the object behavior data is updated through the inference stream and the incremental stream to obtain the growth history, which is used for subsequent prediction and inference.​

[0114] The object behavior data of a single object maintained by HPMN is shown as Figure 3 The process of storing record data is described in Embodiment 2. When prediction is needed, read the memory data to obtain the long-term behavior memory data of the object. The attention mechanism can be used to read the behavior memory data for prediction from different levels of memory slots. For example, after reading the content in each memory slot, the long-term behavior memory data of the object, i.e., the long-term behavior feature vector, can be determined according to the following formula:

[0115]

[0116] where D is the total number of memory slots, j represents the jth memory slot, m j is the content of the last step memory in the memory slot, i.e. at i = T, T is the last one in the object behavior record; w j is the weight of m j , which represents the contribution of each memory slot to the last object behavior feature vector r, where w j can be calculated as follows:

[0117]

[0118] where k represents the kth memory slot, e j = E(m j , v) is an energy model for calculating the relevance of the query vector v and the memory slot m j . The energy function E is a nonlinear multilayer neural network, and the Rule function Relu(x) = max(0, x) can be used as the activation function.

[0119] In the prediction model shown in Figure 7 , according to the query vector of the display information feature, the query vector and the object behavior sequence are combined to obtain the related data of display information 1 (item1), display information 2 (item2), …, display information T (itemT), and then the behavior feature vector of the object (user representation) is obtained. Then, the context features (context features), the display information features (item features) are input into the memory model to obtain the long-term behavior feature vector of the object, the static feature vector of the object and the display information feature vector. Then, the object behavior probability prediction result Pr = (y | u, v, c) is obtained. Where u represents the target object, v represents the target display information, y represents the prediction probability of the object behavior, and c represents the comprehensive feature vector of the object behavior.

[0120] Embodiment five

[0121] Embodiment five of the present application provides another specific implementation process of the object behavior prediction method. The modeling process of object behavior feedback prediction is as shown in Figure 8 The difference between embodiment two is that, in the present embodiment, the long-term object behavior data and the short-term object behavior data are respectively placed in two models for processing.

[0122] As shown in Figure 8 The long-term object behavior data is input into the HPMN model for processing, and the short-term object behavior data is input into a short-term data memory model, such as a base model, for processing. Specifically, the long-term object behavior data in the acquired object behavior data that occurs before the set time is input into the memory network model, and the network model is updated at each level based on the above-described manner, to achieve maintenance of the object behavior data; the short-term object behavior data in the acquired object behavior data that occurs after the set time is input into the base data model for maintenance.

[0123] As shown in Figure 8 The prediction request containing the target object and the target display information and the short-term object behavior data are input into the base model to obtain the object short-term behavior feature vector (Short-term representation) representing the object short-term behavior. The query vector and the long-term object behavior data are input into the HPMN to obtain the serialized object long-term behavior feature vector (long-term representation) representing the object long-term behavior.

[0124] The function of HPMN is similar to that in the above embodiments. HPMN can interact with, write data into, or read data from the memory unit. The specific functions of HPMN are not described here.

[0125] When prediction is needed, the recorded behavior memory data of the object is read from the memory slots at different levels of the memory network model, and weighted summation is performed to obtain the long-term behavior memory data of the object; the short-term behavior memory data representing the short-term behavior of the object is read from the base model; based on the long-term behavior memory data and the long-term behavior memory data, which can both be vector feature form data, the long-term behavior feature vector fused with the short-term behavior is obtained, such as the comprehensive vector (comprehensive representation) shown in the figure; and then the object behavior prediction result is obtained based on the long-term behavior feature vector of the object.

[0126] In the object feedback estimation process in the embodiment, feature extraction is performed on the object static feature information (for example, gender, age, location, etc.), object behavior data, and feature information of the display information, and then the extracted features are put into the model for object feedback estimation and preference ranking of the object to the display information. The long-term historical behavior and the short-term historical behavior of the object are modeled separately for the behavior data of the object. For short-term behavior, a gated recurrent neural network is used for object short-term interest mining and modeling; for longer-term object historical behavior, a hierarchical periodic memory neural network (Hierarchical Periodic Memory Network) is used to mine the long-term and inherent interests of the object, and to model the long-term interest influence chain and the periodic object behavior. The hierarchical periodic memory neural network (Hierarchical Periodic Memory Network) is composed of multiple layers of memory networks. Each layer maintains a memory unit and updates the memory unit at a fixed period to model the multi-period and multi-level interest and behavior characteristics of the object. At the same time, for the display information side, a hierarchical periodic memory network structure is also used for modeling. After modeling the object side and the display information side, the extracted information representation is input into the subsequent neural network for final object feedback estimation.

[0127] The memory network (Memory Network) is a neural network structure for storing patterns of input data. The gated recurrent unit (Gated Recurrent Unit) neural network is a recurrent neural network unit that uses a gating mechanism. The time sequence feature is a feature organized in a time relationship, for example, an object behavior feature organized in a time sequence relationship. Representation refers to the linear or nonlinear mapping of original feature information, and the vector representation of information in a new feature space.

[0128] In addition to using long-term memory networks, the method of the above embodiment can also consider using more complex memory network read-write structures for memory unit maintenance and updating, for example, a neural Turing machine (Neural Turing Machine).

[0129] In the above object behavior estimation method, the object performing the operation can be a user, and the method can be used for user click behavior estimation of display information such as displayed goods or advertisements. In this scenario, the user clicks on the displayed goods or advertisements through the display client of the goods or advertisements, collects user behavior data and provides it to the server side. The memory network model can be set in the server to record the user behavior data, and based on the user behavior data recorded by the memory network model, the probability of the user clicking on the goods or advertisements is estimated.

[0130] The object behavior estimation method can also be used for user click behavior estimation of various display information such as displayed news or videos. In this scenario, the user clicks the displayed news or videos provided by the corresponding news or video provider through the client of the corresponding news or video provider, collects user behavior data and provides the server end, and the corresponding server can set a memory network model to record the user behavior data, and estimate the probability of the user clicking the display information such as news or videos based on the user behavior data recorded by the memory network model.

[0131] Based on the same inventive concept, the embodiments of the present application also provide an object behavior estimation system, the structure of the system is as shown in Figure 9 The system comprises a server 1 and a client 2, wherein:

[0132] The object behavior data storage device is arranged in the server 1, and is used to update the behavior memory data of the object recorded in the memory network model according to the behavior data of the object provided by the client 2 for the display information;

[0133] The object behavior estimation device is arranged in the server 1, and is used to obtain the object behavior probability estimation result of the object for the display information according to the behavior memory data of the object recorded in the memory network model, and determine the display information recommended to the object according to the object behavior probability estimation result;

[0134] The client 2 is used to collect the behavior data of the object when the operation behavior occurs for the display information and send it to the server 1, and recommend the display information to the object according to the display information recommended to the object provided by the server 1.

[0135] The object behavior data storage device can be arranged in the server or the computer device, and the structure of the device is as shown in Figure 10 The device comprises a behavior acquisition module 111 and an update module 112.

[0136] The behavior acquisition module 111 is used to acquire the current behavior of the object for the display information;

[0137] The update module 112 is used to determine the level of the memory slot in the memory network model to be updated when the current operation behavior is the Nth operation behavior made by the object according to the value of N and the memory slot update rule of the preset memory network model, and the memory network model is used to record the behavior memory data of the object; and the behavior data generated by the current operation behavior is used to update the behavior memory data of the object recorded in the memory slot corresponding to the determined level.

[0138] In an optional embodiment, the updating module 112 is configured to determine the level of the memory slot to be updated in the memory network model according to the value of N and a preset memory slot updating rule of the memory network model, including: performing remainder calculation on the updating period respectively by using N, and each level of the memory slot of the memory network model corresponds to a preset updating period; and determining the level of the memory slot with the remainder calculation result of 0 as the level of the memory slot to be updated.

[0139] The object behavior estimation device can be arranged in a server or a computer device, and the structure of the device is as shown in Figure 11 The device includes an object behavior data acquisition module 121 and an estimation module 122.

[0140] The object behavior data acquisition module 121 is configured to acquire behavior memory data of an object to be estimated recorded in the memory slots of each level of the memory network model; and perform weighted calculation on the behavior memory data of the object acquired from the memory slots of different levels to obtain long-term behavior memory data of the object.

[0141] The estimation module 122 is configured to take the feature data of the display information, the static feature data of the object and the long-term behavior memory data of the object as inputs of a click estimation model to obtain an object behavior probability estimation result of the object on the display information.

[0142] Optionally, the object behavior data acquisition module 121 is configured to perform weighted calculation on the behavior memory data of the object acquired from the memory slots of different levels to obtain the long-term behavior memory data of the object, including:

[0143] determining the weight of the memory slot of each level of the memory network model according to a preset energy model;

[0144] weighting the behavior memory data in the memory slot of each level by using the weight of the memory slot of the level, and summing the weighted results of the memory slots of all levels to obtain the long-term behavior memory data of the object.

[0145] Optionally, the object behavior data acquisition module 121 is configured to perform weighted calculation on the behavior memory data of the object acquired from the memory slots of different levels to obtain the long-term behavior memory data of the object, including:

[0146] determining the weight of the memory slot of each level of the memory network model according to a preset energy model;

[0147] weighting the behavior memory data in the memory slot of each level by using the weight of the memory slot of the level, and summing the weighted results of the memory slots of all levels to obtain the long-term behavior memory data of the object.

[0148] acquire short-term behavior memory data representing short-term behavior of the object from a preset short-term data memory model;

[0149] perform weighted summation on the long-term behavior memory data and the short-term behavior memory data to obtain long-term behavior memory data of the object fused with short-term behavior.

[0150] Optionally, the apparatus further comprises a display information feature acquisition module 123 configured to acquire at least one of the following feature data of the display information: display information ID, display information location and display information description; and process the feature data of the display information using a preset network model to obtain feature data of the display information for inputting into the click estimation model.

[0151] Optionally, the apparatus further comprises an object feature acquisition module 124 configured to acquire static feature data of the object, wherein the static feature data is data corresponding to features that do not change due to the operation of the object on the display information; and process the static feature data using a preset network model to obtain static feature data for inputting into the click estimation model.

[0152] Optionally, the apparatus further comprises an object behavior data storage module 125 configured to implement the function of the object behavior data storage apparatus and configured to:

[0153] acquire a current operation behavior of the object on the display information;

[0154] when the current operation behavior is the Nth operation behavior made by the object, determine a level of a memory slot in the memory network model that needs to be updated according to the value of N and a memory slot update rule of the preset memory network model, the memory network model being configured to record behavior memory data of the object;

[0155] update the behavior memory data of the object recorded in the memory slot corresponding to the determined level with behavior data generated by the current operation behavior.

[0156] The embodiments of the present application further provide a computer readable storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the object behavior data storage method and / or the object behavior estimation method.

[0157] The embodiments of the present application further provide a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the object behavior data storage method and / or the object behavior estimation method when executing the program.

[0158] As to the system and apparatus in the above embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0159] The above method, system and device of the embodiment of the present application, based on hierarchical periodic memory network's super long time series data modeling, realizes the object feedback estimation of target display information and target object, estimates the behavior probability of object for given display information, such as click rate, purchase rate, etc. It can better consider the object behavior in different periods, especially in the case that short-term object behavior data cannot obtain accurate estimation result, long-term object behavior data can be used, the implicit object nature preference can be extracted from these historical early behavior sequence, even the periodic behavior information, so as to obtain better estimation result.

[0160] The above method can effectively and individually model the long-term and short-term behavior characteristics of the object, depict the diverse interests of the object, and at the same time, take into account the heat trend of the display information side and the change of the audience, better estimate the object feedback. In the experiment of the offline data set, the accuracy of the object feedback estimation is effectively improved, and the estimation error rate is greatly reduced.

[0161] Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, and the like, can refer to an action and / or process of one or more processing or computing systems, or similar devices, that manipulate and / or transform data represented as physical (e.g., electronic) quantities within the processing system's registers and / or memories into other data similarly represented as physical quantities within the processing system's memories, registers or other such information storage, transmission or display devices. Information and signals can be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0162] It should be borne in mind, that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary methods. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes can be re-arranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in exemplary order and are not intended to be limited to the specific order or hierarchy presented.

[0163] In the detailed description above, various features are grouped together in single embodiments for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting a necessity to disclose features in any single patent. Rather, according to the inventive concept, features can be combined in any single patent in one or more claims. Thus, the disclosure hereof is to be understood as being illustrative of the inventive concept and not a limitation thereof. For example, not every aspect of the creative process is described with every embodiment. It is contemplated that the creative process is a dynamic process that will necessitate implementation of new techniques by those skilled in the art. Those skilled in the art will appreciate that, in the development of this creative process, numerous implementation-specific decisions can be made. These implementation-specific decisions can vary from one implementation to another, and from one environment to another. Those skilled in the art will appreciate that such a development effort might be time-consuming, but that, otherwise, such efforts would not be a contribution to the art of the present disclosure, and would form no part of this disclosure. In this regard, details are not provided hereinafter in order to not unnecessarily obscure the disclosure. Other steps and methods can vary from one implementation to another in accordance with the inventive concept.

[0164] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0165] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal. The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other variations without departing from the scope of the disclosure. For example, one of ordinary skill in the art will understand that the disclosure can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques.

[0166] For a software implementation, the techniques described herein can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes can be stored in memory units and executed by processors. The memory unit can be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.

[0167] The above description includes examples of one or more embodiments. Of course, not all possible combinations of components or methods described above can be claimed as an embodiment, but one of ordinary skill in the art will recognize that many such further combinations and permutations of the embodiments described are possible. Accordingly, the described embodiments are intended to embrace all such alterations, modifications and variations which fall within the scope of the appended claims. Additionally, where the description or the claims recite "a", "an" or a "the" one or more of elements, this does not exclude multiple numbers for this element. Further, where the description or the claims recite that an element can or can not be used, this indicates that the element is optional. Additionally, the use of "a" or "an" or "the" preceding an element does not exclude the presence of more than one of the element. Further, the description or the claims can include a plurality of means plus function components consisting of a specific combination of means. The method or process of claims can be implemented by one or more processors or one or more digital signal processors.

Claims

1. A method for predicting object behavior, characterized in that, include: The system acquires behavioral memory data of the object to be estimated, recorded in memory slots at each level of the memory network model. Specifically, it acquires short-term behavioral memory data of the object occurring after a set time from the base model, and behavioral memory data of the object occurring before a set time from memory slots at different levels of the hierarchical periodic memory network (HPMN). The behavioral memory data recorded in each level's memory slots is updated using behavioral data generated by the object's current operation on the displayed information. When the current operation is the Nth operation performed by the object, the level of the memory slot to be updated in the memory network model is determined based on the value of N and the preset memory slot update rules. If the remainder of N divided by the update period is 0, the content of the current layer in the current memory slot is updated based on the content of the previous memory slot in the same layer and the content of the next layer in the current memory slot. The long-term behavioral memory data of the object is obtained by weighting the short-term behavioral memory data together with the behavioral memory data obtained from the memory slots of different levels of HPMN. Obtain the feature data and static feature data of the object's display information; the static feature data corresponds to features that do not change due to the object's operation on the display information; The feature data of the displayed information, the static feature data of the object, and the long-term behavioral memory data of the object are used as inputs to the click prediction model to obtain the prediction result of the object's behavior probability towards the displayed information.

2. The method as described in claim 1, characterized in that, The step of weighting and calculating the behavioral memory data of the object obtained from memory slots at different levels to obtain the long-term behavioral memory data of the object includes: The weights of memory slots at each level of the memory network model are determined based on the preset energy model. The behavioral memory data in the memory slots of each level are weighted by the weight of the memory slots at that level, and the weighted results of the memory slots at each level are summed to obtain the long-term behavioral memory data of the object.

3. The method as described in claim 2, characterized in that, The method further includes: Short-term behavior memory data characterizing the short-term behavior of the object is obtained from a preset short-term data memory model; The long-term behavioral memory data and the short-term behavioral memory data are weighted and summed to obtain the long-term behavioral memory data of the object that incorporates short-term behavior.

4. The method as described in claim 1, characterized in that, Also includes: Obtain at least one of the following characteristic data of the displayed information: displayed information ID, location of the displayed information, and description of the displayed information; The feature data of the displayed information is processed using a preset network model to obtain feature data of the displayed information used as input to the click prediction model.

5. The method as described in claim 1, characterized in that, Also includes: The static feature data is processed using a preset network model to obtain static feature data for input click prediction model.

6. The method according to any one of claims 1-5, characterized in that, Also includes: Get the current action of the object regarding the displayed information; When the current operation is the Nth operation performed by the object, the level of the memory slot to be updated in the memory network model is determined according to the value of N and the memory slot update rule of the preset memory network model. If the result of N modulo the update period is 0, the content of the current layer in the current memory slot is updated based on the content in the previous memory slot of the same layer and the content of the next layer in the current memory slot. If the result is not 0, the content of the current layer in the current memory slot is updated with the content in the previous memory slot of the same layer. The memory network model is used to record the behavioral memory data of the object; The behavior memory data of the object recorded in the memory slot corresponding to the determined level is updated with the behavior data generated by the current operation.

7. The method as described in claim 6, characterized in that, The step of determining the level of memory slots that need to be updated in the memory network model based on the value of N and the preset memory slot update rules of the memory network model includes: The N pairs of update cycles are moduloed, and each memory slot in the memory network model corresponds to a preset update cycle. The level of the memory slot with a remainder of 0 is determined as the level of the memory slot that needs to be updated.

8. An object behavior prediction device, characterized in that, include: The behavior data acquisition module is used to acquire behavior memory data of the object to be estimated recorded in the memory slots of each level in the memory network model. Specifically, it acquires short-term behavior memory data of the object after a set time from the basic model, and behavior memory data of the object before a set time from the memory slots of different levels of the hierarchical periodic memory network (HPMN). The behavior memory data recorded in the memory slots of each level is updated using the behavior data generated by the object's current operation on the displayed information. Furthermore, when the current operation is the Nth cumulative operation performed by the object, the data is updated based on the value of N and a preset memory... The memory slot update rules of the network model determine the level of the memory slots that need to be updated in the memory network model. If the remainder of N with respect to the update period is 0, the content of the current layer in the current memory slot is updated based on the content of the previous memory slot in the same layer and the content of the next layer in the current memory slot. The system also calculates the long-term behavioral memory data of the object by weighting the short-term behavioral memory data obtained from the memory slots of different levels of HPMN. Furthermore, it acquires the feature data and static feature data of the object's display information. The static feature data corresponds to features that do not change due to the object's operations on the display information. The prediction module is used to take the feature data of the displayed information, the static feature data of the object, and the long-term behavioral memory data of the object as inputs to the click prediction model to obtain the object behavior probability prediction result of the object to the displayed information.

9. The apparatus as claimed in claim 8, characterized in that, include: The behavior acquisition module is used to acquire the current operation behavior of the object in relation to the displayed information; The update module is used to determine the level of the memory slot that needs to be updated in the memory network model when the current operation is the Nth operation cumulatively performed by the object, based on the value of N and the memory slot update rules of the preset memory network model. The memory network model is used to record the behavior memory data of the object. The behavior memory data of the object recorded in the memory slot corresponding to the determined level is updated with the behavior data generated by the current operation.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the object behavior prediction method as described in any one of claims 1-7.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the object behavior prediction method as described in any one of claims 1-7.

12. An object behavior prediction system, characterized in that, include: Server and client, where: The server is equipped with an object behavior prediction device as described in claim 8 or 9, used to update the object behavior memory data recorded in the memory network model according to the object behavior data provided by the client for the displayed information; and to obtain the object behavior probability prediction result of the object for the displayed information according to the object behavior memory data recorded in the memory network model, and to determine the displayed information recommended to the object according to the object behavior probability prediction result. The client is used to collect behavioral data of objects when they perform operations on the displayed information and send it to the server, and to recommend display information to objects based on the display information recommended by the server.

Images