A method, device, equipment and storage medium for pushing interest

Abstract

The present application provides an interest push method, device, equipment and storage medium. The method includes: obtaining multiple user behavior sequences; inputting multiple user behavior sequences into a multi-interest module for feature extraction to obtain interest features corresponding to each user behavior sequence; performing similarity calculation between preset interest classification features and interest features corresponding to each user behavior sequence to obtain target interest features corresponding to each user behavior sequence; inputting the target interest features corresponding to each user behavior sequence into an interest hierarchy module for analysis based on a self-attention mechanism to obtain estimated interest features corresponding to each user behavior sequence; determining interest estimation values ​​according to the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of recommendation systems, and in particular to an interest push method, device, equipment and storage medium. Background Art

[0002] The recommendation system is based on the interaction between users and content. Through recall, rough sorting, and fine sorting, the recommendation system selects the content that users currently like from a rich content pool. Usually, in the recall stage, it is necessary to select thousands of content that users may be interested in from millions or even hundreds of millions of candidate content. In the rough sorting stage, hundreds of candidate content are quickly selected. Finally, in the fine sorting stage, the user's interests are accurately estimated to produce a few of the content that users are most interested in. The fine sorting stage is usually the last link of the recommendation, so it is particularly important.

[0003] Currently, most of the sorting links are targeted at a single interest. Using only one relevant interest will cause information loss, and the use of multiple behavior sequences does not explicitly show differences. Summary of the invention

[0004] The present application provides an interest push method, apparatus, device and storage medium to improve the accuracy of interest push.

[0005] In a first aspect, the present application provides an interest push method, wherein the method is applied to an electronic device, the electronic device includes a multi-layer neural network model, the multi-layer neural network model includes at least: a multi-interest module and an interest hierarchy module; the method includes: obtaining multiple user behavior sequences; inputting the multiple user behavior sequences into the multi-interest module for feature extraction to obtain interest features corresponding to each user behavior sequence; performing similarity calculation on preset interest classification features and interest features corresponding to each user behavior sequence to obtain target interest features corresponding to each user behavior sequence; inputting the target interest features corresponding to each user behavior sequence into the interest hierarchy module for analysis based on the self-attention mechanism to obtain estimated interest features corresponding to each user behavior sequence; determining an interest estimation value based on the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence, wherein the interest estimation value is used to represent the content of interest to the user.

[0006] In a possible implementation, multiple user behavior sequences are input into a multi-interest module for feature extraction to obtain interest features corresponding to each user behavior sequence, including: analyzing multiple user behavior sequences based on a self-attention mechanism to obtain interest features corresponding to each user behavior sequence.

[0007] In a possible implementation, multiple user behavior sequences are analyzed based on a self-attention mechanism to obtain interest features corresponding to each user behavior sequence, including: obtaining three data expression features corresponding to each user behavior sequence, the three data expression features including a first expression feature, a second expression feature, and a third expression feature that are uniformly used for the user behavior sequence after different transformations; obtaining an attention weight corresponding to each user behavior sequence by calculating the similarity between the first expression feature and the second expression feature; and multiplying the attention weight by the third expression feature to obtain an interest feature corresponding to each user behavior sequence.

[0008] In a possible implementation, a similarity calculation is performed between a preset interest classification feature and an interest feature corresponding to each user behavior sequence to obtain a target interest feature corresponding to each user behavior sequence, including: cross-calculating the preset interest classification feature and the interest feature corresponding to each user behavior sequence to obtain a target interest feature corresponding to each user behavior sequence.

[0009] In a possible implementation, the target interest features corresponding to each user behavior sequence are input into the interest hierarchy module for analysis based on a self-attention mechanism to obtain estimated interest features corresponding to each user behavior sequence, including: obtaining a set of position embedding features corresponding to each user behavior sequence according to each user behavior sequence; adding a set of position embedding features corresponding to each user behavior sequence to the target interest features corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence, wherein the interest sequence corresponding to each user behavior sequence is composed of the target interest features corresponding to each user behavior sequence; and performing analysis based on a self-attention mechanism on the interest sequence corresponding to each user behavior sequence to obtain estimated interest features corresponding to each user behavior sequence.

[0010] In one possible implementation, a self-attention-based analysis is performed on the interest sequence corresponding to each user behavior sequence to obtain an estimated interest feature corresponding to each user behavior sequence, including: obtaining three data expression features corresponding to the interest sequence corresponding to each user behavior sequence; the three data expression features include a fourth expression feature, a fifth expression feature, and a sixth expression feature that are uniformly used for the interest sequence after different transformations; obtaining an attention weight corresponding to each interest sequence by calculating the similarity between the fourth expression feature and the fifth expression feature; and multiplying the attention weight by the sixth expression feature to obtain an estimated interest feature corresponding to each user behavior sequence.

[0011] In a possible implementation, the multi-layer neural network model also includes: a multi-layer perceptron (MLP) module; determining an interest estimation value based on the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence, including: inputting the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to determine the interest estimation value.

[0012] In a possible implementation, after acquiring the multiple user behavior sequences, the method further includes: embedding and transforming the multiple user behavior sequences.

[0013] In the second aspect, the present application provides an interest push device, which can be a chip or system on chip in a computer, or a functional module in a computer for implementing the method described in the first aspect and any possible implementation thereof. The interest push device can implement the functions performed by the computer described in the first aspect and any possible implementation thereof, and the functions can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The interest push device includes: an acquisition module, which is used to acquire multiple user behavior sequences; an extraction module, which is used to input the multiple user behavior sequences into a multi-interest module for feature extraction, so as to obtain interest features corresponding to each user behavior sequence; a similarity calculation module, which is used to calculate the similarity between a preset interest classification feature and the interest feature corresponding to each user behavior sequence, so as to obtain a target interest feature corresponding to each user behavior sequence, wherein the target interest feature is the interest feature with the highest similarity to the interest classification feature among the interest features corresponding to each user behavior sequence; an analysis module, which is used to input the target interest feature corresponding to each user behavior sequence into an interest hierarchy module for analysis based on a self-attention mechanism, so as to obtain an estimated interest feature corresponding to each user behavior sequence; a determination module, which is used to determine an interest estimation value according to the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence, wherein the interest estimation value is used to represent the content of interest to the user.

[0014] In a possible implementation, the extraction module is specifically used to: analyze multiple user behavior sequences based on a self-attention mechanism to obtain interest features corresponding to each user behavior sequence.

[0015] In one possible implementation, the extraction module is specifically used to: obtain three data expression features corresponding to each user behavior sequence, the three data expression features including a first expression feature, a second expression feature, and a third expression feature that are uniformly used for the user behavior sequence after different transformations; obtain an attention weight corresponding to each user behavior sequence by calculating the similarity between the first expression feature and the second expression feature; and multiply the attention weight by the third expression feature to obtain an interest feature corresponding to each user behavior sequence.

[0016] In a possible implementation, the similarity calculation module is specifically used to: cross-calculate the preset interest classification feature with the interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence.

[0017] In one possible implementation, the analysis module is specifically used to: obtain a set of position embedding features corresponding to each user behavior sequence according to each user behavior sequence; add a set of position embedding features corresponding to each user behavior sequence and a target interest feature corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence, wherein the interest sequence corresponding to each user behavior sequence is composed of a target interest feature corresponding to a user behavior sequence; and perform a self-attention mechanism-based analysis on the interest sequence corresponding to each user behavior sequence to obtain an estimated interest feature corresponding to each user behavior sequence.

[0018] In one possible implementation, the analysis module is specifically used to: obtain three data expression features corresponding to the interest sequence corresponding to each user behavior sequence; the three data expression features include a fourth expression feature, a fifth expression feature, and a sixth expression feature that are uniformly used for the interest sequence after different transformations; obtain an attention weight corresponding to each interest sequence by calculating the similarity between the fourth expression feature and the fifth expression feature; multiply the attention weight by the sixth expression feature to obtain an estimated interest feature corresponding to each user behavior sequence.

[0019] In a possible implementation, the determination module is specifically used to: input the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to determine the interest estimation value.

[0020] In a possible implementation, the acquisition module is used to: embed and transform multiple user behavior sequences.

[0021] In a third aspect, the present application provides an electronic device, comprising: a memory for storing processor executable instructions; a processor; wherein the processor is configured to: when executing the executable instructions, implement the method described in the first aspect and any possible implementation manner thereof.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. After the computer-executable instructions are executed by a processor, they can implement the method described in the first aspect and any possible implementation manner thereof.

[0023] The technical solution provided by this application may have the following beneficial effects:

[0024] In this application, by introducing multiple interest features, more useful information can be brought, which is conducive to the multi-layer neural network model to learn the dependencies between different interests, improve the prediction ability and robustness of the multi-layer neural network model, and improve the accuracy of pushed content; hierarchical processing of multiple user behavior sequences can maximize the use of hierarchical information of multiple user behavior sequences, observe the importance of different user behavior sequences, and guide the iteration of business goals.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and do not limit the scope of protection of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] Figure 1 A schematic diagram of the structure of a multi-layer neural network model in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of an implementation process of the interest push method in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of a multi-interest module in an embodiment of the present application;

[0030] Figure 4 This is another implementation flow diagram of the interest push method in the embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of the structure of an interest level module in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of the structure of an interest push device in an embodiment of the present application;

[0033] Figure 7 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0035] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0036] The recommendation system is based on the interaction between users and content. Through recall, rough sorting, fine sorting and other steps, it selects the content that users currently like from a rich content pool. Among them, the sorting stage is particularly important.

[0037] The current sorting methods still have the following problems in the application of user behavior sequences:

[0038] (1) Most of the features used in the sorting process are single interest features. Since the user's behavior sequence must be multi-interest, using only one relevant interest feature will cause information loss. Moreover, as the output feature of the user behavior sequence, when a single interest feature and other interest features are input into the multi-layer neural network model together, the interest feature is not enough to reflect the importance of the user sequence.

[0039] (2) The use of multiple user behavior sequences does not explicitly show differences. In business scenarios, the importance of different user behavior sequences is different. For example, in short video scenarios, the click sequence, the completion sequence, the like sequence, and the sharing sequence show the different levels of interest of a user. Using the same processing method for these user behavior sequences does not reflect the differences in the importance of different user behavior sequences.

[0040] It can be seen that the current sorting method has the limitation of a single interest feature and fails to reflect the differences in the behavior sequences of different users.

[0041] In order to solve the above problems, an embodiment of the present application provides an interest push method, which can be applied to an electronic device including a multi-layer neural network model to achieve more accurate learning of the user's various interests and push more interesting content to the user.

[0042] It should be noted that the application scenarios of the interest push method of the present application may include but are not limited to: news push, short video push, interested product push, etc.

[0043] So, Figure 1 This is a schematic diagram of the structure of a multi-layer neural network model in the embodiment of the present application, see Figure 1As shown by the solid line in the middle, the multi-layer neural network model 100 includes: a multi-interest module 101, which is used to extract interest features from multiple input user behavior sequences to output interest features corresponding to each user behavior sequence; an interest classification module 102, which is used to pair preset interest classification features with interest features corresponding to each input user behavior sequence to output target interest features corresponding to each user behavior sequence; an interest hierarchy module 103, which is used to analyze the target interest features corresponding to each input user behavior sequence based on the self-attention mechanism to output estimated interest features corresponding to each user behavior sequence; and an MLP module 104, which is used to output interest estimation values ​​based on the estimated interest features corresponding to each input user behavior sequence and the interest features corresponding to each user behavior sequence.

[0044] Still see Figure 1 As shown by the dotted line, the multi-layer neural network model 100 may also include: an embedding conversion module 105, which is used to embed and convert multiple input user behavior sequences to obtain converted user behavior sequences and input them into the interest module 101.

[0045] In some possible embodiments, the input features of the multi-layer neural network model 100 may also include: user features, classification features, etc. These features can be directly input into the MLP module 104, and the MLP module 104 outputs an interest estimation value based on these features and the estimated interest features corresponding to each input user behavior sequence and the interest features corresponding to each user behavior sequence.

[0046] Accordingly, the above multi-layer neural network model can be applied to an interest push method. Figure 2 This is a schematic diagram of an implementation flow of the interest push method in the embodiment of the present application, see Figure 2 As shown, the interest push method may include:

[0047] S201, obtaining multiple user behavior sequences.

[0048] It is understandable that before the electronic device pushes the content of interest (such as news, short videos, commodities, etc.) to the user, it first obtains the user's historical behavior stored in the client. The user's historical behavior stored in the client may include multiple user behavior sequences, such as: click sequence, like sequence, comment sequence, share sequence, etc. Then, the electronic device obtains multiple user behavior sequences from the user's historical behavior.

[0049] S202, inputting multiple user behavior sequences into a multi-interest module for feature extraction to obtain interest features corresponding to each user behavior sequence.

[0050] It can be understood that after the electronic device obtains multiple user behavior sequences through S201, it inputs the multiple user behavior sequences into the multi-interest module for feature extraction, thereby obtaining the interest features corresponding to each user behavior sequence.

[0051] In some possible implementations, S202 may include: the electronic device analyzes multiple user behavior sequences based on a self-attention mechanism to obtain interest features corresponding to each user behavior sequence.

[0052] In some possible implementations, the above-mentioned electronic device performs analysis of multiple user behavior sequences based on a self-attention mechanism, which may include: the electronic device obtains three data expression features corresponding to each user behavior sequence, the three data expression features include a first expression feature, a second expression feature, and a third expression feature that are uniformly used for the user behavior sequence after different transformations; the electronic device obtains the attention weight corresponding to each user behavior sequence by calculating the similarity between the first expression feature and the second expression feature; and multiplies the attention weight by the third expression feature to obtain the interest feature corresponding to each user behavior sequence.

[0053] Exemplary, combined Figure 3 The above S202 is described. Figure 3 FIG. 1 is a schematic diagram of the structure of a multi-interest module in an embodiment of the present application. Figure 3 As shown, after obtaining multiple user behavior sequences, the electronic device inputs the multiple user behavior sequences into the multi-interest module 300 for feature extraction, and performs analysis on each user behavior sequence based on the self-attention mechanism. The analysis based on the self-attention mechanism may include: outputting three sets of parameters for each user behavior sequence and multiplying the user behavior sequence to obtain data expressions (i.e., data expression features) of three sets of user behavior sequences, Q1, K1, and V1 (query, key, value). Among them, Q1, K1, and V1 are respectively recorded as the first expression feature, the second expression feature, and the third expression feature. Furthermore, the electronic device inputs Q1 and K1 into the first matrix multiplication unit 301 for multiplication (i.e., calculating the similarity between the first expression feature and the second expression feature) to obtain the weights of different behaviors in the user behavior sequence (i.e., the attention weight corresponding to each user behavior sequence), normalizes the weights of different behaviors in the user behavior sequence through the first normalized exponential function unit 302, and then inputs the weights of different behaviors in each user behavior sequence and V1 into the second matrix multiplication unit 303 for multiplication to obtain the final expression of the user behavior sequence, i.e., the interest feature corresponding to each user behavior sequence.

[0054] It should be noted that the multi-layer neural network model includes one or more multi-interest modules, which are used to extract different interest features.

[0055] S203, calculating the similarity between the preset interest classification feature and the interest feature corresponding to each user behavior sequence, and obtaining the target interest feature corresponding to each user behavior sequence.

[0056] Among them, the target interest feature is the interest feature with the highest similarity to the interest classification feature among the interest features corresponding to each user behavior sequence.

[0057] It can be understood that after S202, the electronic device calculates the similarity between the preset interest classification feature and the target interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence.

[0058] In some possible implementations, S203 may include: the electronic device cross-calculates the preset interest classification feature with the interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence.

[0059] Exemplarily, after obtaining the interest features corresponding to each user behavior sequence, the electronic device cross-calculates the interest features corresponding to each user behavior sequence with the preset interest classification features, and performs an inner product to find the interest features that are most similar to the preset interest classification features, that is, the target interest features corresponding to each user behavior sequence.

[0060] S204, inputting the target interest features corresponding to each user behavior sequence into the interest hierarchy module for analysis based on the self-attention mechanism to obtain the estimated interest features corresponding to each user behavior sequence.

[0061] It can be understood that after S103, the electronic device inputs the target interest features corresponding to each user behavior sequence into the interest hierarchy module, and performs analysis based on the self-attention mechanism to obtain the estimated interest features corresponding to each user behavior sequence.

[0062] In some possible implementations, in order to improve the accuracy of pushing content, the above S204 may also include S401 to S405. Specifically, Figure 4 This is another implementation flow diagram of the interest push method in the embodiment of the present application, see Figure 4 As shown, the interest push method may include:

[0063] S401, according to each user behavior sequence, obtain a set of position embedding features corresponding to each user behavior sequence.

[0064] It can be understood that the electronic device generates a set of specific position embedding features for each user behavior sequence, that is, a set of position embedding features corresponding to each user behavior sequence.

[0065] S402, adding a group of position embedding features corresponding to each user behavior sequence and a target interest feature corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence.

[0066] Among them, the interest sequence corresponding to each user behavior sequence is composed of the target interest features corresponding to each user behavior sequence.

[0067] It can be understood that the electronic device adds the position embedding feature and the target interest feature corresponding to the corresponding user behavior sequence. Then, the interest push device combines the target interest feature corresponding to each user behavior sequence into an interest sequence with a structure consistent with the user behavior sequence.

[0068] In some possible implementations, the electronic device obtains a set of position embedding features corresponding to each user behavior sequence based on each user behavior sequence, and then adds the set of position embedding features corresponding to each user behavior sequence to the target interest features corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence.

[0069] Exemplarily, the electronic device inputs the target interest features corresponding to each user behavior sequence into the interest hierarchy module. The electronic device generates a set of specific position embedding features for each user behavior sequence, that is, a set of position embedding features corresponding to each user behavior sequence. Further, the electronic device adds the above position embedding features and the target interest features corresponding to the corresponding user behavior sequence. Then, the electronic device combines the target interest features corresponding to each user behavior sequence into an interest sequence with a structure consistent with the user behavior sequence, that is, an interest sequence corresponding to each user behavior sequence.

[0070] S403, the electronic device obtains the fourth expression feature, the fifth expression feature and the sixth expression feature corresponding to the interest sequence corresponding to each user behavior sequence.

[0071] It can be understood that the electronic device obtains three data expression features corresponding to the interest sequence corresponding to each user behavior sequence, wherein the three data expression features include a fourth expression feature, a fifth expression feature and a sixth expression feature uniformly obtained after different transformations of the interest sequence.

[0072] S404, obtaining the attention weight corresponding to each interest sequence by calculating the similarity between the fourth expression feature and the fifth expression feature.

[0073] It can be understood that the electronic device multiplies the fourth expression feature and the fifth expression feature, calculates the similarity between the fourth expression feature and the fifth expression feature, and obtains the attention weight corresponding to each interest sequence.

[0074] S405, multiplying the attention weight by the sixth expression feature to obtain an estimated interest feature corresponding to each user behavior sequence.

[0075] It can be understood that after the electronic device obtains the attention weight corresponding to each interest sequence, it multiplies the attention weight with the sixth expression feature to obtain the estimated interest feature corresponding to each user behavior sequence.

[0076] In some possible implementations, after the electronic device obtains the interest sequence corresponding to each user behavior sequence, it performs analysis based on the self-attention mechanism on the interest sequence corresponding to each user behavior sequence.

[0077] Exemplary, combined Figure 5 The above S401 to S405 are explained. Figure 5 A schematic diagram of the structure of an interest level module in an embodiment of the present application. Figure 5 As shown, the electronic device inputs the target interest feature corresponding to each user behavior sequence into the interest level module 500 for analysis based on the self-attention mechanism. Specifically, it includes: the electronic device inputs each user behavior sequence into the position embedding unit 501 to obtain a set of position embedding features corresponding to each user behavior sequence. Then a set of position embedding features corresponding to each user behavior sequence is input into the interest sequence generation unit 502 to obtain the interest sequence corresponding to each user behavior sequence. After obtaining the interest sequence corresponding to each user behavior sequence, three sets of parameters are respectively output for the interest sequence corresponding to each user behavior sequence and multiplied by the above interest sequence to obtain the data expression of three sets of interest behavior sequences Q2, K2, and V2, wherein Q2, K2, and V2 are respectively recorded as the fourth expression feature, the fifth expression feature, and the sixth expression feature. Further, the electronic device multiplies Q2 and K2 by the third matrix multiplication unit 503 (i.e., calculates the similarity between the fourth expression feature and the fifth expression feature) to obtain the weights of different behaviors in the interest sequence (i.e., the attention weight corresponding to each interest sequence). The weights of different behaviors in the interest sequence are normalized by the second normalized exponential function unit 504. The weights of different behaviors in each of the above interest sequences are then multiplied by V2 and input into the fourth matrix multiplication unit 505 to obtain the final expression of the interest sequence, which is the estimated interest feature corresponding to each user behavior sequence.

[0078] S205, determining an interest estimation value according to the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence.

[0079] The interest estimation value is used to indicate the content that the user is interested in.

[0080] It can be understood that the electronic device determines the content that the user is interested in, that is, the interest estimation value, based on the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence.

[0081] In some possible implementations, the multi-layer neural network model may further include an MLP module; S205 may include: the electronic device inputs the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to determine the interest estimation value.

[0082] Exemplarily, the electronic device inputs the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to obtain an interest estimation value, wherein the interest estimation value indicates the user's interest in the content and can be represented by a probability value between 0 and 1, and the closer to 1, the higher the probability of the user clicking.

[0083] Furthermore, the input features of the multi-layer neural network model 100 may also include: user features, classification features, etc. These features can be directly input into the MLP module 104, and the MLP 104 estimates the user's interest based on these features and the estimated interest features corresponding to each input user behavior sequence and the interest features corresponding to each user behavior sequence, so as to output an interest estimation value.

[0084] In some possible implementations, after the electronic device obtains multiple user behavior sequences, the interest push method may further include: embedding and converting the multiple user behavior sequences.

[0085] It should be understood that multiple user behavior sequences can be input into the embedding conversion module 105 , and the embedding conversion module 105 converts multiple user behavior features into embedding features and outputs them to the multi-interest module 101 and the MLP module 104 .

[0086] At this point, the content of interest to users is pushed.

[0087] It can be seen that by introducing multiple interest features, more useful information can be obtained, which is conducive to the multi-layer neural network model to learn the dependencies between different interests, improve the prediction ability and robustness of the multi-layer neural network model, and improve the accuracy of pushed content; hierarchical processing of multiple user behavior sequences can maximize the use of hierarchical information of multiple user behavior sequences, observe the importance of different user behavior sequences, and guide the iteration of business goals.

[0088] Based on the same inventive concept, the embodiment of the present application also provides an interest push device, which can be a chip or system on chip in a computer, or a functional module in a computer for the method described in one or more of the above embodiments. The interest push device can implement the functions performed by the computer described in one or more of the above embodiments, and these functions can be implemented by hardware executing corresponding software. These hardware or software include one or more modules corresponding to the above functions. Figure 6 This is a schematic diagram of the structure of the interest push device in the embodiment of the present application, see Figure 6 As shown, the interest pushing device 600 may include: an acquisition module 601, used to acquire multiple user behavior sequences; an extraction module 602, used to input multiple user behavior sequences into a multi-interest module for feature extraction, so as to obtain interest features corresponding to each user behavior sequence; a similarity calculation module 603, used to calculate the similarity between a preset interest classification feature and an interest feature corresponding to each user behavior sequence, so as to obtain a target interest feature corresponding to each user behavior sequence, wherein the target interest feature is the interest feature with the highest similarity to the interest classification feature among the interest features corresponding to each user behavior sequence; an analysis module 604, used to input the target interest feature corresponding to each user behavior sequence into an interest hierarchy module for analysis based on a self-attention mechanism, so as to obtain an estimated interest feature corresponding to each user behavior sequence; a determination module 605, used to determine an interest estimation value based on the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence, wherein the interest estimation value is used to represent the content of interest to the user.

[0089] In a possible implementation, the extraction module 602 is specifically used to: analyze multiple user behavior sequences based on the self-attention mechanism to obtain interest features corresponding to each user behavior sequence.

[0090] In a possible implementation, the extraction module 602 is specifically used to: obtain three data expression features corresponding to each user behavior sequence, the three data expression features including a first expression feature, a second expression feature, and a third expression feature that are uniformly used for the user behavior sequence after different transformations; obtain an attention weight corresponding to each user behavior sequence by calculating the similarity between the first expression feature and the second expression feature; and multiply the attention weight by the third expression feature to obtain an interest feature corresponding to each user behavior sequence.

[0091] In a possible implementation, the similarity calculation module 603 is specifically used to: perform cross calculations on the preset interest classification features and the interest features corresponding to each user behavior sequence to obtain the target interest features corresponding to each user behavior sequence.

[0092] In one possible implementation, the analysis module 604 is specifically used to: obtain a set of position embedding features corresponding to each user behavior sequence according to each user behavior sequence; add a set of position embedding features corresponding to each user behavior sequence and a target interest feature corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence, wherein the interest sequence corresponding to each user behavior sequence is composed of a target interest feature corresponding to a user behavior sequence; and perform a self-attention mechanism-based analysis on the interest sequence corresponding to each user behavior sequence to obtain an estimated interest feature corresponding to each user behavior sequence.

[0093] In one possible implementation, the analysis module 604 is specifically used to: obtain three data expression features corresponding to the interest sequence corresponding to each user behavior sequence; the three data expression features include a fourth expression feature, a fifth expression feature, and a sixth expression feature that are uniformly used for the interest sequence after different transformations; obtain the attention weight corresponding to each interest sequence by calculating the similarity between the fourth expression feature and the fifth expression feature; multiply the attention weight by the sixth expression feature to obtain the estimated interest feature corresponding to each user behavior sequence.

[0094] In a possible implementation, the determination module 605 is specifically used to: input the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to determine the interest estimation value.

[0095] In a possible implementation, the acquisition module 601 may also be used to: embed and transform multiple user behavior sequences.

[0096] It should be noted that the specific implementation process of the acquisition module 601, the extraction module 602, the similarity calculation module 603, the analysis module 604 and the determination module 605 can be referred to Figures 1 to 5 For the sake of brevity, the detailed description of the embodiments will not be repeated here.

[0097] The acquisition module 601, extraction module 602, similarity calculation module 603, analysis module 604 and determination module 605 mentioned in the embodiment of the present application can be one or more processors.

[0098] Based on the same inventive concept, an embodiment of the present application provides an electronic device, which may be the interest push device described in one or more of the above embodiments, or may be a data processing chip in the device. Figure 7 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application, see Figure 7 As shown, the electronic device 700 may adopt general computer hardware, including a processor 701 and a memory 702 .

[0099] In some possible implementations, at least one processor 701 may constitute any physical device having a circuit that performs a logical operation on one or more inputs. For example, at least one processor may include one or more integrated circuits (ICs), including application specific integrated circuits (ASICs), microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or other circuits suitable for executing instructions or performing logical operations. The instructions executed by at least one processor may be preloaded into a memory integrated with or embedded in a controller, or may be stored in a separate memory. The memory may include a random access memory (RAM), a read-only memory (ROM), a hard disk, an optical disk, a magnetic medium, a flash memory, other permanent, fixed or volatile memory, or any other mechanism capable of storing instructions. In some embodiments, at least one processor may include more than one processor. Each processor may have a similar structure, or the processors may have different configurations that are electrically connected or disconnected from each other. For example, the processor may be a separate circuit or integrated in a single circuit. When more than one processor is used, the processors may be configured to operate independently or collaboratively. The processors may be coupled electrically, magnetically, optically, acoustically, mechanically, or by other means that allow them to interact. According to one embodiment of the present application, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which are executed by the processor to perform the steps of the above-mentioned calibration method. The memory 702 may include computer storage media in the form of volatile and / or non-volatile memory, such as read-only memory and / or random access memory. The memory 702 may store operating systems, applications, other program modules, executable code, program data, user data, etc.

[0100] In addition, the memory 702 stores the Figure 6 The computer executes instructions for the functions of the acquisition module 601, the extraction module 602, the similarity calculation module 603, the analysis module 604 and the determination module 605. Figure 6The functions / implementation processes of the acquisition module 601, the extraction module 602, the similarity calculation module 603, the analysis module 604 and the determination module 605 can be realized by Figure 7 The processor 701 in the embodiment calls the computer execution instructions stored in the memory 702 to implement the above-mentioned implementation process and functions.

[0101] Based on the same inventive concept, an embodiment of the present application provides an electronic device, including: a memory storing computer executable instructions; a processor connected to the memory, used to execute computer executable instructions and implement the multi-engine data query method as described in one or more of the above embodiments.

[0102] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. After the computer-executable instructions are executed by a processor, the interest push method described in one or more of the above embodiments can be implemented.

[0103] Those skilled in the art will understand that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0104] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that he or she may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent replacements on some of the technical features therein. Such modifications or replacements shall not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and shall all be included in the protection scope of the present application.

Claims

1. A method for pushing interest, characterized in that: Applied to an electronic device, the electronic device includes a multi-layer neural network model, the multi-layer neural network model at least includes: a plurality of multi-interest modules and an interest level module, the plurality of multi-interest modules are respectively used to extract different interest features; the method includes: Acquire multiple user behavior sequences; the multiple user behavior sequences are acquired from user historical behaviors; Inputting the multiple user behavior sequences into the multi-interest module for feature extraction to obtain interest features corresponding to each user behavior sequence; Calculate the similarity between the preset interest classification feature and the interest feature corresponding to each user behavior sequence to obtain a target interest feature corresponding to each user behavior sequence, wherein the target interest feature is the interest feature with the highest similarity to the interest classification feature among the interest features corresponding to each user behavior sequence; Inputting the target interest feature corresponding to each user behavior sequence into the interest level module for analysis based on the self-attention mechanism to obtain the estimated interest feature corresponding to each user behavior sequence; Determine an interest estimation value according to the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence, wherein the interest estimation value is used to represent the content that the user is interested in; Among them, the target interest features corresponding to each user behavior sequence are input into the interest hierarchy module for analysis based on the self-attention mechanism to obtain the estimated interest features corresponding to each user behavior sequence, including: according to each user behavior sequence, a set of position embedding features corresponding to each user behavior sequence is obtained; a set of position embedding features corresponding to each user behavior sequence is added to the target interest features corresponding to each user behavior sequence to obtain the interest sequence corresponding to each user behavior sequence, and the interest sequence corresponding to each user behavior sequence is composed of the target interest features corresponding to each user behavior sequence; the interest sequence corresponding to each user behavior sequence is analyzed based on the self-attention mechanism to obtain the estimated interest features corresponding to each user behavior sequence.

2. The method according to claim 1, characterized in that The inputting the plurality of user behavior sequences into the multi-interest module for feature extraction to obtain the interest feature corresponding to each user behavior sequence includes: The multiple user behavior sequences are analyzed based on a self-attention mechanism to obtain interest features corresponding to each user behavior sequence.

3. The method according to claim 2, characterized in that The analyzing the multiple user behavior sequences based on the self-attention mechanism to obtain the interest features corresponding to each user behavior sequence includes: Obtaining three data expression features corresponding to each of the user behavior sequences, wherein the three data expression features include a first expression feature, a second expression feature, and a third expression feature uniformly obtained after different transformations of the user behavior sequences; Obtaining an attention weight corresponding to each user behavior sequence by calculating a similarity between the first expression feature and the second expression feature; The attention weight is multiplied by the third expression feature to obtain the interest feature corresponding to each user behavior sequence.

4. The method according to claim 1, characterized in that The calculating of similarity between the preset interest classification feature and the interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence includes: The preset interest classification feature is cross-calculated with the interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence.

5. The method according to claim 1, characterized in that The analyzing the interest sequence corresponding to each user behavior sequence based on the self-attention mechanism to obtain the estimated interest feature corresponding to each user behavior sequence includes: Obtaining three data expression features corresponding to the interest sequence corresponding to each user behavior sequence; the three data expression features include a fourth expression feature, a fifth expression feature, and a sixth expression feature uniformly obtained after different transformations of the interest sequence; Obtaining an attention weight corresponding to each interest sequence by calculating a similarity between the fourth expression feature and the fifth expression feature; The attention weight is multiplied by the sixth expression feature to obtain the estimated interest feature corresponding to each user behavior sequence.

6. The method according to claim 1, characterized in that The multi-layer neural network model also includes: a multi-layer perceptron MLP module; The determining the interest estimation value according to the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence includes: The estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence are input into the MLP module to determine the interest estimation value.

7. The method according to claim 1, characterized in that After obtaining the plurality of user behavior sequences, the method further includes: The multiple user behavior sequences are embedded and transformed.

8. An interest push device, characterized in that: Applied to an electronic device, the electronic device includes a multi-layer neural network model, the multi-layer neural network model at least includes: a plurality of multi-interest modules and an interest level module, the plurality of multi-interest modules are respectively used to extract different interest features; the device includes: An acquisition module, used to acquire multiple user behavior sequences; the multiple user behavior sequences are acquired from user historical behaviors; An extraction module, used for inputting the plurality of user behavior sequences into the multi-interest module for feature extraction to obtain an interest feature corresponding to each user behavior sequence; A similarity calculation module is used to calculate the similarity between the preset interest classification feature and the interest feature corresponding to each user behavior sequence, and obtain a target interest feature corresponding to each user behavior sequence, wherein the target interest feature is the interest feature with the highest similarity to the interest classification feature among the interest features corresponding to each user behavior sequence; An analysis module, used for inputting the target interest feature corresponding to each user behavior sequence into the interest level module for analysis based on a self-attention mechanism, so as to obtain an estimated interest feature corresponding to each user behavior sequence; A determination module, configured to determine an interest estimation value according to the estimated interest feature corresponding to each user behavior sequence and the interest feature corresponding to each user behavior sequence, wherein the interest estimation value is used to represent the content that the user is interested in; Among them, the analysis module is specifically used to: obtain a set of position embedding features corresponding to each user behavior sequence according to each user behavior sequence; add a set of position embedding features corresponding to each user behavior sequence and the target interest features corresponding to each user behavior sequence to obtain an interest sequence corresponding to each user behavior sequence, and the interest sequence corresponding to each user behavior sequence is composed of the target interest features corresponding to each user behavior sequence; analyze the interest sequence corresponding to each user behavior sequence based on the self-attention mechanism to obtain the estimated interest features corresponding to each user behavior sequence.

9. The device according to claim 8, characterized in that The extraction module is specifically used to: analyze the multiple user behavior sequences based on the self-attention mechanism to obtain the interest features corresponding to each user behavior sequence.

10. The device according to claim 9, characterized in that The extraction module is specifically used to: obtain three data expression features corresponding to each user behavior sequence, the three data expression features include a first expression feature, a second expression feature and a third expression feature that are uniformly used for the user behavior sequence after different transformations; obtain an attention weight corresponding to each user behavior sequence by calculating the similarity between the first expression feature and the second expression feature; multiply the attention weight by the third expression feature to obtain an interest feature corresponding to each user behavior sequence.

11. The device according to claim 8, characterized in that The similarity calculation module is specifically used to: cross-calculate the preset interest classification feature with the interest feature corresponding to each user behavior sequence to obtain the target interest feature corresponding to each user behavior sequence.

12. The device according to claim 8, characterized in that The analysis module is specifically used to: obtain three data expression features corresponding to the interest sequence corresponding to each user behavior sequence; the three data expression features include a fourth expression feature, a fifth expression feature and a sixth expression feature that are uniformly used for the interest sequence after different transformations; obtain the attention weight corresponding to each interest sequence by calculating the similarity between the fourth expression feature and the fifth expression feature; multiply the attention weight by the sixth expression feature to obtain the estimated interest feature corresponding to each user behavior sequence.

13. The device according to claim 8, characterized in that The multi-layer neural network model also includes: a multi-layer perceptron MLP module; the determination module is specifically used to: input the estimated interest features corresponding to each user behavior sequence and the interest features corresponding to each user behavior sequence into the MLP module to determine the interest estimation value.

14. The device according to claim 8, characterized in that After acquiring the multiple user behavior sequences, the acquisition module is further used to: embed and transform the multiple user behavior sequences.

15. An electronic device, characterized in that: include: a memory for storing processor-executable instructions; Processor; wherein the processor is configured to: implement the method according to any one of claims 1 to 7 when executing the executable instructions.

16. A computer-readable storage medium, characterized in that: The readable storage medium stores an executable program, wherein the executable program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images