Data processing method and device

Abstract

The present invention discloses a data processing method and device, and relates to the field of computer technology. The method includes: mapping sample data corresponding to target information to obtain a first feature matrix; performing operation processing on the vectors composed of the same dimension elements in the plurality of feature vectors in the first feature matrix, and between the feature vectors and / or between the elements in a preset order, and obtaining the corresponding second feature matrix, and the third feature matrix and / or the fourth feature matrix according to the operation processing results; fusing the second feature matrix, and the third feature matrix and / or the fourth feature matrix to obtain the result after fusion processing; inputting the result after fusion processing into a click-through rate prediction model to obtain the click-through rate prediction result of the target information. Through the above steps, the representation of the Embedding vector can be optimized and the accuracy of the output result of the click-through rate prediction model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method and device. Background Art

[0002] With the development of the Internet, the content on the Internet is becoming more and more abundant. Users often need to spend a lot of time and energy to filter the information they need from these massive contents. Therefore, how to quickly find the content that users are interested in has become a very important research direction of the Internet. Among them, the click-through-rate (CTR) model is one of the key technologies to solve this problem. The click-through-rate prediction model can predict the probability of users clicking on candidate information based on the characteristics and historical behavior of users.

[0003] In the prior art, initially, discrete features were directly input into the CTR prediction model to obtain the output of the model without any mining of the feature data. For example, this resulted in problems such as low estimation accuracy and poor effect in the output of the CTR prediction model. In order to solve the above problems, the prior art proposed the Embedding method. The Embedding method maps the discrete features in the sample into a feature vector, and further mines the hidden features by operating the feature vector.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: the existing Embedding method has problems such as insufficient feature mining and large noise in the feature mining process, which leads to the accuracy of the output results of the final click-through rate prediction model is still relatively low and the prediction effect is not ideal. Summary of the invention

[0005] In view of this, the present invention provides a data processing method and device, which can optimize the representation of embedding vectors and improve the accuracy of the output results of the click rate prediction model.

[0006] To achieve the above objective, according to one aspect of the present invention, a data processing method is provided.

[0007] The data processing method of the present invention includes: mapping sample data corresponding to target information to obtain a first feature matrix; wherein the sample data includes multiple discrete features; the first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features; performing operation processing on the vectors composed of the same dimensional elements in the multiple feature vectors in the first feature matrix, and between the feature vectors and / or between the elements in a preset order, and obtaining the corresponding second feature matrix, third feature matrix and / or fourth feature matrix according to the operation processing results; fusing the second feature matrix, the third feature matrix and / or the fourth feature matrix to obtain the fused result; inputting the fused result into a click rate prediction model to obtain the click rate prediction result of the target information.

[0008] Optionally, the performing of operations on the vectors composed of elements of the same dimension in the multiple eigenvectors, and between the eigenvectors and / or between the elements in the first characteristic matrix in a preset order, and obtaining the corresponding second characteristic matrix, third characteristic matrix and / or fourth characteristic matrix according to the operation processing results includes: performing operations on the vectors composed of elements of the same dimension in the multiple eigenvectors in the first characteristic matrix, and obtaining the second characteristic matrix according to the operation processing results; performing operations on the eigenvectors in the second characteristic matrix, and obtaining the third characteristic matrix according to the operation processing results, and / or performing operations on the elements in the second characteristic matrix, and obtaining the fourth characteristic matrix according to the operation processing results.

[0009] Optionally, performing operation processing on the vectors composed of elements of the same dimension in the multiple feature vectors in the first feature matrix, and obtaining the second feature matrix according to the operation processing results includes: modeling the relationship between the vectors composed of elements of the same dimension in the multiple feature vectors in the first feature matrix based on a self-attention mechanism to obtain a first relationship matrix; and constructing the second feature matrix according to the first relationship matrix and the first feature matrix.

[0010] Optionally, constructing the second characteristic matrix according to the first relationship matrix and the first characteristic matrix includes: performing a matrix multiplication operation on the first relationship matrix and the first characteristic matrix, and performing a matrix addition operation on the result of the matrix multiplication operation and the first characteristic matrix to obtain a second characteristic matrix.

[0011] Optionally, performing operations on the eigenvectors in the second feature matrix and obtaining a third feature matrix based on the results of the operations includes: modeling the relationship between the eigenvectors in the second feature matrix based on a self-attention mechanism to obtain a second relationship matrix; and constructing the third feature matrix based on the second relationship matrix and the second feature matrix.

[0012] Optionally, performing operations on the elements in the second feature matrix and obtaining a fourth feature matrix based on the operations results includes: inputting the second feature matrix into a deep neural network model, performing operations on the elements in the second feature matrix based on the deep neural network model, and using the output results of the deep neural network model as the fourth feature matrix.

[0013] Optionally, the fusing processing of the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fused result includes: splicing the second characteristic matrix, the third characteristic matrix and the fourth characteristic matrix, and using the spliced ​​matrix as the fused result.

[0014] To achieve the above objective, according to another aspect of the present invention, a data processing device is provided.

[0015] The data processing device of the present invention includes: a mapping module, which is used to map sample data corresponding to target information to obtain a first feature matrix; wherein the sample data includes multiple discrete features; the first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features; a mining module, which is used to perform operations on the vectors composed of elements of the same dimension in the multiple feature vectors in the first feature matrix, and between the feature vectors and / or between the elements in a preset order, and obtain the corresponding second feature matrix, third feature matrix and / or fourth feature matrix according to the operation results; a fusion module, which is used to perform fusion processing on the second feature matrix, the third feature matrix and / or the fourth feature matrix to obtain the fusion result; an estimation module, which is used to input the fusion result into a click-through rate estimation model to obtain the click-through rate estimation result of the target information.

[0016] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.

[0017] The electronic device of the present invention comprises: one or more processors; and a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method of the present invention.

[0018] To achieve the above objective, according to yet another aspect of the present invention, a computer readable medium is provided.

[0019] The computer-readable medium of the present invention stores a computer program thereon, and when the program is executed by a processor, the data processing method of the present invention is implemented.

[0020] An embodiment of the above invention has the following advantages or beneficial effects: by mapping the sample data corresponding to the target information, a first feature matrix is ​​obtained; in the first feature matrix, vectors composed of elements of the same dimension in the multiple feature vectors, and feature vectors and / or elements are respectively processed in a preset order, and a second feature matrix, a third feature matrix and / or a fourth feature matrix corresponding thereto are obtained according to the results of the processing; the second feature matrix, the third feature matrix and / or the fourth feature matrix are fused, and the fused results are input into the click-through rate prediction model. These steps can optimize the representation of the Embedding vector and improve the accuracy of the output results of the click-through rate prediction model.

[0021] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0023] Figure 1 is a schematic diagram of the main flow of a data processing method according to a first embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the main flow of a data processing method according to a second embodiment of the present invention;

[0025] Figure 3 It is a schematic diagram of the principle of modeling the relationship between dimension vectors, feature vectors, and elements according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of the principle of constructing a second characteristic matrix from a first characteristic matrix according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of main modules of a data processing device according to a third embodiment of the present invention;

[0028] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0029] Figure 7It is a schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0031] It should be pointed out that the embodiments of the present invention and the features in the embodiments may be combined with each other without affecting the implementation of the present invention.

[0032] Figure 1 FIG. 1 is a schematic diagram of the main flow of the data processing method according to the first embodiment of the present invention. Figure 1 As shown, the data processing method of the embodiment of the present invention includes:

[0033] Step S101: Mapping is performed on sample data corresponding to target information to obtain a first feature matrix.

[0034] Exemplarily, the target information may be in the form of any one or a combination of multiple formats of image information, text information, or video information. For example, the target information may be commodity information in text form or image form.

[0035] The sample data corresponding to the target information includes multiple discrete features. In specific implementation, assuming that a sample contains m original features, the continuous features can be converted into discrete features by dividing the intervals, for example, the age feature can be discretized into an age group of 10 years old, and then m discrete features can be obtained from the m original features.

[0036] In this step, the first feature matrix can be obtained by mapping the sample data corresponding to the target information. The first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features. For example, assuming that there are m discrete features in a sample, each discrete feature is mapped to a d-dimensional feature vector through mapping processing, and then a first feature matrix composed of m feature vectors can be obtained.

[0037] Step S102, perform operations on the vectors composed of elements of the same dimension in the multiple feature vectors in the first feature matrix, and on the feature vectors and / or elements in a preset order, and obtain the corresponding second feature matrix, third feature matrix and / or fourth feature matrix according to the operation results.

[0038] In an optional embodiment, step S102 specifically includes: first performing operation processing on the vectors composed of the same dimensional elements in the multiple eigenvectors in the first characteristic matrix, and obtaining a second characteristic matrix according to the operation processing results; then, performing operation processing on the eigenvectors in the second characteristic matrix, and obtaining a third characteristic matrix according to the operation processing results; performing operation processing on the elements in the second characteristic matrix, and obtaining a fourth characteristic matrix according to the operation processing results.

[0039] For example, suppose the first feature matrix consists of three eigenvectors, namely:

[0040] (v 11 ,v 12 ,v 13 ,v 14 )、(v 21 ,v 22 ,v 23 ,v 24 )、(v 31 ,v 32 ,v 33 ,v 34 ), then the vectors formed by the elements of the same dimension in these three eigenvectors are (v 11 ,v 21 ,v 31 )、(v 12 ,v 22 ,v 32 )、(v 13 ,v 23 ,v 33 )、(v 14 ,v 24 ,v 34 ), we can first perform operations on these four vectors and obtain the second characteristic matrix based on the results of the operations; assuming that the second characteristic matrix consists of three characteristic vectors, namely (e 11 ,e 12 ,e 13 ,e 14 )、(e 21 ,e 22 ,e 23 ,e 24 )、(e 31 ,e 32 ,e 33 ,e34 ), then the three eigenvectors in the second characteristic matrix can be processed, and the third characteristic matrix can be obtained according to the processing results; the elements in the second characteristic matrix can be processed, and the fourth characteristic matrix can be obtained according to the processing results.

[0041] In another optional embodiment, step S102 specifically includes: first performing operation processing on the vectors composed of the same dimensional elements in the multiple eigenvectors in the first characteristic matrix, and obtaining a second characteristic matrix according to the operation processing results; then, performing operation processing on the eigenvectors in the second characteristic matrix, and obtaining a third characteristic matrix according to the operation processing results.

[0042] In another optional implementation, step S102 specifically includes: first performing operation processing on the vectors in the first characteristic matrix composed of elements of the same dimension in the multiple characteristic vectors, and obtaining a second characteristic matrix according to the operation processing results; then, performing operation processing on the elements in the second characteristic matrix, and obtaining a fourth characteristic matrix according to the operation processing results.

[0043] It should be noted that step S102 is not limited to the above three optional implementations. Without affecting the implementation of the present invention, step S102 may also adopt other optional implementations. For example, it is also possible to first perform operation processing on the eigenvectors in the first characteristic matrix, and obtain a third characteristic matrix according to the operation processing result, and then perform operation processing on the vectors composed of the same dimension elements in multiple eigenvectors in the third characteristic matrix, and obtain a second characteristic matrix according to the operation processing result; perform operation processing on the elements in the third characteristic matrix, and obtain a fourth characteristic matrix according to the operation processing result.

[0044] Step S103: perform fusion processing on the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fusion result.

[0045] In an optional implementation, step S103 includes: fusing the second characteristic matrix, the third characteristic matrix, and the fourth characteristic matrix to obtain a fused result. Exemplarily, the fusion process may be a concatenation process. For example, the second characteristic vector, the third characteristic vector, and the fourth characteristic vector may be expanded into a vector of 1 row and d*m columns respectively. Then concatenate these three vectors and concatenate the resulting vector As the result of fusion processing.

[0046] In another optional implementation, step S103 includes: fusing the second characteristic matrix and the third characteristic matrix to obtain a fused result.

[0047] In yet another optional implementation, step S103 includes: fusing the second characteristic matrix and the fourth characteristic matrix to obtain a fused result.

[0048] Step S104: input the fusion result into a click rate prediction model to obtain a click rate prediction result of the target information.

[0049] Exemplarily, the result after fusion processing can be input into a click rate prediction function (such as a logistic regression function), and the output function value is used as the click rate prediction result of the target information.

[0050] In an embodiment of the present invention, after obtaining the first feature matrix through mapping processing, the vectors composed of the same dimensional elements of the multiple feature vectors, and the feature vectors and / or the elements in the first feature matrix are respectively processed in a preset order, and the corresponding second feature matrix, third feature matrix and / or fourth feature matrix are obtained according to the results of the processing; the second feature matrix, the third feature matrix and / or the fourth feature matrix are fused, and the fused results are input into the click-through rate prediction model. These steps can optimize the representation of the Embedding vector and improve the accuracy of the output results of the click-through rate prediction model.

[0051] Figure 2 FIG. 2 is a schematic diagram of the main flow of the data processing method according to the second embodiment of the present invention. Figure 2 As shown, the data processing method of the embodiment of the present invention includes:

[0052] Step S201: Acquire sample data corresponding to target information.

[0053] Exemplarily, the target information may be in the form of any one or a combination of multiple formats of image information, text information, or video information. For example, the target information may be commodity information in text form or image form.

[0054] The sample data corresponding to the target information includes multiple discrete features. In specific implementation, it is assumed that a sample contains m original features. For the continuous features, they can be converted into discrete features by dividing intervals, etc. For example, for age features, the age features can be discretized with every 10 years as an age group, and then m discrete features can be obtained from m original features. For example, when the target information is product information, the sample data may include product features (such as product price, product color, etc.) and user features (such as user age, user gender, etc.) after discretization.

[0055] Step S202: Mapping the sample data corresponding to the target information to obtain a first feature matrix.

[0056] In this step, the first feature matrix can be obtained by mapping the sample data corresponding to the target information. The first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features. For example, assuming that there are m discrete features in a sample, each discrete feature is mapped to a d-dimensional feature vector through mapping processing, and then a first feature matrix composed of m feature vectors can be obtained. Further, the first feature matrix can be expressed as:

[0057]

[0058] Among them, V represents the first characteristic matrix, represents the i-th eigenvector (i.e., the vector representation of the i-th discrete feature), v ij represents the jth element in the i-th eigenvector.

[0059] Step S203: performing operation processing on the vectors in the first characteristic matrix that are composed of elements of the same dimension in the plurality of characteristic vectors, and obtaining a second characteristic matrix according to the operation processing result.

[0060] Exemplarily, step S203 specifically includes: step A1, modeling the relationship between vectors composed of elements of the same dimension in the multiple feature vectors in the first feature matrix based on the self-attention mechanism to obtain a first relationship matrix; step A2, constructing the second feature matrix based on the first relationship matrix and the first feature matrix.

[0061] The attention mechanism mimics the internal process of biological observation behavior, that is, a mechanism that aligns internal experience and external sensations to increase the observation precision of some areas. The attention mechanism can quickly extract important features of sparse data, so it is widely used in natural language processing tasks. The self-attention mechanism is a special attention mechanism that is good at capturing the internal correlation of data.

[0062] Further, in step A1, the first relationship matrix may be calculated based on the following formula:

[0063] S = Softmax(VV T )

[0064] Among them, S represents the first relationship matrix, S j,i is an element in the first relationship matrix, which represents the influence of “the vector composed of the i-th dimension elements in each eigenvector in the first feature matrix” on “the vector composed of the j-th dimension elements in each eigenvector in the first feature matrix”; V represents the first feature matrix, which consists of d rows and m columns of elements, and each column represents an eigenvector; V T represents the transpose of the first eigenvalue matrix.

[0065] Furthermore, in step A2, the first relationship matrix and the first characteristic matrix may be subjected to a matrix multiplication operation, and the result of the matrix multiplication operation and the first characteristic matrix may be subjected to a matrix addition operation to obtain a second characteristic matrix. The specific calculation formula is as follows:

[0066] E = SV + V;

[0067] Among them, E represents the second characteristic matrix, S represents the first relationship matrix, and V represents the first characteristic matrix.

[0068] In the above calculation formula, not only the influence of the relationship between the vectors composed of the same dimensional elements in each feature vector in the first feature matrix on the second feature matrix is ​​considered to distinguish the importance of different dimensions, but also the information of the previous layer (specifically the information of the first feature matrix) is retained. Furthermore, by calculating the second feature matrix through the above calculation formula, richer features can be obtained, and the vector expression of the features can be optimized, which is helpful to improve the subsequent click-through rate prediction results.

[0069] Step S204: perform operations on the eigenvectors in the second characteristic matrix, and obtain a third characteristic matrix according to the operation results.

[0070] Exemplarily, step S204 includes: step B1, modeling the relationship between the feature vectors in the second feature matrix based on the self-attention mechanism to obtain a second relationship matrix; step B2, constructing the third feature matrix based on the second relationship matrix and the second feature matrix.

[0071] Further, in step B1, the second relationship matrix may be calculated based on the following formula:

[0072] H = Softmax(E T E)

[0073] Among them, H represents the second relationship matrix, Hj,i is an element in the second relationship matrix, which represents the influence of “the i-th eigenvector in the first feature matrix” on “the j-th eigenvector in the first feature matrix”; E represents the second feature matrix, which consists of d rows and m columns, and each column represents an eigenvector; E T Represents the transpose of the second characteristic matrix.

[0074] Furthermore, in step B2, the second relationship matrix and the second characteristic matrix may be subjected to a matrix multiplication operation, and the result of the matrix multiplication operation and the second characteristic matrix may be subjected to a matrix addition operation to obtain a third characteristic matrix. The specific calculation formula is as follows:

[0075] F = EH + E;

[0076] Among them, F represents the third characteristic matrix, H represents the second relationship matrix, and E represents the second characteristic matrix.

[0077] In the above calculation formula, not only the influence of the relationship between the eigenvectors in the first feature matrix on the third feature matrix is ​​considered to distinguish the importance of different eigenvectors, but also the information of the previous layer (specifically the information of the second feature matrix) is retained. Furthermore, by calculating the third feature matrix through the above calculation formula, richer features can be obtained, and the vector expression of the features can be optimized, which is helpful to improve the subsequent click-through rate prediction results.

[0078] Step S205: perform operations on the elements in the second characteristic matrix, and obtain a fourth characteristic matrix according to the operation results.

[0079] Exemplarily, step S205 includes: inputting the second feature matrix into a deep neural network model, performing operations on the elements in the second feature matrix based on a deep neural network model (DNN model), and using the output of the deep neural network model as a fourth feature matrix. In specific implementation, in order to mine high-order feature information, multiple DNN models can be superimposed for feature mining.

[0080] The DNN model uses a fully connected modeling method, relying on nonlinear activation functions and deep structures, and can learn high-order nonlinear relationships in data. In the DNN model, the specific calculation formula is as follows:

[0081] h i+1 =f(W i h i +b i )

[0082] Among them, h i represents the input of the i-th layer in the DNN model (for example, the input of the first layer is the second feature matrix); h i+1represents the output of the i-th layer in the DNN model (for example, the output of the last layer is the fourth feature matrix); W i represents the weight matrix; b i represents the offset; f(x) represents the activation function, for example, the linear rectified unit (ReLU) function can be used as the activation function.

[0083] Step S206: perform fusion processing on the second characteristic matrix, the third characteristic matrix and the fourth characteristic matrix to obtain a fusion result.

[0084] Exemplarily, the fusion process may be a splicing process. For example, the second eigenvector, the third eigenvector, and the fourth eigenvector may be expanded into a vector of 1 row and d*m columns respectively. Then concatenate these three vectors and concatenate the resulting vector As the result of fusion processing.

[0085] Step S207: input the fusion result into a click rate prediction model to obtain a click rate prediction result of the target information.

[0086] Exemplarily, the result after fusion processing can be input into a click rate prediction function (such as a logistic regression function), and the output function value is used as the click rate prediction result of the target information.

[0087] In an embodiment of the present invention, after obtaining the first feature matrix through mapping processing, the vectors composed of the same dimensional elements of the multiple feature vectors, the feature vectors, and the elements in the first feature matrix are respectively processed in a preset order, and the corresponding second feature matrix, third feature matrix and fourth feature matrix are obtained according to the results of the processing; the second feature matrix, the third feature matrix and the fourth feature matrix are fused, and the fused results are input into the click-through rate prediction model. These steps can mine richer hidden features, optimize the vector representation of features, reduce noise in feature mining, and improve the accuracy of the output results of the click-through rate prediction model.

[0088] Figure 3 Schematic diagram of the principle of modeling the relationship between dimension vectors, feature vectors, and elements according to an embodiment of the present invention. Figure 3As shown, assuming that the first feature matrix is ​​composed of three 1*4-dimensional feature vectors, namely feature vector 1 (for example, it is a vector representation of the discrete feature "age"), feature vector 2 (for example, it is a vector representation of the discrete feature "gender"), and feature vector 3 (for example, it is a vector representation of the discrete feature "category"), then in an embodiment of the present invention, the relationship between the dimensional vectors in the first feature matrix (that is, the vector composed of the same dimensional elements in the three feature vectors in the first feature matrix), the relationship between the feature vectors, and the relationship between the elements can be modeled respectively to mine richer hidden features and optimize the representation of the Embedding vector.

[0089] Figure 4 Schematic diagram of the principle of constructing a second characteristic matrix from a first characteristic matrix according to an embodiment of the present invention. Figure 4 As shown, after the first characteristic matrix V is input, the first characteristic matrix V can be transposed, and then the first characteristic matrix V and the transposed V of the first characteristic matrix are T Perform matrix multiplication operation, and input the result of the matrix multiplication operation into the Softmax function (normalized exponential function) to obtain the first relationship matrix S; then perform matrix multiplication operation on the first relationship matrix S and the first characteristic matrix, and perform matrix addition operation on the result of the matrix multiplication operation and the first characteristic matrix V to obtain the second characteristic matrix E.

[0090] Figure 5 Schematic diagram of the main modules of a data processing device according to the third embodiment of the present invention. Figure 5 As shown, the data processing device 500 of the embodiment of the present invention includes: a mapping module 501, a mining module 502, a fusion module 503, and an estimation module 504.

[0091] The mapping module 501 is used to perform mapping processing on the sample data corresponding to the target information to obtain a first feature matrix.

[0092] Exemplarily, the target information may be in the form of any one or a combination of multiple formats of image information, text information, or video information. For example, the target information may be commodity information in text form or image form.

[0093] The sample data corresponding to the target information includes multiple discrete features. In specific implementation, assuming that a sample contains m original features, the continuous features can be converted into discrete features by dividing the intervals, for example, the age feature can be discretized into an age group of 10 years old, and then m discrete features can be obtained from the m original features.

[0094] The first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features. For example, assuming that there are m discrete features in a sample, each discrete feature can be mapped to a d-dimensional feature vector through mapping module 501, and then a first feature matrix consisting of m feature vectors can be obtained.

[0095] The mining module 502 is used to perform operations on the vectors composed of the same dimensional elements of the multiple feature vectors in the first feature matrix, and between the feature vectors and / or between the elements in a preset order, and obtain the corresponding second feature matrix, third feature matrix and / or fourth feature matrix according to the operation results.

[0096] In an optional embodiment, the mining module 502 is specifically used to: first perform operation processing on the vectors composed of the same dimensional elements in the multiple feature vectors in the first feature matrix, and obtain a second feature matrix according to the operation processing results; then, perform operation processing on the feature vectors in the second feature matrix, and obtain a third feature matrix according to the operation processing results; perform operation processing on the elements in the second feature matrix, and obtain a fourth feature matrix according to the operation processing results.

[0097] For example, suppose the first feature matrix consists of three eigenvectors, namely:

[0098] (v 11 ,v 12 ,v 13 ,v 14 )、(v 21 ,v 22 ,v 23 ,v 24 )、(v 31 ,v 32 ,v 33 ,v 34 ), then the vectors formed by the elements of the same dimension in these three eigenvectors are (v 11 ,v 21 ,v 31 )、(v 12 ,v 22 ,v 32 )、(v 13 ,v 23 ,v 33 )、(v 14 ,v 24 ,v 34 ), we can first perform operations on these four vectors and obtain the second characteristic matrix based on the results of the operations; assuming that the second characteristic matrix consists of three characteristic vectors, namely (e 11 ,e12 ,e 13 ,e 14 )、(e 21 ,e 22 ,e 23 ,e 24 )、(e 31 ,e 32 ,e 33 ,e 34 ), then the three eigenvectors in the second characteristic matrix can be processed, and the third characteristic matrix can be obtained according to the processing results; the elements in the second characteristic matrix can be processed, and the fourth characteristic matrix can be obtained according to the processing results.

[0099] In another optional embodiment, the mining module 502 is specifically used to: first perform operation processing on the vectors composed of the same dimensional elements in the multiple feature vectors in the first feature matrix, and obtain a second feature matrix according to the operation processing results; then, perform operation processing on the feature vectors in the second feature matrix, and obtain a third feature matrix according to the operation processing results.

[0100] In another optional implementation, the mining module 502 is specifically used to: first perform operation processing on the vectors in the first feature matrix composed of elements of the same dimension in the multiple feature vectors, and obtain a second feature matrix according to the operation processing results; then, perform operation processing on the elements in the second feature matrix, and obtain a fourth feature matrix according to the operation processing results.

[0101] It should be noted that the mining module 502 is not limited to the above three optional implementations. Without affecting the implementation of the present invention, the mining module 502 may also adopt other optional implementations. For example, the mining module 502 may also first perform operation processing on the feature vectors in the first feature matrix, and obtain a third feature matrix according to the operation processing results, and then perform operation processing on the vectors composed of the same dimension elements in multiple feature vectors in the third feature matrix, and obtain a second feature matrix according to the operation processing results; perform operation processing on the elements in the third feature matrix, and obtain a fourth feature matrix according to the operation processing results.

[0102] The fusion module 503 is used to perform fusion processing on the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fusion result.

[0103] In an optional implementation, the fusion module 503 is specifically used to: perform fusion processing on the second feature matrix, the third feature matrix and the fourth feature matrix to obtain a fusion result. Exemplarily, the fusion processing may be a splicing process. For example, the fusion module 503 may expand the second feature vector, the third feature vector and the fourth feature vector into a vector of 1 row and d*m columns respectively. Then concatenate these three vectors and concatenate the resulting vector As the result of fusion processing.

[0104] In another optional implementation, the fusion module 503 is specifically used to: perform fusion processing on the second characteristic matrix and the third characteristic matrix to obtain a fusion result.

[0105] In yet another optional implementation, the fusion module 503 is specifically configured to: perform fusion processing on the second characteristic matrix and the fourth characteristic matrix to obtain a fusion result.

[0106] The estimation module 504 is used to input the fusion processing result into the click rate estimation model to obtain the click rate estimation result of the target information.

[0107] Exemplarily, the estimation module 504 may input the result after fusion processing into a click rate estimation function (such as a logistic regression function), and use the output function value as the click rate estimation result of the target information.

[0108] In an embodiment of the present invention, after the first feature matrix is ​​obtained by mapping through the mapping module, the vectors composed of the same dimensional elements of the multiple feature vectors, and the feature vectors and / or the elements in the first feature matrix are respectively processed in a preset order through the mining module, and the corresponding second feature matrix, third feature matrix and / or fourth feature matrix are obtained according to the results of the processing; the second feature matrix, the third feature matrix and / or the fourth feature matrix are fused through the fusion module, and the fused results are input into the click-through rate prediction model through the estimation module, which can optimize the representation of the Embedding vector and improve the accuracy of the output results of the click-through rate prediction model.

[0109] Figure 6 An exemplary system architecture 600 is shown to which the data processing method or data processing device according to the embodiment of the present invention can be applied.

[0110] like Figure 6As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0111] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as vehicle dispatch management applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0112] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0113] The server 605 may be a server that provides various services, such as a background management server that provides support for shopping applications or websites browsed by users using the terminal devices 601, 602, and 603. The background management server may analyze and process the received click rate estimation request and other data, and use the processing results (such as click rate estimation results) in subsequent processing flows such as product recommendations.

[0114] It should be noted that the data processing method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the data processing device is generally disposed in the server 605 .

[0115] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0116] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of an electronic device suitable for implementing an embodiment of the present invention. Figure 7 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0117] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0118] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0119] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0120] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a mapping module, a mining module, a fusion module and an estimation module. The names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the mapping module may also be described as a "module for mapping sample data corresponding to target information".

[0123] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: mapping the sample data corresponding to the target information to obtain a first feature matrix; wherein the sample data includes multiple discrete features; the first feature matrix includes multiple feature vectors with the same dimension and corresponding to the discrete features; the vectors composed of the same dimension elements in the multiple feature vectors in the first feature matrix, and the feature vectors and / or the elements are respectively operated in a preset order to obtain the corresponding second feature matrix, and the third feature matrix and / or the fourth feature matrix; the second feature matrix, the third feature matrix and / or the fourth feature matrix are fused to obtain the fused result; the fused result is input into the click rate prediction model to obtain the click rate prediction result of the target information.

[0124] According to the technical solution of the embodiment of the present invention, the representation of the Embedding vector can be optimized and the accuracy of the output results of the click-through rate prediction model can be improved.

[0125] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Mapping the sample data corresponding to the target information to obtain a first feature matrix; wherein the sample data includes a plurality of discrete features; the first feature matrix includes a plurality of feature vectors having the same dimension and corresponding to the discrete features; Performing operations on the vectors composed of the elements of the same dimension in the plurality of feature vectors, and on the feature vectors and / or elements in the first feature matrix in a preset order, and obtaining corresponding second feature matrices, third feature matrices and / or fourth feature matrices according to the operation results; Performing a fusion process on the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fusion result; The result after the fusion processing is input into a click rate prediction model to obtain a click rate prediction result of the target information.

2. The method according to claim 1, characterized in that The vectors formed by the elements of the same dimension in the plurality of feature vectors in the first feature matrix, the feature vectors and / or the elements are respectively subjected to operation processing in a preset order, and the corresponding second feature matrix, third feature matrix and / or fourth feature matrix are obtained according to the operation processing results, including: Performing operations on the vectors in the first characteristic matrix that are composed of elements of the same dimension in the multiple characteristic vectors, and obtaining a second characteristic matrix based on the operation processing results; performing operations on the characteristic vectors in the second characteristic matrix, and obtaining a third characteristic matrix based on the operation processing results, and / or performing operations on the elements in the second characteristic matrix, and obtaining a fourth characteristic matrix based on the operation processing results.

3. The method according to claim 2, characterized in that The performing operation processing on the vectors in the first feature matrix formed by the elements of the same dimension in the plurality of feature vectors, and obtaining the second feature matrix according to the operation processing result comprises: Based on the self-attention mechanism, the relationship between the vectors composed of the elements of the same dimension in the multiple feature vectors in the first feature matrix is ​​modeled to obtain a first relationship matrix; and the second feature matrix is ​​constructed according to the first relationship matrix and the first feature matrix.

4. The method according to claim 3, characterized in that The constructing the second feature matrix according to the first relationship matrix and the first feature matrix includes: A matrix multiplication operation is performed on the first relationship matrix and the first characteristic matrix, and a matrix addition operation is performed on the result of the matrix multiplication operation and the first characteristic matrix to obtain a second characteristic matrix.

5. The method according to claim 2, characterized in that: The performing operation processing on the eigenvectors in the second characteristic matrix to obtain a third characteristic matrix according to the operation processing result comprises: Based on the self-attention mechanism, the relationship between the feature vectors in the second feature matrix is ​​modeled to obtain a second relationship matrix; and the third feature matrix is ​​constructed according to the second relationship matrix and the second feature matrix.

6. The method according to claim 2, characterized in that The performing operation processing on the elements in the second characteristic matrix to obtain a fourth characteristic matrix according to the operation processing result comprises: The second feature matrix is ​​input into the deep neural network model to perform operations on the elements in the second feature matrix based on the deep neural network model, and the output result of the deep neural network model is used as the fourth feature matrix.

7. The method according to claim 1, characterized in that The fusing the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fused result includes: The second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix are concatenated, and the concatenated matrix is ​​used as the result of the fusion processing.

8. A data processing device, characterized in that: The device comprises: A mapping module, used for mapping the sample data corresponding to the target information to obtain a first feature matrix; wherein the sample data includes a plurality of discrete features; the first feature matrix includes a plurality of feature vectors having the same dimension and corresponding to the discrete features; A mining module, configured to perform operations on the vectors composed of the elements of the same dimension in the plurality of feature vectors, and on the feature vectors and / or elements in the first feature matrix in a preset order, and obtain a second feature matrix, a third feature matrix and / or a fourth feature matrix corresponding thereto according to the operation results; A fusion module, used for fusing the second characteristic matrix, the third characteristic matrix and / or the fourth characteristic matrix to obtain a fusion result; The estimation module is used to input the result after the fusion processing into the click rate estimation model to obtain the click rate estimation result of the target information.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images