Cultural product value prediction method and device based on deep learning

Abstract

The invention relates to a cultural product value prediction method and device based on deep learning, and the method achieves the value prediction of a cultural product through the construction of a deep learning network, the network employs a brand new attention module, and the module carries out the linear transformation of input data into a query feature matrix and a value feature matrix, and carries out the prediction of the value of the cultural product. A query feature matrix PLU is decomposed into a lower triangular feature matrix and an upper triangular feature matrix to replace Q and K in an original multi-head attention mechanism, so that attention weight calculation does not depend on interaction between tokens any more, the whole input data can be fully mined, and the accuracy of attention weight calculation is improved. And calculating the multiplication of the lower triangular feature matrix and the upper triangular feature matrix, and using an activation function to obtain the attention coefficient of each feature, the attention coefficient reflecting the importance of the features on value prediction, thereby improving the explanation and understanding of the deep learning network on the prediction result, and finally improving the accuracy of predicting the product value by the deep learning network.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of cultural product value prediction, and in particular, to a method and device for cultural product value prediction based on deep learning. Background Art

[0002] There are many factors that influence the property value of cultural products, and there is a complex nonlinear relationship between property value and its influencing factors, making value assessment a fundamental and restrictive issue in the cultural industry.

[0003] The current prediction of the value of cultural products mainly relies on the subjective judgment of experts and the supply and demand relationship of the market. Therefore, there are problems such as high subjectivity and inconsistent evaluation results, which makes the accuracy and reliability of the evaluation easily questionable. At present, there are also some research works that use machine learning or deep learning methods to predict value. Among them, the machine learning-based methods mainly use decision tree models. Deep learning-based methods include: recursive neural network RNN, convolutional neural network CNN, etc. However, RNN is prone to gradient disappearance or gradient explosion; CNN uses gradient descent for iteration, which makes it easy for the training results to converge to the local optimum rather than the global optimum; and the RNN and CNN neural network models do not consider the weight of the features, and the interpretability is poor. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a method and device for predicting the value of cultural products based on deep learning, which can improve the accuracy of the encoder in predicting the product value.

[0006] The first aspect of the embodiment of the present application proposes a method for predicting the value of cultural products based on deep learning, the method comprising:

[0007] Constructing a deep learning network; the deep learning network is composed of L encoders connected in sequence, wherein L is a positive integer greater than 2, the L encoders have the same structure, and the encoder includes an attention module and a residual module with a matrix decomposition attention mechanism;

[0008] Inputting input data into the deep learning network to obtain the predicted value of the cultural product data to be tested output by the deep learning network;

[0009] Wherein, if the i-th encoder is the first encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the second encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder;

[0010] The process of obtaining output data by the i-th encoder includes:

[0011] Input the input data into the attention module to perform a linear transformation of the multi-head attention mechanism through the attention module to obtain a query feature matrix and a value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use a Softmax activation function to obtain an attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain a weighted value feature matrix;

[0012] According to the weighted value feature matrix, the output data of the i-th encoder is generated by the residual module.

[0013] This application proposes a method for predicting the value of cultural products based on deep learning, which has the following beneficial effects:

[0014] This method constructs a deep learning network and realizes the value prediction of cultural products through the deep learning network. The attention module used in the deep learning network proposes a new matrix decomposition attention mechanism, through which the input data is first linearly transformed into a query feature matrix and a value feature matrix, and the query feature matrix PLU matrix is ​​decomposed into a lower triangular feature matrix and an upper triangular feature matrix, which replaces Q and K in the original multi-head attention mechanism, so that the calculation of attention weights no longer depends on the interaction between tokens, and the entire input data can be fully mined. The lower triangular feature matrix and the upper triangular feature matrix are multiplied and the activation function is used to obtain the attention coefficient of each feature. This attention coefficient reflects the importance of the feature to the value prediction, thereby improving the encoder's interpretation and understanding of the prediction results, and finally improving the accuracy of the encoder in predicting the product value.

[0015] In some embodiments of the present application, generating the output data of the i-th encoder through the residual module according to the weighted value feature matrix includes:

[0016] Passing the weighted value feature matrix through a first fully connected layer and a Dropout layer in sequence to obtain a first feature matrix;

[0017] Adding the first characteristic matrix and the input data to obtain a second characteristic matrix;

[0018] Input the second feature matrix into the LayerNorm layer to obtain a third feature matrix;

[0019] Adding the input data and the third characteristic matrix to obtain a fourth characteristic matrix;

[0020] The fourth characteristic matrix is ​​input into the residual module, and the residual module outputs output data of the i-th encoder.

[0021] In some embodiments of the present application, the step of inputting the fourth characteristic matrix into the residual module to obtain the output data of the i-th encoder output by the residual module includes:

[0022] Inputting the fourth feature matrix into a convolutional layer to obtain a convolutional layer output residual feature matrix;

[0023] Performing global average pooling on the residual feature matrix to obtain a fifth feature matrix;

[0024] Input the fifth feature matrix into the second fully connected layer and use an activation function to obtain a sixth feature matrix output by the second fully connected layer;

[0025] Multiplying the sixth characteristic matrix and the residual characteristic matrix to obtain a seventh characteristic matrix;

[0026] The seventh characteristic matrix is ​​added to the fourth characteristic matrix to obtain an eighth characteristic matrix, and the eighth characteristic matrix is ​​used as output data of the i-th encoder.

[0027] In some embodiments of the present application, if the i-th encoder is the L-th encoder, generating the predicted value of the cultural product data to be tested according to the output data of the i-th encoder includes:

[0028] The eighth feature matrix is ​​input into the third fully connected layer to obtain the predicted value of the cultural product data to be tested output by the third fully connected layer.

[0029] In some embodiments of the present application, the data matrix to be tested obtained by preprocessing the cultural product data to be tested includes:

[0030] Performing feature scaling on each of the cultural product data to be tested to obtain the scaled cultural product data to be tested;

[0031] Normalizing each of the scaled cultural product data to be tested to obtain the normalized cultural product data to be tested;

[0032] In some embodiments of the present application, each test data after preprocessing is subjected to the operations of adding prediction tokens and position encoding:

[0033] A prediction token is added to each data to be tested in the normalized cultural product data to obtain the data matrix to be tested.

[0034] Some embodiments of the present application, after adding prediction tokens to each of the normalized data to be tested in the cultural product data to be tested to obtain the data matrix to be tested, further include:

[0035] A position code is added to each data to be tested in the normalized cultural product data to be tested.

[0036] In some embodiments of the present application, the step of inputting the fourth feature matrix into a convolutional layer to obtain the convolutional layer output residual feature matrix includes:

[0037] The fourth feature matrix is ​​input into two one-dimensional convolutional layers to obtain a ninth feature matrix, and the ninth feature matrix and the fourth feature matrix are added and a ReLU activation function is used to obtain the residual feature matrix.

[0038] A second aspect of the embodiments of the present application provides a cultural product value prediction device based on deep learning, the device comprising:

[0039] A network construction unit, used to construct a deep learning network; the deep learning network is composed of L encoders connected in sequence, wherein L is a positive integer greater than 2, the L encoders have the same structure, and the encoder includes an attention module and a residual module with a matrix decomposition attention mechanism;

[0040] A data input unit, used for inputting input data into the deep learning network to obtain the predicted value of the cultural product data to be tested output by the deep learning network;

[0041] Wherein, if the i-th encoder is the 1-th encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the 2-th encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder;

[0042] The data encoding unit is used to execute the process of the i-th encoder to obtain the output data:

[0043] Input the input data into the attention module to perform a linear transformation of the multi-head attention mechanism through the attention module to obtain a query feature matrix and a value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use a Softmax activation function to obtain an attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain a weighted value feature matrix;

[0044] According to the weighted value feature matrix, the output data of the i-th encoder is generated by the residual module.

[0045] The third aspect of an embodiment of the present application proposes an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the above-mentioned deep learning-based cultural product value prediction method.

[0046] The fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the cultural product value prediction method based on deep learning as described above.

[0047] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 It is a flow chart of the method for predicting the value of cultural products based on deep learning provided by this application;

[0050] Figure 2 is a coding diagram of any encoder provided in this application;

[0051] Figure 3 It is a schematic diagram of data preprocessing provided by this application;

[0052] Figure 4 is a schematic diagram of the architecture of L encoders provided in this application;

[0053] Figure 5 A schematic diagram of the architecture of the residual network provided in this application;

[0054] Figure 6 It is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] like Figure 1 and Figure 2 One embodiment of the present application provides a method for predicting the value of cultural products based on deep learning, the method comprising:

[0059] Step S100, constructing a deep learning network, inputting input data into the deep learning network, and obtaining the predicted value of the cultural product data to be tested output by the deep learning network;

[0060] The deep learning network is composed of L encoders connected in sequence, where L is a positive integer greater than 2, the L encoders have the same structure, and the encoder includes an attention module and a residual module with a matrix decomposition attention mechanism;

[0061] Wherein, if the i-th encoder is the first encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the second encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder;

[0062] The process of obtaining output data by the i-th encoder in step S100 includes:

[0063] Step S110, input the input data to the attention module, so as to perform a linear transformation of the multi-head attention mechanism through the attention module to obtain a query feature matrix and a value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use the Softmax activation function to obtain an attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain a weighted value feature matrix;

[0064] Step S120, generating output data of the i-th encoder through a residual module according to the weighted value feature matrix. In some embodiments of the present application, the cultural product data to be tested is introduced as follows:

[0065] Taking movies as an example, the indicator system of cultural products such as movies is listed below:

[0066] The film index system includes: basic information (awards, length), director (director professional index, director total box office, director awards, director popularity index), screenwriter (screenwriter total box office), actor (actor reputation index, actor total box office, actor awards, actor popularity index), film marketing (search volume), ideology (mainstream values), film quality (story, performance), etc.; the evaluation indicators in the index system are divided into objective indicators and subjective indicators. Objective indicators can directly obtain numerical values, such as "length" and "director total box office". Subjective indicators rely on people's subjective feelings and judgments, such as "story", "performance" and "mainstream values" of film works, "artistic language innovation", "creative skills" and "mainstream values" of painting works, so crowdsourcing users with certain professional knowledge are selected to score the performance of cultural works on subjective indicators.

[0067] After obtaining the values ​​of the cultural products on various evaluation indicators, they are used as the cultural product data to be tested, and then the L encoders provided in this application are used to predict the value of the cultural product data to be tested.

[0068] In some embodiments of the present application, the data matrix to be tested obtained by preprocessing the cultural product data to be tested in step S110 includes:

[0069] Step S210 , performing feature scaling on each of the cultural product data to be tested to obtain the scaled cultural product data to be tested.

[0070] Step S220 , normalizing each of the scaled cultural product data to be tested to obtain normalized cultural product data to be tested.

[0071] like Figure 3 , assuming that the cultural product data to be tested is X.

[0072] Because the attribute value ranges of the cultural product data X to be tested are very different, if X is used directly for modeling, the role of features with larger numerical scales in modeling will be highlighted, while the role of features with smaller numerical scales will be relatively weakened or ignored. Therefore, in order to ensure the effectiveness and reliability of the L encoders, it is necessary to perform feature scaling on X.

[0073] Taking movie data as an example, firstly, the attributes of each data to be tested in the cultural product data X to be tested are judged, and the attributes with a larger feature range are logarithmically operated. Then, all the data are uniformly normalized to reduce the influence of dimensional units on the modeling of L encoders.

[0074] In step S210, since the director box office and screenwriter box office values ​​in the movie data are too large, the logarithm operation is performed on these two features to obtain the scaled cultural product data X to be tested. ln, where X ln ={x1,x2,…,x 12}. Since the logarithmic function is a monotonically increasing function within its domain, taking the logarithm will not change the relative relationship of the data. The purpose of taking the logarithm is to reduce the absolute value of the data to facilitate calculation. At the same time, the logarithmic operation compresses the data scale, making the data more stable.

[0075] In step S220, then X ln Perform normalization operation. It represents the minimum value of each attribute after taking the logarithm. Indicates the maximum value corresponding to each attribute after taking the logarithm, and uses the MinMaxScaler normalization method to obtain the normalized data X scaled , as shown in formula (1):

[0076]

[0077] The normalized data X scaled As the data matrix to be tested.

[0078] like Figure 4 In some embodiments, after step S220, the method further includes adding prediction tokens and position encoding to the test data matrix, wherein the step of adding prediction tokens is:

[0079] Step S230, adding prediction tokens to each data to be tested in the normalized cultural product data to be tested, to obtain a data matrix to be tested.

[0080] The steps to add positional encoding include:

[0081] Step S240, adding a position code to each data to be tested in the normalized cultural product data to be tested.

[0082] Here's how to add positional encodings and prediction tokens:

[0083] Taking movie data as an example, assuming that the cultural product data to be tested is X∈R d×c , where d is the number of attributes and c is the number of original channels. Considering the impact of different indicator scales and dimensional units on data analysis, the cultural product data X∈R d×c The various index data in the data are subjected to feature scaling processing (as in the above embodiment) through data preprocessing to obtain the normalized cultural product data X to be tested. scaled ∈R d×c The following is the normalized data of the culture product to be tested X scaled ∈R d×c Add positional encoding and prediction tokens.

[0084] First, the normalized cultural product data X scaled ∈R d×c Perform linear mapping to obtain X′;

[0085] Then add the prediction token X to the first position of each data in X′ reg ∈R 1×dim Get X″∈R (d +1)×dim ;

[0086] Prediction Token X reg Randomly initialized, and the parameters are continuously updated as the encoder network is trained; X reg Perform global feature aggregation on all information on X′. reg It does not contain information about the input sequence, so use X reg This can avoid bias towards a specific feature in X′. reg Aggregating the information of X′, this application will predict the token X reg Used to predict the final value of cultural products.

[0087] The order of attributes of the tested cultural product data X will have a certain impact on the prediction results, and the L encoders process the sequence. In order to retain the position of the feature in the tested cultural product data X, a position code E is added to each feature. pos ∈R (d+1)×dim , and then obtain the feature matrix X in , as shown in formula (2):

[0088]

[0089] in, Yes X scaled The i-th attribute in , E is the transformation matrix, which transforms X scaled Mapped to the embedding space, the position encoding E pos ~N(0,1), by adding position encoding, L encoders can understand the relative position of the feature in the input sequence. At this time, the cultural product data to be tested X∈R d×c Change to X in ∈R (d+1)×dim , where d is the number of attributes and d+1 is the number of attributes added to X reg The number of attributes after the linear mapping, dim is the number of channels after the linear mapping.

[0090] like Figure 4 In some embodiments, the process of obtaining output data by the i-th encoder includes the above steps S110 and S120, which are specifically explained as follows:

[0091] The encoder of this embodiment includes an attention module and a residual module, wherein the attention module proposes a new attention mechanism, which is called the matrix factorization attention mechanism (MFAM); the residual module is called FCRM (Feature Calibration Residual Module).

[0092] Assume i is equal to 1, and the input data of the first encoder is X in ;

[0093] The original attention mechanism uses a dot product operation to calculate the attention coefficient, which basically learns self-alignment, that is, determines the relative importance of a single token relative to all other tokens in the sequence. The attention weights are learned at the instance level, and the weights are generated by pairwise interactions at the instance level. Therefore, these instance-specific interactions tend to fluctuate freely between different instances, making them lack consistent global context information.

[0094] This embodiment proposes a new attention module. The first step is to transform the input data X in Perform a linear transformation of the multi-head attention mechanism to obtain the query feature matrix X Q ∈R (d+1)×h×dim Sum feature matrix X V ∈R (d+1)×h×dim , where h is the number of attention heads.

[0095] Then use the PLU matrix decomposition method to transform the input feature matrix X Q Decomposed into a permutation matrix P∈R h ×(d+1)×(d+1) , the lower triangular matrix Q L ∈R h×(d+1)×(d+1) , upper triangular matrix V U ∈R h×(d+1)×dim Since the permutation matrix P is used to adjust the query feature matrix X Q The row order of does not change the original values, so the matrix P is not used in subsequent calculations.

[0096] Next, Q L and V U Perform matrix multiplication to obtain the similarity feature matrix H∈R h×(d+1)×(d+1) , then the matrix H is activated by the softmax function to obtain the attention coefficient feature matrix A∈R h×(d+1)×(d+1) .

[0097] Finally, the attention coefficient feature matrix A is combined with the input value feature matrix X V Multiply to obtain the weighted value feature matrix S∈R h×(d+1)×dim .

[0098] In some embodiments, generating output data of the i-th encoder through a residual module according to the weighted value feature matrix in step S120 includes:

[0099] Step S410, passing the weighted value feature matrix through the first fully connected layer and the Dropout layer in sequence to obtain a first feature matrix;

[0100] Step S420, adding the first characteristic matrix and the input data to obtain a second characteristic matrix;

[0101] Step S430, input the second feature matrix to the LayerNorm layer to obtain a third feature matrix;

[0102] Step S440, adding the input data and the third characteristic matrix to obtain a fourth characteristic matrix;

[0103] Step S450: input the fourth feature matrix into the residual module to obtain output data of the i-th encoder.

[0104] The weighted value feature matrix is ​​fed into the first fully connected layer to remap the features of different dimensions of the h heads to one dimension. The Dropout layer can be further processed to avoid excessive dependence on certain features;

[0105] The first feature matrix and the input data are added to obtain the second feature matrix; the second feature matrix is ​​input to the LayerNorm layer to obtain the third feature matrix, which can prevent information loss;

[0106] The input data and the third feature matrix are added (i.e., residual connection) to prevent overfitting.

[0107] The content of step S450 is introduced in the subsequent embodiments.

[0108] The following is a detailed introduction to the attention module:

[0109] First, the input feature matrix X in Perform linear transformation to obtain the query feature matrix Value feature matrix As shown in formulas (3) and (4):

[0110]

[0111]

[0112] Among them, i represents the i-th head of the multi-head attention mechanism, Respectively represent the weight matrices used to calculate the query feature matrix and the value feature matrix. After linear transformation, Xin Different features are mapped to different spaces, each head only needs to focus on some features, and the prediction model can better understand the data from multiple dimensions.

[0113] Then After PLU decomposition, PLU decomposition uses Gaussian elimination to decompose a complex matrix into a product of simple matrices by performing multiple transformations on the rows. The calculation process of PLU decomposition is as follows:

[0114] In the first round, let the feature matrix Select The number with the largest absolute value in the first column is used as the pivot. If the pivot is not in the first row, it is swapped with the first row. To record the exchange process, as shown in formula (5):

[0115]

[0116] in, is the permutation matrix used in the first exchange process. If there is no exchange, is the identity matrix.

[0117] Then, use Gaussian elimination to change the first column of numbers on the right side of the equation except the pivot to 0, as shown in formula (6):

[0118]

[0119] Among them, i represents the i-th head of the multi-head attention mechanism, j represents the number of rounds of the current operation, 1≤j≤n, and n is the number of eliminations. Then choose The number with the largest absolute value in the first column is taken as the principal variable, and formula (6) is calculated repeatedly until the final result is is an upper triangular matrix. After PLU decomposition, we get a permutation matrix P i , the lower triangular characteristic matrix and the upper triangular characteristic matrix As shown in formulas (7) and (8),

[0120]

[0121]

[0122] Since the feature matrix and It is obtained by decomposing the input feature matrix. Both feature matrices contain part of the features of the input feature matrix, so they can be used as the feature matrix Q and feature matrix K in the attention mechanism to calculate the attention coefficient.

[0123] The feature matrix generated after PLU decomposition After a fully connected layer, it is mapped to the feature matrix In the same space, the feature matrix and Multiply them together, and then pass through a softmax layer to calculate the attention coefficient of each head, as shown in formula (9):

[0124]

[0125] Where Q, K, and V are query, key, and value. Each header S i The calculation of is shown in formula (10):

[0126]

[0127] The final output S of multi-head self-attention is obtained by formula (11).

[0128] S = Concat(S 1 ,S 2 ,…,S h )W O (11)

[0129] Among them, S∈R h×(d+1)×dim , h is the total number of heads, which can be set to 4 here, W O is the learnable weight feature matrix.

[0130] Formula (11) can be used to obtain the multi-head feature representation S after weighting by the attention mechanism. S is then sent to the first fully connected layer to remap the features of different dimensions of the four heads to one dimension, as shown in formula (12):

[0131] S′=FC(S) (12)

[0132] Where s′∈R (d+1)×dim .

[0133] In order to prevent overfitting, Dropout is selected to further process s′. Each neuron is ignored with a certain probability to prevent over-reliance on certain features. The probability of Dropout can be set to 0.1. The features after Dropout processing (i.e., the first feature matrix) are compared with the input data X of the i-th encoder. in Addition, by adding the transformed features to the original information, can help the encoder learn better and prevent information loss.

[0134] Then after passing through the LayerNorm layer, the output third feature matrix MFAM (X in ) is shown in formula (13):

[0135] MFAM(X in ) = LayerNorm(Dropout(S′0+X in 0 (13)

[0136] After obtaining the weighted feature representation, in order to enable the L encoders to learn more complex features and better capture the diversity of data, MFAM(X in ) and the input data X of the first encoder in Add together to get the output data Z′1∈R of the first encoder (d+1)×dim , as shown in formula (14):

[0137] Z′1=MFAM(X in )+X in (14)

[0138] Because this embodiment uses L encoders to process the features in sequence. Assuming i is greater than 1, it is extended to the i-th encoder. The output data Z′ of the i-th encoder i It can be calculated by formula (15):

[0139] Z′ i =MFAM(Z i-1 )+Z i-1 (15)

[0140] Where i = 1, ..., L, L is the number of encoders used. i-1 Please refer to the subsequent embodiments for the explanation of , which is the output data of the residual module of the i-1th encoder.

[0141] In some embodiments, the step S450 of inputting the fourth feature matrix into the residual module to obtain the output data of the i-th encoder output by the residual module includes:

[0142] Step S510, inputting the fourth feature matrix into the convolution layer to obtain a convolution layer output residual feature matrix;

[0143] Step S520, performing global average pooling on the residual feature matrix to obtain a fifth feature matrix;

[0144] Step S530, inputting the fifth feature matrix into the second fully connected layer and using an activation function to obtain a sixth feature matrix output by the second fully connected layer;

[0145] Step S540, multiplying the sixth characteristic matrix and the residual characteristic matrix to obtain a seventh characteristic matrix;

[0146] Step S550: Add the seventh characteristic matrix to the fourth characteristic matrix to obtain an eighth characteristic matrix, and use the eighth characteristic matrix as output data of the i-th encoder.

[0147] In step S510, the fourth feature matrix is ​​input into the convolution layer to obtain the convolution layer output residual feature matrix, including:

[0148] Step S610: Input the fourth feature matrix into two one-dimensional convolutional layers to obtain a ninth feature matrix, and residually connect the ninth feature matrix and the fourth feature matrix to obtain a residual feature matrix.

[0149] Wherein step S540 generates the predicted value of the cultural product data to be tested according to the output data of the i-th encoder, including:

[0150] Step S710, input the eighth feature matrix into the third fully connected layer to obtain the predicted value of the cultural product data to be tested output by the third fully connected layer.

[0151] like Figure 5 In this embodiment, the original Transformer encoder uses a feedforward neural network to process the features weighted by the attention mechanism. However, the feedforward neural network has a large amount of computation and is prone to overfitting, and has poor performance on small data sets. Therefore, in order to improve the overall prediction accuracy of the L encoders and improve the feature learning ability, the weighted features output by the attention module are re-learned through the residual module, and the features are adaptively calibrated to improve the nonlinear representation ability of the encoder.

[0152] The residual module here is a residual structure including convolution operation, channel squeezing operation and compressed channel mapping operation. The residual module is used to process Z′1.

[0153] The residual module of this embodiment is a residual structure including convolution operation, channel squeezing operation and compressed channel mapping operation, wherein the convolution operation is used to extract local feature information, the channel squeezing operation compresses the feature channel information, and the compressed channel mapping operation fully mines the compressed features.

[0154] The residual module is used to process the weighted features (i.e., the fourth feature matrix) obtained through the attention network.

[0155] The feature first passes through two one-dimensional (1D) convolutional layers with skip connections to obtain the residual feature matrix X res ∈R (d+1)×dim , then the residual feature matrix X resAfter global average pooling (GAP), the features are compressed in the channel dimension to obtain the fifth feature matrix, and then the fifth feature matrix is ​​learned by the FC layer (i.e., the second fully connected layer) and the activation function. The sixth feature matrix and the residual feature matrix X res Multiply them to get the calibrated seventh feature matrix X rec .

[0156] In some embodiments, to further help the encoder converge and alleviate gradient disappearance, a residual connection is used to convert the seventh feature matrix X rec Added to Z′1, we get the output of the residual module That is the output data Z1 of the first encoder.

[0157] The L encoder structures consisting of attention modules and residual modules are used to process the features in sequence, and the output data of the last encoder is sent to the third fully connected layer (FC) to obtain the value prediction result y of the cultural product. Therefore, the overall implementation process is shown in formulas (16) to (18).

[0158]

[0159] Among them, Z i Refers to the output of the i-th layer encoder, and the input data of the 1st layer encoder is Z0=X in , L is the total number of encoders. Here, L=5 can be selected.

[0160] The following is a detailed introduction to the residual network:

[0161] Although the attention mechanism can better mine the dependencies in sequence data, is not restricted by distance, and has good global feature extraction capabilities, its local information learning ability is weak. The convolutional layer has good inductive biases, such as locality and translation equivariance, and is therefore better at extracting local information. Therefore, in order to extract more expressive feature information, the convolution operation is used to extract local feature information from the attention-weighted features to fully capture the local dependencies between data. The specific implementation details are shown in Table 1.

[0162] Table 1

[0163]

[0164] In order to avoid the problem of gradient disappearance, the residual structure is used to transform Z′1 and the ninth feature matrix X after two layers of convolution. vonv Make the connection as shown in formula (19):

[0165] Xres =δ(Z′1+X conv ) (19)

[0166] Among them, δ is the ReLU activation function.

[0167] Then the feature channel information is compressed to reduce the number of model parameters. Here, global average pooling is used to perform channel average pooling on the features, as shown in formula (20):

[0168]

[0169] Where dim is the channel dimension of the second layer convolutional mapping, that is, dim=64.

[0170] In order to make full use of the feature information after channel compression and improve the nonlinear feature learning ability of the model, the FC layer and activation function are used to map the features and the mapping output is compared with X. tes Multiply them together to obtain the calibrated feature information, as shown in formula (21):

[0171] X rec =X res ×σ(FC(δ(FC(X GAP )))) (twenty one)

[0172] Among them, σ is the Sigmoid activation function and δ is the ReLU activation function.

[0173] Finally, in order to further help the L encoders converge quickly and alleviate the gradient vanishing problem, Z′1 and X are again rec Add together to obtain the final output FCRM(Z′1) of the residual network, that is, the final output Z1 of the first encoder, as shown in formula (22):

[0174] FCRM(Z′1)=Z′1+X rec (twenty two)

[0175] Extending formula (22) to the i-th layer, the final output Z of the i-th layer encoder can be calculated by formula (23): i .

[0176] Z i =FCRM(Z′ i ) (twenty three)

[0177] Wherein, i=1,…,L, and L is the number of encoders used.

[0178] This embodiment has at least the following beneficial effects:

[0179] (1) This method constructs a deep learning network to realize the value prediction of cultural products, and uses a new attention module (i.e., matrix decomposition attention module). The attention module uses the information of the entire input data through matrix decomposition, rather than relying solely on a single test data, to reduce the impact of individual test data on the weight, thereby improving the prediction accuracy of L encoders; the attention module decomposes the query feature matrix PLU matrix into a lower triangular feature matrix and an upper triangular feature matrix, replacing Q and K in the original multi-head attention mechanism, so that the calculation of attention weights no longer depends on the interaction between tokens, and can fully mine the entire input data to calculate the attention coefficient of each feature. This attention coefficient reflects the importance of the feature to value prediction, thereby improving the encoder's interpretation and understanding of the prediction results, and finally improving the accuracy of the encoder in predicting the product value.

[0180] (2) Each encoder is equipped with a residual module, which extracts more expressive feature information. The convolution operation is used to extract local feature information from the attention-weighted features, fully capturing the local dependencies between data. The weighted features output by the attention module are re-learned to achieve adaptive calibration of the features, thereby improving the accuracy of the encoder in predicting product value.

[0181] The present application provides a cultural product value prediction device based on deep learning, the device comprising:

[0182] A data input unit, used for inputting input data into the i-th encoder among L encoders connected in sequence, to obtain output data of the i-th encoder; wherein L is a positive integer greater than 2; the L encoders have the same structure, and the encoder includes an attention module and a residual module;

[0183] Wherein, if the i-th encoder is the first encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the second encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder;

[0184] The data encoding unit is used to execute the process of the i-th encoder to obtain the output data:

[0185] Input the input data to the attention module to perform a linear transformation of the multi-head attention mechanism through the attention mechanism to obtain the query feature matrix and the value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use the activation function to obtain the attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain the weighted value feature matrix;

[0186] According to the weighted value feature matrix, the output data of the i-th encoder is generated through the residual module.

[0187] It should be noted that the cultural product value prediction device based on deep learning provided in this embodiment and the cultural product value prediction method based on deep learning mentioned above are based on the same inventive concept. Therefore, the relevant content of the cultural product value prediction method based on deep learning mentioned above is also applicable to the embodiment of the cultural product value prediction device based on deep learning and will not be repeated here.

[0188] like Figure 6 The embodiment of the present application further provides an electronic device, the electronic device comprising:

[0189] at least one memory;

[0190] at least one processor;

[0191] at least one program;

[0192] The program is stored in the memory, and the processor executes at least one program to implement the cultural product value prediction method based on deep learning as described above in the present disclosure.

[0193] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0194] The electronic device according to the embodiment of the present application is described in detail below.

[0195] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0196] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes the cultural product value prediction method based on deep learning in the embodiment of the present invention.

[0197] Input / output interface 1800, used to implement information input and output;

[0198] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0199] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0200] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0201] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned cultural product value prediction method based on deep learning.

[0202] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The embodiments described in the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0204] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0205] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0207] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0212] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A method for predicting the value of cultural products based on deep learning, the method comprising: Build deep learning networks; The deep learning network is composed of L encoders connected in sequence, wherein L is a positive integer greater than 2, the L encoders have the same structure, and the encoder includes an attention module and a residual module with a matrix decomposition attention mechanism; Inputting input data into the deep learning network to obtain the predicted value of the cultural product data to be tested output by the deep learning network; Wherein, if the i-th encoder is the first encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the second encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder; The process of obtaining output data by the i-th encoder includes: Input the input data into the attention module to perform a linear transformation of the multi-head attention mechanism through the attention module to obtain a query feature matrix and a value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use a Softmax activation function to obtain an attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain a weighted value feature matrix; According to the weighted value feature matrix, the output data of the i-th encoder is generated by the residual module.

2. The method for predicting the value of cultural products based on deep learning according to claim 1, characterized in that: The step of generating output data of the i-th encoder through the residual module according to the weighted value feature matrix comprises: Passing the weighted value feature matrix through a first fully connected layer and a Dropout layer in sequence to obtain a first feature matrix; Adding the first characteristic matrix and the input data to obtain a second characteristic matrix; Input the second feature matrix into the LayerNorm layer to obtain a third feature matrix; Adding the input data and the third characteristic matrix to obtain a fourth characteristic matrix; The fourth characteristic matrix is ​​input into the residual module, and the residual module outputs output data of the i-th encoder.

3. The method for predicting the value of cultural products based on deep learning according to claim 2 is characterized in that: The step of inputting the fourth characteristic matrix into the residual module to obtain output data of the i-th encoder outputted by the residual module includes: Inputting the fourth feature matrix into a convolutional layer to obtain a convolutional layer output residual feature matrix; Performing global average pooling on the residual feature matrix to obtain a fifth feature matrix; Input the fifth feature matrix into the second fully connected layer and use an activation function to obtain a sixth feature matrix output by the second fully connected layer; Multiplying the sixth characteristic matrix and the residual characteristic matrix to obtain a seventh characteristic matrix; The seventh characteristic matrix is ​​added to the fourth characteristic matrix to obtain an eighth characteristic matrix, and the eighth characteristic matrix is ​​used as output data of the i-th encoder.

4. The method for predicting the value of cultural products based on deep learning according to claim 3 is characterized in that: If the i-th encoder is the L-th encoder, then generating the predicted value of the to-be-tested cultural product data according to the output data of the i-th encoder includes: The eighth feature matrix is ​​input into the third fully connected layer to obtain the predicted value of the cultural product data to be tested output by the third fully connected layer.

5. The method for predicting the value of cultural products based on deep learning according to claim 1, characterized in that: The test data matrix obtained by preprocessing the test cultural product data includes: Performing feature scaling on each of the cultural product data to be tested to obtain the scaled cultural product data to be tested; Each of the scaled data to be tested in the cultural product data to be tested is normalized to obtain the normalized cultural product data to be tested.

6. The method for predicting the value of cultural products based on deep learning according to claim 5, characterized in that: After adding prediction tokens to each of the normalized data to be tested in the cultural product data to be tested to obtain the data matrix to be tested, the method further includes: A position code is added to each data to be tested in the normalized cultural product data to be tested.

7. The method for predicting the value of cultural products based on deep learning according to claim 3 is characterized in that: Inputting the fourth feature matrix into a convolutional layer to obtain the convolutional layer output residual feature matrix includes: The fourth feature matrix is ​​input into two one-dimensional convolutional layers to obtain a ninth feature matrix, and the ninth feature matrix and the fourth feature matrix are added together and a ReLU activation function is used to obtain the residual feature matrix.

8. A cultural product value prediction device based on deep learning, characterized in that: The device comprises: A network construction unit, used to construct a deep learning network; the deep learning network is composed of L encoders connected in sequence, wherein L is a positive integer greater than 2, the L encoders have the same structure, and the encoder includes an attention module and a residual module with a matrix decomposition attention mechanism; A data input unit, used to input input data into the deep learning network to obtain the predicted value of the cultural product data to be tested output by the deep learning network; Wherein, if the i-th encoder is the first encoder, the input data is a test data matrix obtained by preprocessing the test cultural product data, and each test data in the test data matrix contains a prediction token; if the i-th encoder is any encoder from the second encoder to the L-1-th encoder, the input data is the output data of the i-1-th encoder; if the i-th encoder is the L-th encoder, the prediction value of the test cultural product data is generated according to the output data of the i-th encoder; The data encoding unit is used to execute the process of the i-th encoder to obtain the output data: Input the input data into the attention module to perform a linear transformation of the multi-head attention mechanism through the attention module to obtain a query feature matrix and a value feature matrix; decompose the query feature matrix PLU into a lower triangular feature matrix and an upper triangular feature matrix; multiply the lower triangular feature matrix and the upper triangular feature matrix and use a Softmax activation function to obtain an attention coefficient feature matrix; multiply the attention coefficient feature matrix and the value feature matrix to obtain a weighted value feature matrix; According to the weighted value feature matrix, the output data of the i-th encoder is generated by the residual module.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the cultural product value prediction method based on deep learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the cultural product value prediction method based on deep learning as described in any one of claims 1 to 7.

Images