A take-out evaluation text classification method based on deep learning and gradient reversal

Abstract

The application discloses a kind of take-out evaluation text classification method based on deep learning and gradient inversion, comprising the following steps: take-out evaluation data acquisition and pre-processing;Convolutional Neural Network is constructed and initialized;Bert network is constructed and initialized;Back propagation;Take-out evaluation text classification;Take-out evaluation text classification effect evaluation.The application builds the complete process of data preprocessing to end-to-end training through Transformers, and obtains the initial representation space of the text through the embedding layer, and sends the representation space vector into the model.In the feature vector extracted by the original model, the application further utilizes the gradient flip layer to obtain the representation space of the non-converged data, and projects the representation space of the non-converged data into the feature vector space extracted by the original model, obtains the abstract data that can be used, and finally combines the two kinds of feature vectors for classification to achieve better generalization ability and improve the accuracy of text classification.

Description

Technical Field

[0001] This invention relates to the field of food delivery review text classification, specifically to a food delivery review text classification method based on deep learning and gradient inversion. Background Technology

[0002] Food delivery systems are ubiquitous in my country, boasting significant timeliness and substantial economic and social impact. They have profoundly influenced the country's catering industry, allowing people to enjoy local delicacies from anywhere within a radius of tens of kilometers without leaving home. Within these systems, users' most immediate impressions of restaurants largely stem from reviews. To enhance user first impressions, major platforms select well-regarded restaurants for their homepages. Therefore, these platforms analyze restaurant reviews, using this data as reliable input for their recommendation systems, thus influencing the overall recommendation algorithm. For example, the homepages of major food delivery apps typically feature highly-rated restaurants; this automatic update method is largely achieved by analyzing user sentiment towards different restaurants and their interests.

[0003] However, with the development of food delivery systems, user reviews of restaurants on various platforms have become increasingly varied. Some reviews conceal the true emotions behind the text, some contain no useful information at all, some only include emojis, and some are too brief. These types of reviews place higher demands on text classification models.

[0004] Most existing methods for analyzing text focus on deep learning. In sentiment recognition, after the preprocessed text of a food delivery order review is input into the model, the network outputs whether the text belongs to positive or negative sentiment. When the network makes a mistake, it uses a pre-set loss function to find the difference between the network prediction and the true label, and then derives the gradient of the feature vector from the value of the loss function. Finally, it updates the model parameters through the gradient and the learning rate, and iterates continuously so that the model can fit the review text to be predicted in the future.

[0005] However, some of the deep learning models mentioned above cannot accurately predict actual text due to insufficient training data. For example, CNN-type and BERT-type models have limited context capture when analyzing food delivery review text, which limits their ability to classify such text. Summary of the Invention

[0006] To address the above problems, this invention provides a method for classifying food delivery review text based on deep learning and gradient inversion, comprising the following steps:

[0007] S1 Food Delivery Review Data Acquisition and Preprocessing:

[0008] Obtain order review data from food delivery platforms Where N represents the number of training samples, This represents a training sample from a Chinese corpus. This represents the label for a single sample, with a value of either 0 or 1.

[0009] Extract training data from the training dataset without replacement. It uses a general function Tokenizer and a pre-trained embedding layer to process Chinese short sentences, mapping individual characters or words to integers, and then mapping the corresponding integers to E-dimensional vectors;

[0010] Each sentence is truncated and expanded: if a sentence exceeds a certain length L, the information exceeding L is truncated; if the length of the entire sentence is less than L, it is filled with 0 to the length L.

[0011] After processing by Tokenizer and Embedding, we get:

[0012]

[0013] in, It is a processed matrix. Represents a ;

[0014] S2 constructs a network model for classifying text classification of food delivery reviews.

[0015] S2-1 Construct and initialize the convolutional neural network

[0016] Three convolutional kernels are defined in the F-Net convolutional network, and one convolutional kernel is defined in the A-Net convolutional network;

[0017] The A-Net convolutional network also requires stacking a gradient inversion layer on top of the convolutional network layers to modify the gradient derivation equation of the feature vectors;

[0018] In F-Net and A-Net convolutional networks, pooling layers are paired with convolutional kernels.

[0019] Max pooling is used to obtain the most salient features extracted by the convolution kernel for food delivery review classification;

[0020] In convolutional neural networks, text convolution has two layers. The first layer uses an embedding layer to obtain an abstract representation of the text. The second layer uses convolutional kernels of different sizes and max pooling layers to extract feature vectors from the text. The prediction step is formalized as follows:

[0021]

[0022]

[0023]

[0024] in, It is the output of the embedding layer. These are the convolution kernel weights. It is the kernel offset. It is the kernel size. This is the maximum pooling operation;

[0025] The result is derived from formula (4). It is a single numerical value; produced by formula (4) These are numerous A composition Feature vectors; Formula (6) is used to extract The element with the largest value ;

[0026] A convolutional kernel of one size can extract 100 different Finally, 100 different merge into one The vector becomes a feature vector under a convolution kernel of a certain size;

[0027] F-Net's three different sizes of convolutional kernels will yield... eigenvectors A convolutional kernel of a certain size in A-Net will yield... eigenvectors ;

[0028] Using formula (3) Projected onto the corresponding position From and will and merge into one vector ;

[0029] Will As input to the classification layer, the final classification result is obtained. Classification vectors The model's prediction results are then obtained.

[0030] S2-2 Construct and initialize the Bert-class network

[0031] The BERT model and the RoBERTA model from the BERT class of networks were selected.

[0032] Select a copy of the encoder layer as a sub-network of A-Net;

[0033] Two fully connected layers are used as classification layers in the model (existing open-source network name). The probability magnitude and predicted classification are used to inform the recommendation system whether the evaluation has positive or negative sentiment.

[0034] F-Net has 12 encoder layers. A-Net's encoder is copied from F-Net's 12th encoder, and A-Net's input is the output of F-Net's 11th encoder layer.

[0035] Initializing a BERT-type model requires importing a pre-trained model; this pre-trained model is learned from unsupervised food delivery review corpora.

[0036] In BERT-like models, the number of encoders, the encoder output dimension, and the embedding output dimension are all fixed.

[0037] In this case, the parameter E of the Embedding layer is 768, and the dimension of the Encoder's output is 768.

[0038] The output of Embedding and each Encoder has the following structure:

[0039]

[0040] in,{ }yes A vector of dimension It is the output of each encoder layer, and the input of each encoder layer is the output of the encoder layer above it. A generator is produced after passing through 12 layers of encoders. eigenvectors Generate a in A-Net eigenvectors Operations in the feature projection layer are only for and of ; Projected features and of Combined into eigenvectors Finally The data is fed into the classification layer to obtain the classification results of the BERT-type model. ;

[0041] S3 Backpropagation

[0042] The result obtained from S2-2 The loss of the cross-entropy calculation model is used to update the parameters of the entire embedding layer, convolutional kernel parameters, encoder parameters, and classification layer parameters. During the update process, the learning rate used by A-Net is larger than that of F-Net. F-Net uses the Adam optimization strategy, while A-Net uses the SGD optimization strategy.

[0043] S4 Food Delivery Review Text Classification

[0044] The trained model is deployed on a server to perform the prediction process. When a user writes a comment, the data is sent to the food delivery platform server, and after calling the corresponding prediction interface, the trained model reads a piece of text data. The model calls the internal Tokenizer function to process the Chinese text, mapping it into an L-dimensional integer vector. Then, the vector is fed into the feature extraction layer of the convolutional neural network and the BERT-like network to obtain the feature vector of the sentence. Finally, the feature vector is fed into the classification layer to obtain the probability that the model predicts the sentence as a positive or negative evaluation.

[0045] Furthermore, in step 2-2, the method for informing the recommendation system whether this evaluation has positive or negative sentiment through probability magnitude and predicted classification is as follows;

[0046] The results calculated by the Bert model and the RoBERTA model In the vector, the value of subscript 0 represents the probability value that the model predicts the food delivery review belongs to positive sentiment, and the value of subscript 1 indicates the probability value that the food delivery review belongs to negative sentiment. The sum of the above two probability values ​​is 1. When the value of subscript 0 exceeds 0.5, it is considered that the model predicts the sentiment of the food delivery review to be positive, and vice versa.

[0047] Furthermore, following step S4, the following steps are also included:

[0048] S5 Food Delivery Review Text Classification Performance Evaluation:

[0049] Read the test data and use a text classification network model for food delivery reviews to classify the test data. Do not perform backpropagation during the classification process.

[0050] All the test data read is classified, and the accuracy of the model on the test data is obtained using standard accuracy calculation methods, thereby evaluating the model's predictive ability.

[0051] The gradient reversal and feature projection methods introduced in this invention can generate more feature vectors during the model's prediction and training process, thereby improving the model's ability to capture information and enabling the model to achieve higher classification accuracy in short texts such as food delivery reviews.

[0052] This invention utilizes an existing general framework to develop a deep learning model capable of performing complete prediction functions to classify food delivery review text data. In a normal network, the model updates its parameters based on gradients derived from the loss function. These gradients help the model fit more food delivery review data, thus achieving model generalization. However, data is diverse, and the overall training data cannot represent all data in the food delivery domain. Therefore, this invention incorporates backgrading into the model to reduce the model's fit to the training data, thereby improving its generalization ability. In this invention, the gradient inversion layer provides a function operation to invert the gradient, while feature projection uses vector operations to reduce the direct impact of backgradient feature vectors on the model, minimizing the damage to the model's classification ability caused by backgradient feature vectors. Furthermore, to achieve the purpose of using gradient inversion and feature projection layers, this invention designs a diamond-shaped structure. The original model portion in the diamond structure is called F-Net, and the network layers copied from the original model are called A-Net. The main purpose of the diamond structure is to extract the feature vector of the inverse gradient, so that the model can obtain two parts of the textual abstract information in one update, and the model can take into account the network layers affected by the gradient inversion layer when updating its own parameters.

[0053] Since this invention relates to multiple original models, such as CNN and BERT, the internal elements of the diamond structure of this invention are symmetrical. In short, the sub-networks of BERT-like models and the diamond-structured sub-networks designed in this invention use the same encoder, while the sub-network layers of the diamond structure in CNN-like models are convolutional neural network layers. That is, the internal elements of the diamond structure change with the original model. Furthermore, since the feature vectors generated by the backpropagation gradient cannot be directly used for classification, this invention additionally designs a feature projection layer to remove harmful information from the backpropagation gradient feature vectors and retain effective information. Finally, the processed feature vectors are concatenated with the feature vectors of the original model and then fed into the classification layer to obtain the model's prediction result. To better illustrate this invention, the specific formula for its gradient inversion layer is as follows:

[0054]

[0055]

[0056] In formula (1), GRL represents the gradient inversion layer. This represents the output of A-Net. This represents the output after passing through the gradient inversion layer. It is the same feature vector, and its internal values ​​are not changed by the gradient reversal layer. Instead, its gradient derivation equation is modified, as shown in formula (2). In formula (2), express For the original The partial derivatives, This controls the degree of gradient inversion, typically set between 0 and 1. In this invention, The initial value is 0, and it changes dynamically with model iterations. The higher the iteration number, the more dynamic the value changes. The closer it is to 1. Combining the two formulas, we know that the gradient reversal layer only changes the equation of gradient derivation, without modifying the original values ​​of the feature vectors. Therefore, in order to eliminate redundant data in the feature vectors extracted by the reverse gradient, a dedicated feature projection layer needs to be designed to cooperate.

[0057] The formula for the feature projection layer is as follows:

[0058]

[0059] In formula (3), This represents the feature vector extracted by A-Net. This represents the feature vector extracted by F-Net. This is the output of the feature projection layer. As can be seen from the formula, the feature projection layer will... Project to The feature space, in essence, is obtained through... To get A subset of, this subset is and Highly correlated. Attached Figure Description

[0060] Figure 1 The overall principle diagram of the method involved in this invention.

[0061] Figure 2 Network architecture diagram

[0062] Figure 3 Network flowchart

[0063] Figure 4 : Feature projection layer execution graph

[0064] Figure 5 Attention analysis diagram extracted from the last layer Encoder of the Bert-type network in this invention.

[0065] Figure 6 Attention analysis diagram extracted from the last layer of the original model's encoder. Detailed Implementation

[0066] To better understand the implementation of this invention, the following description, in conjunction with the accompanying drawings, will provide a more detailed explanation of the invention, thereby clarifying its scope and reducing the difficulty of understanding it.

[0067] Figure 1 This demonstrates the entire training process of the food delivery rating classification model. Figure 2 This demonstrates the architecture of the entire food delivery rating classification model. The original model represents the F-Net mentioned above, and the original model sub-networks and gradient inversion layers together constitute A-Net. Figure 3 This illustrates the flow of the model training and update phases. The black arrows represent the prediction process, and the blue arrows represent the process of updating model parameters. Since this invention relates to multiple models, the specific implementation steps of the model are simply described using a CNN.

[0068] Before any actual training, it is necessary to obtain order review data from food delivery platforms. Where N represents the number of training samples, This represents a training sample of a Chinese corpus. This represents the label for a single sample, with a value of 0 or 1.

[0069] Before training a CNN model, the model parameters need to be defined. In solving the problem of food delivery review text classification, this invention uses three different sized convolutional kernels in F-Net, each with 100 different initialization parameters, while A-Net uses one sized convolutional kernel, also with 100 different initialization parameters. Furthermore, Chinese character review data cannot be directly input into the CNN network; a preprocessing function needs to be defined for the Chinese data. Since this invention does not involve modifying the preprocessing, a general interface can be directly called to process the Chinese data.

[0070] The detailed execution steps for each part of the model are as follows:

[0071] 1. Data Preprocessing

[0072] Extract training data from the training dataset without replacement. The model uses a general-purpose function, Tokenizer, and a pre-trained embedding layer to process Chinese short sentences. This process maps individual characters or words to integers, and then maps the corresponding integers to E-dimensional vectors. Simultaneously, to meet the model's requirements, each sentence needs to be truncated and padded. When a sentence exceeds a certain length L, information exceeding L is truncated; if the entire sentence length is less than L, it is padded with 0s up to length L. This is followed by processing with Tokenizer and Embedding. In the form of:

[0073]

[0074] in, It is a processed matrix. Represents a The vectors. This tokenizer and embedding encoding method trains more expressive feature vectors from a large amount of unlabeled text, making text encoding more convenient and efficient.

[0075] 2. Define the network and initialize it.

[0076] a) Initialize the convolutional neural network

[0077] Based on the parameter settings mentioned above, three convolutional kernels are defined in F-Net, while one convolutional kernel is defined in A-Net. Simultaneously, A-Net also requires stacking a gradient inversion layer on top of the convolutional network layers to modify the gradient derivation equation of the feature vectors. In convolutional networks, pooling layers are paired with convolutional kernels. In the food delivery rating classification, this invention uses max pooling to obtain the most salient features extracted by the convolutional kernel.

[0078] b) Initialize the Bert class network

[0079] Since BERT-like networks already have open-source pre-trained models, and the parameters of these pre-trained models cannot be modified, we only need to directly import the pre-trained model without changing the original model. In BERT-like models, because the original F-Net model has numerous encoders, and each encoder has different classification capabilities, choosing which encoder layer to copy as a sub-network of A-Net is also a hyperparameter in this model. Hyperparameters refer to parameters that are pre-set by the model and are not modified by the model during training. (Therefore, this invention requires manually selecting 1 to 2 encoder copies of A-Net as sub-network layers.)

[0080] Although different A-Net models differ, the classification layers of each model are almost identical. CNN uses one fully connected layer, while BERT uses two fully connected layers as classification layers. Their function is to inform the recommendation system whether the evaluation has positive or negative sentiment by using probability magnitude and predicted classification.

[0081] 3. Perform network prediction and backpropagation.

[0082] c) Convolutional Neural Networks

[0083] The network's prediction requires a series of matrix operations to derive the feature vector of the text data. In convolutional neural networks, text convolution has only two layers: the first layer uses an embedding layer to obtain an abstract representation of the text. The second layer uses different convolutional kernels and max pooling layers to extract the feature vectors of the text. The prediction steps are formalized as follows:

[0084]

[0085]

[0086]

[0087] in, It is the output of the embedding layer. These are the convolution kernel weights. It is the kernel offset. It is the kernel size. This is the maximum pooling operation. The result is derived from formula (4). It is a single numerical value; produced by formula (4) These are numerous A composition Feature vectors; Formula (6) is then used for extraction. The element with the largest value in the middle. It's worth noting that each convolutional kernel of different sizes initializes 100 different convolutional kernels with different parameters. That is, a convolutional kernel of one size will extract 100 different... Finally, 100 different merge into one The vector is then the feature vector under a convolution kernel of a certain size. Therefore, it can be seen that the three different sizes of convolution kernels in F-Net will yield... eigenvectors A convolutional kernel of a certain size in A-Net will yield... eigenvectors .

[0088] Then, using formula (3) Projected onto the corresponding position From and will and merge into one vector Then As input to the classification layer, the final classification result is obtained. Classification vectors The model's predictions are then derived.

[0089] d) Transformer-type networks

[0090] The basic BERT-like model consists of 12 stacked encoder layers. In the BERT-like model of this invention, F-Net has 12 encoder layers, but A-Net only has 1 or 2 encoder layers. Because A-Net has fewer encoder layers than F-Net, it is necessary to manually select which encoder layer in F-Net is copied as the encoder in A-Net, and also manually select which encoder layer in F-Net the input to A-Net comes from. In this invention, the encoder of A-Net is copied from the 12th encoder of F-Net, and the input of A-Net is the output of the 11th encoder layer of F-Net.

[0091] Given the availability of mature third-party libraries and pre-trained models for BERT-type models, the initialization of BERT-type models in this invention will import pre-trained models. These pre-trained models are learned from a large unsupervised dataset and possess high generalization ability. In the food delivery review corpus, only minor adjustments to the BERT-type model are needed to achieve high accuracy. This minor adjustment is equivalent to model training, as the BERT-type model undergoes a considerable amount of pre-training, allowing it to achieve satisfactory results with only a few training iterations when imported into a specific context, such as food delivery corpora.

[0092] Because a pre-trained model is imported during training, the model size and parameters are fixed. For example, the parameter E of the embedding layer is fixed at 768, and the dimension of the encoder's output is also fixed at 768. Therefore... The output after embedding and each encoder is in the form of:

[0093]

[0094] in,{ }yes A vector of dimension It is the output of each encoder layer, and the input of each encoder layer is the output of the encoder layer above. This shows that... After passing through 12 layers of encoders, a result will be generated. eigenvectors Similarly, an A-Net will also be generated. eigenvectors However, in the feature projection layer, due to the special classification features of the Bert-type model, that is... Therefore, the operations of this invention in the feature projection layer will only target and of Then the projected features and of Combined into eigenvectors Finally The data is fed into the classification layer to obtain the classification results of the BERT-type model. .

[0095] Backpropagation process: derived from the model The cross-entropy algorithm calculates the model's loss and updates the entire model's parameters using the loss. During this update process, A-Net uses a larger learning rate than F-Net. Although updates occur simultaneously, they employ different update strategies: F-Net uses the Adam optimization strategy, while A-Net uses the SGD optimization strategy. It's worth noting that, as previously mentioned, GRL modifies the gradient derivation formula for the vectors; therefore, during backpropagation, no additional operations are needed, and only methods from third-party libraries need to be called.

[0096] Predicting real-time reviews on food delivery platforms:

[0097] After training all the data and saving the model, it can be deployed on the server to perform the prediction process. When a user writes a comment, the data is sent to the food delivery platform's server, and after calling the corresponding prediction interface, the trained model reads a text data, such as: "A very delicious hot pot, with ample portions and excellent taste." The model then calls its internal Tokenizer function to process the Chinese text, mapping it to an L-dimensional integer vector. This vector is then fed into the feature extraction layer (CNN and BERT modules) to obtain the sentence's feature vector. Finally, the feature vector is fed into the classification layer to obtain the model's predicted probability for the sentence. For example, if the model predicts a 0.98 probability of a positive evaluation, the result is positive; otherwise, it's negative.

[0098] 4. Evaluation Results

[0099] After the model has trained on all the training data, its weights are saved. During the evaluation phase, the model imports the saved weights and reads the test data, classifying it. This process does not involve backpropagation. After predicting all the test data, the model's accuracy on the test data is observed to evaluate its predictive ability. The accuracy calculation method is derived from machine learning evaluation methods and is a commonly used method in classification models. It calculates the proportion of correctly predicted samples in the pre-validation set to the total number of samples in the validation set. In addition to using standard accuracy to measure the model's predictive ability, to better illustrate the effectiveness of this invention, it abstracts and extracts the model's processing procedure for a given English sentence during the prediction phase and compares it with the original model, outputting the results of the two models calculating the attention between words in the same sentence. Figure 5 As can be seen, when the classification is correct, the model designed in this invention is able to better focus on words with negative emotions (“superficial”) in the sentence. In contrast, the attention between words in the original model becomes scattered and disorganized, failing to focus well on the key points of the text.

[0100] The horizontal and vertical axes of the attention heatmap are words in a sentence, and the brightness of the color represents the degree of attention paid to the words. The brighter the color, the stronger the correlation between the words.

[0101] from Figure 5 As can be seen, the model focuses more on "superficial," which aligns with the logic of ordinary people analyzing text sentiment. Figure 6 This is the attention analysis diagram extracted from the last layer of the Encoder in the original model. As can be seen from the diagram, the attention between words is scattered, and the connection between words is not well learned.

[0102] Practice has shown that this invention can improve the generalization ability of the model and enhance the classification ability of the original model.

Claims

1. A text classification method for food delivery reviews based on deep learning and gradient inversion, characterized in that, Includes the following steps: S1 Food Delivery Review Data Acquisition and Preprocessing: Obtain order review data from food delivery platforms Where N represents the number of training samples, This represents a training sample from a Chinese corpus. This represents the label for a single sample, with a value of either 0 or 1. Extract training data from the training dataset without replacement. It uses a general function Tokenizer and a pre-trained embedding layer to process Chinese short sentences, mapping individual characters or words to integers, and then mapping the corresponding integers to E-dimensional vectors; Each sentence is truncated and expanded: if a sentence exceeds a certain length L, the information exceeding L is truncated; if the length of the entire sentence is less than L, it is filled with 0 to the length L. After processing by Tokenizer and Embedding, we get: ； in, It is a processed matrix. Represents a ; S2 constructs a network model for classifying text classification of food delivery reviews. S2-1 Construct and initialize the convolutional neural network Three convolutional kernels are defined in the F-Net convolutional network, and one convolutional kernel is defined in the A-Net convolutional network; The A-Net convolutional network also requires stacking a gradient inversion layer on top of the convolutional network layers to modify the gradient derivation equation of the feature vectors; In F-Net and A-Net convolutional networks, pooling layers are paired with convolutional kernels. Max pooling is used to obtain the most salient features extracted by the convolution kernel for food delivery review classification; In convolutional neural networks, text convolution has two layers. The first layer uses an embedding layer to obtain an abstract representation of the text. The second layer uses convolutional kernels of different sizes and max pooling layers to extract feature vectors from the text. The prediction step is formalized as follows: ； ； ； in, It is the output of the embedding layer. These are the convolution kernel weights. It is the kernel offset. It is the kernel size. This is the maximum pooling operation; The result is derived from formula (4). It is a single numerical value; produced by formula (5) These are numerous A feature vector is formed; formula (6) is used to extract The element with the largest value ; A convolutional kernel of one size can extract 100 different Finally, 100 different merge into one The vector becomes a feature vector under a convolution kernel of a certain size; F-Net's three different sizes of convolutional kernels will yield... eigenvectors A convolutional kernel of a certain size in A-Net will yield... eigenvectors ; Will Projected onto the corresponding position From and will and merge into one vector ; Will As input to the classification layer, the final classification result is obtained. Classification vectors The model's prediction results are then obtained. S2-2 Construct and initialize the Bert-class network The BERT model and the RoBERTA model from the BERT class of networks were selected. Select a copy of the encoder layer as a sub-network of A-Net; Two fully connected layers are used as the classification layers of the model. The probability magnitude and predicted classification are used to inform the recommendation system whether the evaluation has positive or negative sentiment. F-Net has 12 encoder layers. A-Net's encoder is copied from F-Net's 12th encoder, and A-Net's input is the output of F-Net's 11th encoder layer. Initializing a BERT-type model requires importing a pre-trained model; this pre-trained model is learned from unsupervised food delivery review corpora. In BERT-like models, the number of encoders, the encoder output dimension, and the embedding output dimension are all fixed. In this context, the parameter E of the Embedding layer is 768, and the dimension of the Encoder's output is 768. The output of Embedding and each Encoder has the following structure: ； in,{ }yes A vector of dimension It is the output of each encoder layer, and the input of each encoder layer is the output of the encoder layer above it. A generator is produced after passing through 12 layers of encoders. eigenvectors ; Generate a in A-Net eigenvectors Operations in the feature projection layer are only for and of ; Projected features and of Combined into eigenvectors Finally The data is fed into the classification layer to obtain the classification results of the BERT-type model. ; S3 Backpropagation The result obtained from S2-2 The cross-entropy calculation model's loss is used to update the parameters of the entire embedding layer, convolutional kernel parameters, encoder parameters, and classification layer parameters. During the update process, A-Net uses a larger learning rate than F-Net. F-Net uses the Adam optimization strategy, while A-Net uses the SGD optimization strategy. The gradient inversion layer controls the degree of gradient inversion through the parameter λ, and the value of λ dynamically increases with the number of model iterations. S4 Food Delivery Review Text Classification The trained model is deployed on the server to perform the prediction process. When a user writes a comment, the data is sent to the food delivery platform server, and after calling the corresponding prediction interface, the trained model reads a piece of text data. The model calls the internal Tokenizer function to process the Chinese text, mapping it into an L-dimensional integer vector. Then, the vector is fed into the feature extraction layer of the convolutional neural network and the BERT-like network to obtain the feature vector of the sentence. Finally, the feature vector is fed into the classification layer to obtain the probability that the model predicts the sentence as a positive or negative evaluation. In S2-2, the method for informing the recommendation system whether a review has positive or negative sentiment by using probability magnitude and predicted classification is as follows; The results calculated by the Bert model and the RoBERTA model In the vector, the value of subscript 0 represents the probability value that the model predicts the food delivery review belongs to positive sentiment, and the value of subscript 1 indicates the probability value that the food delivery review belongs to negative sentiment. The sum of the above two probability values ​​is 1. When the value of subscript 0 exceeds 0.5, it is considered that the model predicts the sentiment of the food delivery review to be positive, and vice versa.

2. The method for classifying food delivery review text based on deep learning and gradient inversion as described in claim 1, characterized in that, Following step S4, the following steps are also included: S5 Food Delivery Review Text Classification Performance Evaluation: Read the test data and use a text classification network model for food delivery reviews to classify the test data. Do not perform backpropagation during the classification process. All the test data read is classified, and the accuracy of the model on the test data is obtained using standard accuracy calculation methods, thereby evaluating the model's predictive ability.

Images