Model training, comment information ranking methods, devices, equipment and storage media

Abstract

This invention provides a model training method, apparatus, device, and storage medium for ranking comment information, relating to the field of data processing technology. The specific implementation scheme involves: acquiring sample features of sample comment information and labeled scores of sample comment information relative to push priority; wherein the sample features include at least one of the following features: a first multimedia content feature; a first interaction feature; a first text content feature; and a first publisher feature; inputting the sample features into a preset neural network model to obtain the estimated scores of the sample comment information relative to push priority corresponding to each sample feature output by the neural network model, and the ranking result of the sample comment information output based on the estimated scores; adjusting the network parameters of the neural network model based on the difference between the estimated scores and the labeled scores to obtain a comment information ranking model. Applying the scheme provided by this invention can improve the fluidity of comments pushed to users.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to methods, apparatus, devices and storage media for model training and comment information sorting. Background Technology

[0002] With the development of internet technology, users can access increasingly more information through various content platforms, such as videos, audio, and text. After browsing this information, users often post comments to express their opinions and feelings. They can also like or comment on other users' posts. To attract users and increase user engagement, content platforms often sort the comments on various content and then push comments to users according to the sorting results.

[0003] In existing technologies, comments are generally sorted according to the number of likes they receive, so that comments with a high number of likes are pushed to users first.

[0004] However, the number of likes on a comment is accumulated over time, and new comments often have fewer likes. This makes it difficult for new comments with topicality to be prioritized and pushed to users, resulting in fixed and stagnant comments pushed to users, which can reduce users' enthusiasm for participating in comments. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and storage medium for model training and comment information sorting, so as to improve the fluidity of comments pushed to users. The specific technical solution is as follows:

[0006] In a first aspect of this invention, a model training method is provided, the method comprising:

[0007] The process involves obtaining sample features of sample comment information and a labeled score for the sample comment information in relation to push priority. The sample features include at least one of the following: a first multimedia content feature associated with the sample comment information; a first interaction feature of the sample comment information; a first text content feature of the sample comment information; and a first publisher feature of the publisher of the sample comment information. The sample features are then input into a preset neural network model to obtain the estimated score for each sample feature output by the neural network model in relation to push priority, and a ranking result for the sample comment information output based on the estimated score. Based on the difference between the estimated score and the labeled score, the network parameters of the neural network model are adjusted to obtain a comment information ranking model.

[0008] In a second aspect of the invention, a method for sorting comment information is provided, the method being applicable to a business server deploying a comment sorting model, comprising:

[0009] A pool of comment information to be sorted is obtained, wherein each comment information in the pool is associated with the same target multimedia content; for the comment information in the pool, at least one of the following features is obtained, and the obtained features are input into a comment sorting model pre-trained according to the above model training method to sort the comment information in the pool, thereby obtaining the sorting result: the content features of the target multimedia content; the interaction features of each comment information; the text content features of each comment information; and the publisher features corresponding to each comment information.

[0010] In a third aspect of the invention, a model training apparatus is provided, the apparatus comprising:

[0011] A sample feature acquisition module is used to acquire sample features of sample comment information and labeled scores of the sample comment information for push priority. The sample features include at least one of the following: a first multimedia content feature associated with the sample comment information; a first interaction feature of the sample comment information; a first text content feature of the sample comment information; and a first publisher feature of the publisher of the sample comment information. A sample feature input module is used to input the sample features into a preset neural network model to obtain the estimated scores of the sample comment information for push priority corresponding to each sample feature output by the neural network model, and the ranking result of the sample comment information output based on the estimated scores. A parameter adjustment module is used to adjust the network parameters of the neural network model based on the difference between the estimated scores and the labeled scores to obtain a comment information ranking model.

[0012] In a fourth aspect of the invention, a comment information sorting device is also provided, the device comprising:

[0013] The comment information pool acquisition module is used to acquire a pool of comment information to be sorted, wherein each comment in the comment information pool is associated with the same target multimedia content; the comment information sorting module is used to acquire at least one of the following features for the comment information in the comment information pool, input the obtained features into a comment sorting model pre-trained according to the above model training method, and sort the comment information in the comment information pool to obtain the sorting result: the content features of the target multimedia content; the interaction features of each comment; the text content features of each comment; and the publisher features corresponding to each comment.

[0014] In a fifth aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0015] Memory, used to store computer programs;

[0016] The processor, when executing programs stored in memory, implements the aforementioned model training method or comment information sorting method.

[0017] In a sixth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements the above-described model training method or comment information sorting method.

[0018] In a seventh aspect of the present invention, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to execute the above-described model training method or comment information sorting method.

[0019] As can be seen from the above, when training the model using the solution provided in this embodiment of the invention, the sample features of the sample comment information are first obtained. Then, the obtained sample features are input into a preset neural network model. Based on the difference between the model output and the sample annotation, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The sample features can be multiple features among the following: the first multimedia content feature associated with the sample comment information, the first interaction feature of the sample comment information, the first text content feature of the sample comment information, and the first publisher feature of the sample comment information publisher. This allows the model to learn the relationship between the above features and the rating of the comment information during training. After training, a model for ranking comment information based on the above ratings can be obtained. When using this model to rank comments, compared to using only the number of likes, it helps prevent the top-ranked comments from always having a high number of likes, improving the fluidity of comments during ranking. Furthermore, after the ranked comments are pushed to the user's client, it helps improve the fluidity of the comments pushed to the user. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0021] Figure 1 A flowchart illustrating the first model training method provided in an embodiment of the present invention;

[0022] Figure 2A flowchart illustrating the second model training method provided in this embodiment of the invention;

[0023] Figure 3 A flowchart illustrating the third model training method provided in this embodiment of the invention;

[0024] Figure 4 A flowchart illustrating a method for sorting comment information provided in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of a comment information sorting device provided in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0029] Because existing comment push schemes tend to solidify the comments pushed to users, reducing users' enthusiasm for interacting with comments, this invention provides a model training, comment information sorting method, apparatus, device, and storage medium to improve the fluidity of comments pushed to users.

[0030] In one embodiment of the present invention, a model training method is provided, the method comprising:

[0031] The sample features of the sample comment information are obtained, as well as the labeled score of the sample comment information for push priority, wherein the sample features include at least one of the following features:

[0032] The first multimedia content feature associated with the sample comment information;

[0033] The first interactive feature of the sample comment information;

[0034] First text content features of sample comment information;

[0035] The primary publisher characteristic of the sample comment information publishers;

[0036] The sample features are input into a preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the sorting result of the sample comment information output according to the estimated score.

[0037] Based on the difference between the estimated score and the labeled score, the network parameters of the neural network model are adjusted to obtain a comment information ranking model.

[0038] As can be seen from the above, when training the model using the solution provided in this embodiment of the invention, the sample features of the sample comment information are first obtained. Then, the obtained sample features are input into a preset neural network model. Based on the difference between the model output and the sample annotation, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The sample features can be multiple features among the following: the first multimedia content feature associated with the sample comment information, the first interaction feature of the sample comment information, the first text content feature of the sample comment information, and the first publisher feature of the sample comment information publisher. This allows the model to learn the relationship between the above features and the rating of the comment information during training. After training, a model for ranking comment information based on the above ratings can be obtained. When using this model to rank comments, compared to using only the number of likes, it helps prevent the top-ranked comments from always having a high number of likes, improving the fluidity of comments during ranking. Furthermore, after the ranked comments are pushed to the user's client, it helps improve the fluidity of the comments pushed to the user.

[0039] The model training method provided by the embodiments of the present invention will be described in detail below through specific examples.

[0040] See Figure 1 , Figure 1 This is a flowchart illustrating the first model training method provided in an embodiment of the present invention. The method includes the following steps S101-S103.

[0041] Step S101: Obtain the sample features of the sample comment information, and the labeled score of the sample comment information for the push priority.

[0042] The sample features include at least one of the following features:

[0043] The first multimedia content feature associated with the sample comment information;

[0044] The first interactive feature of the sample comment information;

[0045] First text content features of sample comment information;

[0046] The primary publisher characteristic of the sample comment information publishers.

[0047] The following section will first explain the sample characteristics of the above-mentioned dimensions of the sample comment information.

[0048] (1) The first multimedia content feature associated with the sample comment information.

[0049] The multimedia content associated with the sample review information can be movies, TV series, short videos, e-books, etc. The first multimedia content feature can take different forms depending on the multimedia content; see the examples below:

[0050] Taking multimedia content such as movies, TV series, and short videos as examples, the first feature of multimedia content can be video length, video type tags, video style tags, cast and crew names, plot summary, and the name of the background music in the video.

[0051] Taking multimedia content as an example, the first feature of multimedia content can be the number of words in the e-book, the author's name, a brief description of the content, e-book style tags, e-book type tags, etc.

[0052] As can be seen from the examples above, the first multimedia content feature can characterize the information of the multimedia content itself. Depending on the multimedia content, the obtained first multimedia content feature will differ. Examples of other first multimedia content features will not be provided for each type of multimedia content.

[0053] Specifically, the content providing platform stores information about multimedia content, so the first multimedia content feature can be extracted from the information stored on the content providing platform.

[0054] (2) The first interactive feature of the sample comment information.

[0055] The first interaction feature of the sample comment information is used to characterize the user interaction of the sample comment information.

[0056] Specifically, for each preset interactive item, interactive sub-features representing the interaction between the user and each sample comment can be obtained, and the obtained interactive sub-features can be used as the first interactive feature.

[0057] The preset interactive items can be user behaviors in response to comments, such as replying, copying, liking, disliking, and reporting. The interactive sub-features can be the user interaction frequency of the preset interactive items, such as the average number of user likes per hour for a comment; or the user interaction quantity of the preset interactive items, such as the number of user replies to a comment in the last three days.

[0058] (3) First text content features of sample comment information.

[0059] In one implementation, the first text content feature includes semantic sub-features of the sample comment information.

[0060] The semantic sub-features described above represent at least one of the following pieces of information corresponding to the sample comment information:

[0061] Text fluency;

[0062] Count of text characters;

[0063] The emotional tone of the text content;

[0064] Is the text meaningless?

[0065] Specifically, semantic analysis can be performed on the text related to the sample comment information to obtain semantic sub-features representing the aforementioned information. The text related to the sample comment information can include the text of the sample comment information itself. In some cases, there may be user replies to a sample comment, so the text related to the sample comment information can also include the text of the replies.

[0066] The embodiments of the present invention do not limit the method of semantic analysis of the text related to the sample comment information. For example, the semantic analysis of the above text can be performed based on an NLP (Natural Language Processing) model, such as a Hidden Markov Model, an LSTM (Long Short-Term Memory) network model, etc.

[0067] In this way, semantic sub-features can represent a variety of different textual semantic information, thereby improving the comprehensiveness of the information represented by semantic sub-features.

[0068] (4) The characteristics of the first publisher of the sample comment information publishers.

[0069] The first publisher feature of the sample comment information publisher represents the self-characteristics of the sample comment information publisher.

[0070] Taking multimedia content such as movies or TV series as an example, the characteristics of the first publisher can be the publisher's gender, age, celebrity preferences, viewing preferences, number of likes received in the past, and whether they have passed celebrity verification.

[0071] Specifically, the aforementioned first multimedia content feature can be obtained from the user profiles recorded by the multimedia content providing platform.

[0072] In one embodiment of the present invention, the sample features of the sample comment information may include features of all four dimensions mentioned above.

[0073] The following explains the labeling and scoring of sample comment information based on push priority.

[0074] The above-mentioned ratings represent the priority of sample comments in terms of their push priority, which can also be understood as representing the potential of sample comments to become popular comments.

[0075] For example, the higher the rating of a sample comment, the higher its push priority and the greater its potential to become a hot comment; conversely, the lower the rating of a sample comment, the lower its push priority and the smaller its potential to become a hot comment.

[0076] Step S102: Input the sample features into the preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the sorting result of the sample comment information output according to the estimated score.

[0077] The embodiments of the present invention do not limit the specific network structure of the above-mentioned preset neural network model.

[0078] The aforementioned neural network model can be a Wide & Deep model, a DNN (Deep Neural Networks) model, etc.

[0079] After the sample features are input into the preset neural network model, the neural network model will output the estimated score of the sample comment information corresponding to each sample feature for the push priority, and will output the ranking result determined according to the estimated score corresponding to the sample comment information.

[0080] Specifically, the above sorting result can be obtained by sorting the sample comment information from high to low according to the estimated scores corresponding to the sample comment information.

[0081] Step S103: Based on the difference between the estimated score and the labeled score, adjust the network parameters of the neural network model to obtain the comment information ranking model.

[0082] Specifically, the loss value between the estimated score and the labeled score can be calculated, and the network parameters of the neural network model can be adjusted based on the loss value until the loss value is less than the preset value or the model has been trained to the preset number of times.

[0083] The above can be calculated using a loss function, such as the mean squared error loss function or the cross-entropy loss function.

[0084] As can be seen from the above, when training the model using the solution provided in this embodiment of the invention, the sample features of the sample comment information are first obtained. Then, the obtained sample features are input into a preset neural network model. Based on the difference between the model output and the sample annotation, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The sample features can be multiple features among the following: the first multimedia content feature associated with the sample comment information, the first interaction feature of the sample comment information, the first text content feature of the sample comment information, and the first publisher feature of the sample comment information publisher. This allows the model to learn the relationship between the above features and the rating of the comment information during training. After training, a model for ranking comment information based on the above ratings can be obtained. When using this model to rank comments, compared to using only the number of likes, it helps prevent the top-ranked comments from always having a high number of likes, improving the fluidity of comments during ranking. Furthermore, after the ranked comments are pushed to the user's client, it helps improve the fluidity of the comments pushed to the user.

[0085] exist Figure 1 Based on the illustrated embodiment, multiple features can be used simultaneously to train the preset neural network model. Furthermore, when adjusting the model's network parameters, adjustments can be made not only based on the difference between the model's predicted score and the labeled score, but also based on the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features. In view of the above, this embodiment of the invention provides a second model training method.

[0086] See Figure 2 , Figure 2 This is a flowchart illustrating the second model training method provided in an embodiment of the present invention. The method includes steps S201-S203.

[0087] Step S201: Obtain the sample features of the sample comment information, and the labeled score of the sample comment information for the push priority.

[0088] The sample features include: the first multimedia content feature associated with the sample comment information, the first interactive feature of the sample comment information, the first text content feature of the sample comment information, and the first publisher feature of the sample comment information publisher.

[0089] The meaning and acquisition method of the above features have been described above. Figure 1 Step S101 in the illustrated embodiment is explained and will not be repeated here.

[0090] Step S202: Input the sample features into the preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the ranking result of the sample comment information output according to the estimated score.

[0091] The estimated score is: after the neural network model obtains the first sub-score of the push priority of sample comment information using the first multimedia content feature, the second sub-score of the push priority of sample comment information using the first interaction feature, the third sub-score of the push priority of sample comment information using the first text content feature, and the fourth sub-score of the push priority of sample comment information using the first publisher feature, the comprehensive score of the push priority of sample comment information obtained from the first sub-score, the second sub-score, the third sub-score and the fourth sub-score is obtained.

[0092] Specifically, the overall score can be the sum of the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score, or it can be the average of the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score.

[0093] The model outputs a predicted score that is a comprehensive score based on the first sub-score corresponding to the first multimedia content feature, the second sub-score corresponding to the first interaction feature, the third sub-score corresponding to the first text content feature, and the fourth sub-score corresponding to the first publisher feature. By training the model with multiple features, the model can obtain a predicted score based on multiple features, which improves the comprehensiveness of the predicted score and thus improves the comprehensiveness of the model when ranking comment information based on the predicted score.

[0094] In addition, sub-scores for push priority of the above features were predicted based on the first multimedia content feature, the first interaction feature, the first text content feature, and the first publisher feature, respectively. The sub-scores are not affected by other features and each represents the priority of different dimensions of features for push. Then, the sub-scores are weighted according to the weight coefficients corresponding to each feature. The resulting score for push priority of comments can comprehensively represent the priority of the above-mentioned features for push. Therefore, the obtained score can more comprehensively represent the priority of comments for push.

[0095] Step S203: Based on the weight coefficients corresponding to multimedia content features, interaction features, text content features, and publisher features, as well as the difference between the predicted score and the labeled score, the network parameters of the neural network model are adjusted to obtain the comment information ranking model.

[0096] First, we will explain how the weight coefficients for the aforementioned multimedia content features, interactive features, text content features, and publisher features are obtained.

[0097] In one implementation, the aforementioned weighting coefficients can be determined based on the promotion goals of multimedia content. For example, if the first interaction feature f1 is a recent interaction frequency feature, and the promotion goal for the target audience is to promote comments with a high recent interaction frequency, then the weighting coefficient of the first interaction feature f1 can be set to a higher coefficient; similarly, if the first publisher feature f2 is a celebrity verification feature, and the promotion goal for the target audience is to promote comments from publishers who have passed celebrity verification, then the weighting coefficient of the first publisher feature f2 can be set to a higher coefficient.

[0098] In another implementation, the most active positive sample comments from the first sample object being commented on within a preset time period regarding preset interactive items can be collected. Based on the second multimedia content features, second interaction features, second text content features, and second publisher features corresponding to the positive sample comments, the total score of the positive sample comments for push priority is predicted. Then, the weight coefficients of the above features are obtained such that the total score represents the positive sample comment as a hot comment. Detailed implementation methods will be provided later. Figure 3 The illustrated embodiment will not be described in detail here.

[0099] After obtaining the weight coefficients of each feature, the network parameters of the neural network model can be adjusted based on the weight coefficients of each feature and the difference between the predicted score and the labeled score output by the model.

[0100] Among them, the method of adjusting the network parameters of the neural network model based on the difference between the predicted score and the labeled score has been described above. Figure 1 The steps in step S103 of the illustrated embodiment are described and will not be repeated here.

[0101] The following explains how to adjust the network parameters of a neural network model based on the weight coefficients of each feature.

[0102] Specifically, the aforementioned weight coefficients can be set for the output layer of the neural network model that outputs the sub-scores corresponding to each feature, so that the final output of the model for each feature is the product of the sub-score before adjustment and the weight coefficient.

[0103] It should be noted that the embodiments of the present invention do not limit the order in which the network parameters are adjusted based on the weight coefficients of each feature and the network parameters are adjusted based on the difference between the estimated score and the labeled score. Of course, the two adjustments can be performed simultaneously.

[0104] As can be seen from the above, this approach not only adjusts the network parameters based on the difference between the model's predicted score and the labeled score, but also adjusts the network parameters according to the weight coefficients corresponding to multimedia content features, interaction features, text content features, and publisher features. By adjusting the weight coefficients to adjust the importance of different features to the predicted score, the adjusted model outputs sub-scores for each feature according to the aforementioned weight coefficients, improving the rationality of the predicted score obtained based on the aforementioned sub-scores. This, in turn, helps to improve the rationality of the model when ranking comment information based on the predicted score.

[0105] exist Figure 2 Based on the illustrated embodiment, when obtaining the weight coefficients corresponding to the features, the most active positive sample comments of the first sample object being commented on within a preset time period regarding the interaction with preset interactive items can be collected. Based on the second multimedia content feature, second interaction feature, second text content feature, and second publisher feature corresponding to the positive sample comments, the total score of the aforementioned positive sample comments for push priority is predicted. Then, the weight coefficients of the aforementioned features are obtained such that the total score characterizes the aforementioned positive sample comments as highly rated. In view of the above, this embodiment of the invention provides a third model training method.

[0106] See Figure 3 , Figure 3 This is a flowchart illustrating the third model training method provided in an embodiment of the present invention. The method includes steps S301-S308.

[0107] Step S301: Obtain the sample features of the sample comment information, and the labeled score of the sample comment information for the push priority.

[0108] Step S302: Input the sample features into the preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the ranking result of the sample comment information output according to the estimated score.

[0109] Steps S301 and S302 described above are the same as those mentioned earlier. Figure 2 In the illustrated embodiment, steps S201-S202 are the same and will not be repeated here.

[0110] Step S303: Collect the most active positive sample comments from the first sample object being commented on within a preset time period for the preset interactive items.

[0111] The first sample object mentioned above can be the aforementioned multimedia content.

[0112] Specifically, positive sample comments can be identified in the following ways.

[0113] In one implementation, a preset number of comments with the highest interaction frequency for each preset interactive item within a preset time period can be selected as the most active positive sample comments. The preset time period and preset number can be set by staff based on experience.

[0114] In another implementation, a preset number of comments with the highest number of interactions for each preset interactive item within a preset time period can be used as the positive sample comments with the most active interaction.

[0115] Step S304: Obtain the second multimedia content features associated with the first sample object, the second interaction features of the positive sample comment, the second text content features of the positive sample comment, and the second publisher features of the publisher of the positive sample comment.

[0116] Specifically, the meanings and acquisition methods of the aforementioned second multimedia content features, second interactive features, second text content features, and second publisher features are similar to those of the first multimedia content features, first interactive features, first text content features, and first publisher features described in the foregoing embodiments, except that the names are different and the corresponding objects are different, which will not be repeated here.

[0117] Step S305: Based on the second multimedia content features, the second interaction features, the second text content features, and the second publisher features, predict the fifth, sixth, seventh, and eighth sub-ratings of positive sample comments in relation to push priority.

[0118] Specifically, the fifth, sixth, seventh, and eighth sub-scores mentioned above can be predicted in the following ways.

[0119] In one implementation, the sub-score can be obtained based on the similarity between the second multimedia content feature, the second interaction feature, the second text content feature, the second publisher feature, and the preset feature. For details of the implementation, please refer to the following embodiments, which will not be described in detail here.

[0120] In another implementation, the second multimedia content feature, the second interaction feature, the second text content feature, and the second publisher feature can be input into the aforementioned trained comment information ranking model. The comment information ranking model can output the fifth, sixth, seventh, and eighth sub-ratings of positive sample comments in relation to push priority.

[0121] Step S306: Obtain the weighting coefficient that makes the first total score greater than the first preset score corresponding to the positive sample comments.

[0122] The first overall score is calculated by weighting the fifth, sixth, seventh, and eighth sub-scores according to their respective weighting coefficients.

[0123] The aforementioned first preset score can be set by staff based on their experience.

[0124] If the first total score of the positive sample reviews is greater than the first preset score, it can be considered that the predicted score for the positive sample reviews is in line with expectations. Therefore, the weight coefficients corresponding to each feature at this time can be obtained.

[0125] Specifically, the above weighting coefficients can be determined in the following ways.

[0126] In one implementation, the weight coefficients of the fifth, sixth, seventh, and eighth sub-scores can be continuously changed until the sub-scores corresponding to the features are weighted according to the weight coefficients, and the resulting first total score is greater than the first preset score, thus obtaining the weight coefficients corresponding to the fifth, sixth, seventh, and eighth sub-scores at this time.

[0127] In another implementation, the aforementioned weighting coefficients can be determined by a review rating model. This review rating model can be a neural network model different from the aforementioned review information ranking model. Specifically, based on the loss value generated when the review rating model outputs the first total score, the model parameters of the review rating model can be adjusted until the loss value is less than a preset loss value, thus obtaining the weighting coefficients corresponding to the fifth, sixth, seventh, and eighth sub-ratings determined by the review rating model at this point.

[0128] The first total score is calculated by weighting the fifth, sixth, seventh, and eighth sub-scores according to their respective weight coefficients. If the first total score is greater than the first preset score corresponding to the positive sample comments, it indicates that the first total score output by the above comment scoring model for positive sample comments is as expected. Therefore, it can be considered that weighting each sub-score according to the weight coefficients determined by the comment scoring model at this time is beneficial to improving the rationality of the comment scoring.

[0129] Step S307: Determine the obtained weight coefficients as the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, respectively.

[0130] The fifth sub-score is the score for the second multimedia content feature in terms of push priority; the sixth sub-score is the score for the second interaction feature in terms of push priority; the seventh sub-score is the score for the second text content feature in terms of push priority; and the eighth sub-score is the score for the second publisher feature in terms of push priority. Therefore, the weight coefficients of the multimedia content feature can be determined as the weight coefficients corresponding to the fifth sub-score, the weight coefficients of the interaction feature can be determined as the weight coefficients corresponding to the sixth sub-score, the weight coefficients of the text content feature can be determined as the weight coefficients corresponding to the seventh sub-score, and the weight coefficients of the publisher feature can be determined as the weight coefficients corresponding to the eighth sub-score.

[0131] Step S308: Based on the weight coefficients corresponding to multimedia content features, interaction features, text content features, and publisher features, as well as the difference between the estimated score and the labeled score, adjust the network parameters of the neural network model to obtain the comment information ranking model.

[0132] The above steps are the same as those described above. Figure 2 Step S203 is the same in the illustrated embodiment, and will not be repeated here.

[0133] This ensures that the determined weight coefficients align with the characteristics of positive sample comments with active interaction. Adjusting the network parameters of the comment ranking model according to these weight coefficients helps ensure that the predicted score output by the model accurately reflects the likelihood of active interaction in the comment, i.e., the probability that the comment is a "hot" comment. Ranking the comments according to the predicted score helps identify comments with a higher probability of active interaction as "hot" comments, thus ranking them higher and improving the model's rationality in ranking comment information.

[0134] The following explains another method for predicting the fifth, sixth, seventh, and eighth sub-scores.

[0135] (1) The fifth sub-score corresponding to the second multimedia content feature.

[0136] Specifically, the fifth sub-score can be determined based on the similarity between the second multimedia content feature and the first preset feature. Taking video as an example, if the second multimedia content feature is a video duration feature, and the first preset feature could be 20 minutes, then the higher the similarity between the second multimedia content feature and the preset feature (i.e., the closer the second multimedia content feature is to 20 minutes), the higher the fifth sub-score. For example, if the second multimedia content feature f3 is a video duration of 18 minutes, the fifth sub-score could be 90 points; if the second multimedia content feature f4 is a video duration of 10 minutes, the fifth sub-score could be 50 points. Similarly, if the second multimedia content feature is a video type feature, the first preset feature could be a video type tag such as "suspense" or "comedy," etc., which will not be listed here. The preset features can be set by staff based on popular trends.

[0137] (2) The sixth sub-rating corresponding to the second interactive feature.

[0138] Specifically, the first score of the interaction sub-feature for the push priority can be predicted according to the correspondence between the information represented by the second interaction feature and the first preset score. For example, if the second interaction feature is the interaction frequency feature, the second interaction feature f5 indicates that the interaction frequency of the comment in the past 3 days is 100 times / hour, and the corresponding first score could be a; the second interaction feature f6 indicates that the interaction frequency of the comment in the past 3 days is 100 times / hour, and the corresponding first score could be b, etc.

[0139] (3) The seventh sub-score corresponding to the second text content feature.

[0140] Specifically, the seventh sub-score can be determined based on the similarity between the second text content features and the second preset features. Taking multimedia content such as video as an example, the second preset features could be the names of the video's cast and crew, the names of characters in the video, recent popular phrases, etc.

[0141] (4) The eighth sub-rating corresponding to the second publisher feature.

[0142] Specifically, the eighth sub-rating can be predicted based on the correspondence between the information represented by the second publisher's features and the second preset rating. For example, if the second publisher's feature is the total number of likes received by the publisher, then the second publisher feature f7 indicates that the publisher has received a total of 1000 likes, and the corresponding third sub-rating is c; if the second publisher's feature is the publisher's celebrity certification feature, then the second publisher feature f8 indicates that the publisher has passed celebrity certification, and the corresponding third sub-rating is d, etc.

[0143] In one embodiment of the present invention, in addition to determining the weight coefficients corresponding to multimedia content features, interaction features, text content features, and publisher features based on the features of positive sample comments, the aforementioned weight coefficients can also be determined based on the features of negative sample comments. This will be explained below through steps A-E.

[0144] Step A: Collect the most active positive comments from the first sample object being commented on within a preset time period regarding the preset interactive items, and the negative sample comments from the second sample object being commented on regarding negative sentiment.

[0145] The method for collecting the most active positive sample comments from the first sample object being commented on regarding the preset interactive items is detailed in step S306 above, and will not be repeated here.

[0146] The second sample object mentioned above can be the same as the first sample object or different from the second sample object.

[0147] Specifically, negative sample comments can be identified in the following ways.

[0148] In one implementation, the number of downvotes for each comment on the second sample object can be obtained, and comments with a number of downvotes greater than a preset number are identified as the aforementioned negative sample comments. The preset number can be set by staff based on experience.

[0149] In another implementation, the semantic features of each comment targeting the second sample object can be analyzed, and the sentiment tendency of each comment can be determined based on these semantic features. Comments with a negative sentiment tendency can then be identified as negative sample comments. The method for analyzing the semantic features of each comment targeting the second sample object can be found in the previously described method for analyzing the semantic features of text related to comments, and will not be repeated here.

[0150] Step B: Obtain the second multimedia content features associated with the first sample object, the second interaction features of the positive sample comments, the second text content features of the positive sample comments, and the second publisher features of the publisher of the positive sample comments; and obtain the third multimedia content features associated with the second sample object, the third interaction features of the negative sample comments, the third text content features of the negative sample comments, and the third publisher features of the publisher of the negative sample comments.

[0151] Specifically, the meanings and acquisition methods of the aforementioned third multimedia content features, third interactive features, third text content features, and third publisher features are similar to those of the first multimedia content features, first interactive features, first text content features, and first publisher features described in the foregoing embodiments, except that the names are different and the corresponding objects are different, so they will not be repeated here.

[0152] Step C: Based on the second multimedia content feature, the second interaction feature, the second text content feature, and the second publisher feature, predict the fifth, sixth, seventh, and eighth sub-ratings of positive sample comments regarding push priority, respectively. And based on the third multimedia content feature, the third interaction feature, the third text content feature, and the third publisher feature, predict the ninth, tenth, eleventh, and twelfth sub-ratings of negative sample comments regarding push priority, respectively.

[0153] The method for predicting the fifth, sixth, seventh, and eighth sub-scores in this step has been explained in step S305 above, and will not be repeated here.

[0154] Specifically, the ninth, tenth, eleventh, and twelfth sub-scores mentioned above can be predicted in the following ways.

[0155] In one implementation, the prediction methods for the ninth, tenth, eleventh, and twelfth sub-scores can be derived from the prediction methods for the fifth, sixth, seventh, and eighth sub-scores described above. The only difference is that the sub-scores target different objects, which will not be elaborated here.

[0156] In another implementation, the aforementioned third multimedia content feature, third interaction feature, third text content feature, and third publisher feature can be input into the aforementioned trained comment information ranking model. The comment information ranking model can output the ninth, tenth, eleventh, and twelfth sub-ratings of negative sample comments in relation to push priority.

[0157] Step D: Obtain the weighting coefficients that make the first total score greater than the first preset score corresponding to the positive sample reviews and make the second total score less than the second preset score corresponding to the negative sample reviews.

[0158] The aforementioned second preset score can be set by staff based on their experience.

[0159] Since the negative sample comments mentioned above are comments with a negative sentiment, they are often unpopular with users and are not popular comments.

[0160] If the second total score is not greater than the second preset score, then the second total score can be considered to indicate that the above negative sample comments are not popular comments.

[0161] Specifically, the above weighting coefficients can be determined in the following ways.

[0162] In one implementation, the weight coefficients of multimedia content features, interactive features, text content features, and publisher features can be continuously changed until the sub-scores corresponding to the features are weighted according to the weight coefficients. The resulting first total score is greater than the first preset score corresponding to the positive sample comments, and the resulting second total score is less than the second preset score corresponding to the negative sample comments. The weight coefficients corresponding to the multimedia features, interactive features, text content features, and publisher features at this time are obtained.

[0163] In another implementation, the aforementioned weighting coefficients can be determined by a review rating model. Specifically, based on the loss values ​​generated when the model outputs the first and second total scores, the model parameters can be adjusted until the loss values ​​are less than a preset loss value, thus obtaining the weighting coefficients determined by the model at this point, corresponding to multimedia content features, interaction features, text content features, and publisher features, respectively.

[0164] The first total score is calculated by weighting the fifth, sixth, seventh, and eighth sub-scores according to their respective weight coefficients. If the first total score is greater than the first preset score, it indicates that the positive sample comments are highly rated, meaning the first total score output by the comment scoring model for positive sample comments is as expected. Similarly, the second total score is calculated by weighting the ninth, tenth, eleventh, and twelfth sub-scores according to their respective weight coefficients. If the second total score is less than the second preset score, it indicates that the negative sample comments are not highly rated, meaning the comment scoring model for negative sample comments is also as expected. Therefore, it can be considered that the weights determined by the model at this point are beneficial in improving the rationality of the comment scoring.

[0165] Step E: Determine the obtained weight coefficients as the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, respectively.

[0166] This approach ensures that the determined weighting coefficients align with both the characteristics of positive comments with active interaction and those with negative sentiment. By adjusting the network parameters of the comment ranking model according to these weighting coefficients, the predicted score output by the model can represent both the likelihood of active interaction and the likelihood of a negative sentiment. In other words, the predicted score provides a more comprehensive representation of whether a comment is a "hot" comment. Ranking the comments based on the total score helps to identify comments with a higher probability of active interaction as "hot" comments and rank them higher, while identifying comments with a higher probability of negative sentiment as "not hot" comments and rank them lower, thus improving the model's rationality in ranking comment information.

[0167] In one embodiment of the present invention, if the user's level of interaction with the sorted comment information is less than a preset indicator, the weight coefficients corresponding to the multimedia content features, text content features, and publisher features can be updated respectively.

[0168] In one scenario, the preset metric could be the duration of time spent in the comment section. In this case, it can be determined whether the average duration of time a user spends in the comment section of the sorted comment information is less than the preset metric.

[0169] In another scenario, the aforementioned preset metrics can be preset interaction frequency or interaction quantity. In this case, it can be determined whether the user's interaction frequency or interaction quantity for each preset interaction item in the sorted comments is less than the aforementioned preset metrics.

[0170] The following steps, F and G, illustrate how to update the aforementioned weighting coefficients.

[0171] Step F: From the sorted comment information corresponding to each comment of the target object, select the target comment information with the most active interaction for the preset interaction item.

[0172] Specifically, for the sorted comments, a preset number of comments with the highest user interaction frequency for each preset interaction item in the sorted comments can be selected as the target comments with the most active interaction. Alternatively, a preset number of comments with the highest number of user interactions for each preset interaction item in the sorted comments can be selected as the target comments with the most active interaction.

[0173] Step G: Based on the interaction features of the target comment, the text content features of the target comment, the publisher features of the target comment, and the multimedia content features of the target object, update the weight coefficients corresponding to the interaction features, text content features, publisher features, and multimedia content features, respectively.

[0174] In this step, the interactive features of the target comment, the text content features of the target comment, the publisher features of the target comment, and the multimedia content features of the target object can be input into the aforementioned comment scoring model to obtain a weight coefficient that makes the third total score greater than the second preset score, which is then used as the updated weight coefficient. For detailed implementation methods, please refer to the method for determining weight coefficients using a comment scoring model described in step S306 above; it will not be repeated here.

[0175] As can be seen from the above, after pushing the sorted comments to the client, if the appeal of the sorted comments to the user is insufficient, the weight coefficients can be continuously updated based on the characteristics of the active comments after sorting. This makes the updated weight coefficients more consistent with the characteristics of the active comments, and thus, after adjusting the model parameters of the comment information sorting model based on the updated weight coefficients, the adjusted model can rank the active comments higher, making the model's sorting of comment information more reasonable.

[0176] Corresponding to the model training method described above, this embodiment of the invention also provides a method for sorting comment information.

[0177] See Figure 4 The above is a flowchart illustrating a method for sorting comment information provided in an embodiment of the present invention. The method includes the following steps S401-S402.

[0178] Step S401: Obtain the pool of comment information to be sorted, where each comment in the pool is associated with the same target multimedia content.

[0179] The above comment information pool contains multiple comments that are yet to be sorted.

[0180] Specifically, comment information on the aforementioned multimedia content can be obtained from the backend server of the content providing platform, and the obtained comment information can be stored as the aforementioned comment information pool.

[0181] Step S402: For the comment information in the comment information pool, obtain at least one of the following features, input the obtained features into the pre-trained comment ranking model according to the embodiment corresponding to the model training method, and rank the comment information in the comment information pool to obtain the ranking result:

[0182] Content characteristics of the target multimedia content;

[0183] The interactive features of each comment message;

[0184] The text content characteristics of each comment message;

[0185] The characteristics of the publisher for each comment.

[0186] The meanings of the above-mentioned content features, interaction features, text content features, and publisher features are the same as those of the multimedia content features, interaction features, text content features, and publisher features in the aforementioned model training method embodiments, and will not be repeated here.

[0187] In one implementation, the features of the comment information may include all four dimensions of features mentioned above.

[0188] As can be seen from the above, when sorting comment information using the solution provided in the embodiments of the present invention, at least one of the following can be obtained for the comment information in the comment information pool: the content features of the target multimedia content associated with the comment information, the interaction features of each comment information, the text content features of each comment information, and the publisher features corresponding to each comment information. The above features are then input into the comment sorting model to sort the comment information in the comment information pool. Compared with sorting comments using only the number of likes, this helps to prevent the comments at the top of the sorting from always being the comments with a high number of likes, thus improving the fluidity of comments during sorting.

[0189] In one embodiment of the present invention, in response to a request for obtaining comment information sent by a client, sorted comment information can also be pushed to the client.

[0190] It is evident that pushing sorted comments using the solution provided in this embodiment of the invention to the user's client facilitates the delivery of fresh comments to the user, thereby increasing the user's enthusiasm for interacting with the comments.

[0191] Furthermore, taking a video as the target object as an example, when pushing comments to users using the solution provided in this embodiment of the invention, the increased fluidity of the comments push to users makes it more likely that the comments pushed to users will meet the preferences of different users, increase users' enthusiasm for participating in comment interaction, and thus help stimulate users' interest in the video, thereby helping to increase the potential number of video views.

[0192] Corresponding to the above-described model training method, this embodiment of the invention also provides a model training device.

[0193] See Figure 5 The above is a schematic diagram of a model training device provided in an embodiment of the present invention. The device includes the following modules 501-503.

[0194] The sample feature acquisition module 501 is used to acquire sample features of sample comment information and the annotation score of the sample comment information for push priority, wherein the sample features include at least one of the following features:

[0195] The first multimedia content feature associated with the sample comment information;

[0196] The first interactive feature of the sample comment information;

[0197] First text content features of sample comment information;

[0198] The primary publisher characteristic of the sample comment information publishers;

[0199] The sample feature input module 502 is used to input the sample features into a preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the sorting result of the sample comment information output according to the estimated score.

[0200] The parameter adjustment module 503 is used to adjust the network parameters of the neural network model based on the difference between the estimated score and the labeled score, so as to obtain the comment information ranking model.

[0201] As can be seen from the above, when training the model using the solution provided in this embodiment of the invention, the sample features of the sample comment information are first obtained. Then, the obtained sample features are input into a preset neural network model. Based on the difference between the model output and the sample annotation, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The sample features can be multiple features among the following: the first multimedia content feature associated with the sample comment information, the first interaction feature of the sample comment information, the first text content feature of the sample comment information, and the first publisher feature of the sample comment information publisher. This allows the model to learn the relationship between the above features and the rating of the comment information during training. After training, a model for ranking comment information based on the above ratings can be obtained. When using this model to rank comments, compared to using only the number of likes, it helps prevent the top-ranked comments from always having a high number of likes, improving the fluidity of comments during ranking. Furthermore, after the ranked comments are pushed to the user's client, it helps improve the fluidity of the comments pushed to the user.

[0202] In one embodiment of the present invention, the first text content feature of the sample comment information includes: semantic sub-features of the sample comment information;

[0203] The semantic sub-features represent at least one of the following information corresponding to the sample comment information:

[0204] Text fluency;

[0205] Count of text characters;

[0206] The emotional tone of the text content;

[0207] Is the text meaningless?

[0208] In this way, semantic sub-features can represent a variety of different textual semantic information, thereby improving the comprehensiveness of the information represented by semantic sub-features.

[0209] In one embodiment of the present invention, the estimated score is: after the neural network model obtains a first sub-score for the push priority of sample comment information using the first multimedia content features, a second sub-score for the push priority of sample comment information using the first interaction features, a third sub-score for the push priority of sample comment information using the first text content features, and a fourth sub-score for the push priority of sample comment information using the first publisher features, the comprehensive score of the push priority of sample comment information obtained from the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score is obtained.

[0210] The model outputs a predicted score that is a comprehensive score based on the first sub-score corresponding to the first multimedia content feature, the second sub-score corresponding to the first interaction feature, the third sub-score corresponding to the first text content feature, and the fourth sub-score corresponding to the first publisher feature. By training the model with multiple features, the model can obtain a predicted score based on multiple features, which improves the comprehensiveness of the predicted score and thus improves the comprehensiveness of the model when ranking comment information based on the predicted score.

[0211] In addition, sub-scores for push priority of the above features were predicted based on the first multimedia content feature, the first interaction feature, the first text content feature, and the first publisher feature, respectively. The sub-scores are not affected by other features and each represents the priority of different dimensions of features for push. Then, the sub-scores are weighted according to the weight coefficients corresponding to each feature. The resulting score for push priority of comments can comprehensively represent the priority of the above-mentioned features for push. Therefore, the obtained score can more comprehensively represent the priority of comments for push.

[0212] In one embodiment of the present invention, the parameter adjustment module 503 is specifically used to adjust the network parameters of the neural network model based on the weight coefficients corresponding to the multimedia content features, interaction features, text content features, and publisher features, as well as the difference between the estimated score and the labeled score, to obtain a comment information ranking model.

[0213] As can be seen from the above, this approach not only adjusts the network parameters based on the difference between the model's predicted score and the labeled score, but also adjusts the network parameters according to the weight coefficients corresponding to multimedia content features, interaction features, text content features, and publisher features. By adjusting the weight coefficients to adjust the importance of different features to the predicted score, the adjusted model outputs sub-scores for each feature according to the aforementioned weight coefficients, improving the rationality of the predicted score obtained based on the aforementioned sub-scores. This, in turn, helps to improve the rationality of the model when ranking comment information based on the predicted score.

[0214] In one embodiment of the present invention, the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features are determined according to the following modules:

[0215] The positive sample comment collection module is used to collect the most active positive sample comments from the first sample object being commented on within a preset time period for the interaction with a preset interactive item;

[0216] The first feature acquisition module is used to obtain the second multimedia content feature associated with the first sample object, the second interaction feature of the positive sample comment, the second text content feature of the positive sample comment, and the second publisher feature of the publisher of the positive sample comment;

[0217] The first sub-rating prediction module is used to predict the fifth, sixth, seventh, and eighth sub-ratings of the positive sample comments for push priority based on the second multimedia content features, the second interaction features, the second text content features, and the second publisher features, respectively.

[0218] The weight coefficient acquisition module is used to obtain a weight coefficient that makes the first total score greater than the first preset score corresponding to the positive sample comment, wherein the first total score is: a score obtained by weighting the fifth sub-score, the sixth sub-score, the seventh sub-score, and the eighth sub-score according to the weight coefficient;

[0219] The weight coefficient determination module is used to determine the obtained weight coefficients as the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, respectively.

[0220] This ensures that the determined weight coefficients align with the characteristics of positive sample comments with active interaction. Adjusting the network parameters of the comment ranking model according to these weight coefficients helps ensure that the predicted score output by the model accurately reflects the likelihood of active interaction in the comment, i.e., the probability that the comment is a "hot" comment. Ranking the comments according to the predicted score helps identify comments with a higher probability of active interaction as "hot" comments, thus ranking them higher and improving the model's rationality in ranking comment information.

[0221] In one embodiment of the present invention, the apparatus further includes:

[0222] The negative sample comment collection module is used to collect negative sample comments on the negative sentiment of the second sample object being commented on;

[0223] The second feature acquisition module is used to acquire the third multimedia content feature associated with the second sample object, the third interaction feature of the negative sample comment, the third text content feature of the negative sample comment, and the third publisher feature of the publisher of the negative sample comment;

[0224] The second sub-rating prediction module is used to predict the ninth, tenth, eleventh, and twelfth sub-ratings of the negative sample comments for the push priority based on the third multimedia content features, the third interaction features, the third text content features, and the third publisher features, respectively.

[0225] The weight coefficient acquisition module is specifically used to obtain weight coefficients that make the first total score greater than the first preset score corresponding to the positive sample comment and make the second total score less than the second preset score corresponding to the negative sample comment, wherein the second total score is a score obtained by weighting the ninth sub-score, tenth sub-score, eleventh sub-score and twelfth sub-score according to the weight coefficients.

[0226] This approach ensures that the determined weighting coefficients align with both the characteristics of positive comments with active interaction and those with negative sentiment. By adjusting the network parameters of the comment ranking model according to these weighting coefficients, the predicted score output by the model can represent both the likelihood of active interaction and the likelihood of a negative sentiment. In other words, the predicted score provides a more comprehensive representation of whether a comment is a "hot" comment. Ranking the comments based on the total score helps to identify comments with a higher probability of active interaction as "hot" comments and rank them higher, while identifying comments with a higher probability of negative sentiment as "not hot" comments and rank them lower, thus improving the model's rationality in ranking comment information.

[0227] In one embodiment of the present invention, if the user's level of interaction with the sorted comment information is less than a preset indicator, the weight coefficients corresponding to the multimedia content features, text content features, and publisher features are updated according to the following modules:

[0228] The target comment information selection module is used to select the target comment information with the most active interaction with a preset interactive item from each comment information of the target object corresponding to the sorted comment information;

[0229] The weight coefficient update module is used to update the weight coefficients corresponding to the interaction features, text content features, publisher features, and multimedia content features of the target comment based on the interaction features of the target comment, the text content features of the target comment, the publisher features of the target comment, and the multimedia content features of the target object.

[0230] As can be seen from the above, after pushing the sorted comments to the client, if the appeal of the sorted comments to the user is insufficient, the weight coefficients can be continuously updated based on the characteristics of the active comments after sorting. This makes the updated weight coefficients more consistent with the characteristics of the active comments, and thus, after adjusting the model parameters of the comment information sorting model based on the updated weight coefficients, the adjusted model can rank the active comments higher, making the model's sorting of comment information more reasonable.

[0231] Corresponding to the above-mentioned comment information sorting method, this embodiment of the invention also provides a comment information sorting device.

[0232] See Figure 6 , Figure 6 This is a schematic diagram of a comment information sorting device provided in an embodiment of the present invention. The device includes the following modules 601-602.

[0233] The comment information pool acquisition module 601 is used to acquire a pool of comment information to be sorted, wherein each comment information in the comment information pool is associated with the same target multimedia content;

[0234] The comment information sorting module 602 is used to obtain at least one of the following features from the comment information in the comment information pool, input the obtained features into a comment sorting model pre-trained according to the aforementioned model training method embodiment, and sort the comment information in the comment information pool to obtain a sorting result:

[0235] The content characteristics of the target multimedia content;

[0236] The interactive features of each of the aforementioned comment messages;

[0237] The text content characteristics of each of the aforementioned comment messages;

[0238] The publisher characteristics corresponding to each of the aforementioned comment information.

[0239] As can be seen from the above, when sorting comment information using the solution provided in the embodiments of the present invention, at least one of the following can be obtained for the comment information in the comment information pool: the content features of the target multimedia content associated with the comment information, the interaction features of each comment information, the text content features of each comment information, and the publisher features corresponding to each comment information. The above features are then input into the comment sorting model to sort the comment information in the comment information pool. Compared with sorting comments using only the number of likes, this helps to prevent the comments at the top of the sorting from always being the comments with a high number of likes, thus improving the fluidity of comments during sorting.

[0240] In one embodiment of the present invention, the apparatus further includes:

[0241] The comment information push module is used to respond to a comment information retrieval request sent by a client and push sorted comment information to the client.

[0242] It is evident that pushing sorted comments using the solution provided in this embodiment of the invention to the user's client facilitates the delivery of fresh comments to the user, thereby increasing the user's enthusiasm for interacting with the comments.

[0243] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0244] Memory 703 is used to store computer programs;

[0245] The processor 701, when executing the program stored in the memory 703, implements the steps of the model training method or comment information sorting method provided in the embodiments of the present invention.

[0246] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0247] The communication interface is used for communication between the aforementioned terminal and other devices.

[0248] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0249] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0250] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program, when executed by a processor, implements the model training method or the comment information sorting method provided in the embodiments of the present invention.

[0251] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the model training method or the comment information sorting method provided in the embodiments of the present invention.

[0252] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0253] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0254] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for apparatus, electronic devices, and computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0255] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that, The method includes: The sample features of the sample comment information are obtained, as well as the labeled score of the sample comment information for push priority, wherein the sample features include at least one of the following features: The first multimedia content feature associated with the sample comment information; The first interactive feature of the sample comment information; First text content features of sample comment information; The primary publisher characteristic of the sample comment information publishers; The sample features are input into a preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the sorting result of the sample comment information output according to the estimated score. Based on the difference between the estimated score and the labeled score, the network parameters of the neural network model are adjusted to obtain a comment information ranking model; The step of adjusting the network parameters of the neural network model based on the difference between the estimated score and the labeled score to obtain the comment information ranking model includes: Based on the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, as well as the difference between the estimated score and the labeled score, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The weight coefficients for multimedia content features, interactive features, text content features, and publisher features are determined as follows: Collect the most active positive comments from the first sample object being commented on within a preset time period for a preset interactive item; Obtain the second multimedia content feature associated with the positive sample comment, the second interaction feature of the positive sample comment, the second text content feature of the positive sample comment, and the second publisher feature of the publisher of the positive sample comment; Based on the second multimedia content features, the second interaction features, the second text content features, and the second publisher features, the fifth, sixth, seventh, and eighth sub-ratings of the positive sample comments for push priority are predicted respectively. Obtain a weighting coefficient that makes the first total score greater than the first preset score corresponding to the positive sample comments, wherein the first total score is: a score obtained by weighting the fifth sub-score, the sixth sub-score, the seventh sub-score, and the eighth sub-score according to the weighting coefficient, and the first preset score is set by staff based on experience; The obtained weight coefficients are determined as the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, respectively.

2. The method according to claim 1, characterized in that, The first text content feature of the sample comment information includes: semantic sub-features of the sample comment information; The semantic sub-features represent at least one of the following information corresponding to the sample comment information: Text fluency; Count of text characters; The emotional tone of the text content; Is the text meaningless? 3. The method according to claim 1 or 2, characterized in that, The estimated score is: after the neural network model obtains a first sub-score for the push priority of sample comment information using the first multimedia content features, a second sub-score for the push priority of sample comment information using the first interaction features, a third sub-score for the push priority of sample comment information using the first text content features, and a fourth sub-score for the push priority of sample comment information using the first publisher features, the comprehensive score of the push priority of sample comment information obtained from the first sub-score, the second sub-score, the third sub-score, and the fourth sub-score is obtained.

4. The method according to claim 1, characterized in that, Also includes: Collect negative sample comments reflecting negative sentiment from the second sample of subjects being commented on; Obtain the third multimedia content feature associated with the negative sample comment, the third interaction feature of the negative sample comment, the third text content feature of the negative sample comment, and the third publisher feature of the publisher of the negative sample comment; Based on the third multimedia content feature, the third interaction feature, the third text content feature, and the third publisher feature, respectively, predict the ninth, tenth, eleventh, and twelfth sub-ratings of the negative sample comments in relation to the push priority; The step of obtaining the weighting coefficient that makes the first total score greater than the first preset score corresponding to the positive sample comments includes: Obtain weighting coefficients that make the first total score greater than the first preset score corresponding to the positive sample comments and make the second total score less than the second preset score corresponding to the negative sample comments. The second total score is a score obtained by weighting the ninth, tenth, eleventh, and twelfth sub-scores according to the weighting coefficients. The second preset score is set by staff based on experience.

5. The method according to claim 4, characterized in that, If the user's level of interaction with the sorted comments is less than the preset indicator, the weight coefficients corresponding to the multimedia content features, text content features, and publisher features will be updated as follows: From the sorted comment information corresponding to each comment of the target object, select the target comment information with the most active interaction for the preset interaction item; Based on the interaction features of the target comment, the text content features of the target comment, the publisher features of the target comment, and the multimedia content features of the target object, update the weight coefficients corresponding to the interaction features, text content features, publisher features, and multimedia content features, respectively.

6. A method for sorting comment information, characterized in that, The method is applicable to business servers that deploy comment ranking models, including: Obtain a pool of comment information to be sorted, wherein each comment in the pool is associated with the same target multimedia content; For the comment information in the comment information pool, at least one of the following features is obtained, and the obtained features are input into a pre-trained comment ranking model according to any one of claims 1-5 to rank the comment information in the comment information pool, thereby obtaining the ranking result: The content characteristics of the target multimedia content; The interactive features of each of the aforementioned comment messages; The text content characteristics of each of the aforementioned comment messages; The publisher characteristics corresponding to each of the aforementioned comment information.

7. The method according to claim 6, characterized in that, Also includes: In response to a request from a client to retrieve comment information, the sorted comment information is pushed to the client.

8. A model training device, characterized in that, The device includes: The sample feature acquisition module is used to acquire sample features of sample comment information and the labeled rating of the sample comment information for push priority, wherein the sample features include at least one of the following features: The first multimedia content feature associated with the sample comment information; The first interactive feature of the sample comment information; First text content features of sample comment information; The primary publisher characteristic of the sample comment information publishers; The sample feature input module is used to input the sample features into a preset neural network model to obtain the estimated score of the sample comment information corresponding to each sample feature output by the neural network model for the push priority, and the sorting result of the sample comment information output according to the estimated score. The parameter adjustment module is used to adjust the network parameters of the neural network model based on the difference between the estimated score and the labeled score, so as to obtain the comment information ranking model. The parameter adjustment module is specifically used for: Based on the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, as well as the difference between the estimated score and the labeled score, the network parameters of the neural network model are adjusted to obtain a comment information ranking model. The weighting coefficients for multimedia content features, interactive features, text content features, and publisher features are determined according to the following modules: The positive sample comment collection module is used to collect the most active positive sample comments from the first sample object being commented on within a preset time period for the interaction with a preset interactive item; The first feature acquisition module is used to obtain the second multimedia content feature associated with the positive sample comment, the second interaction feature of the positive sample comment, the second text content feature of the positive sample comment, and the second publisher feature of the publisher of the positive sample comment; The first sub-rating prediction module is used to predict the fifth, sixth, seventh, and eighth sub-ratings of the positive sample comments for push priority based on the second multimedia content features, the second interaction features, the second text content features, and the second publisher features, respectively. The weight coefficient acquisition module is used to obtain a weight coefficient that makes the first total score greater than the first preset score corresponding to the positive sample comment. The first total score is a score obtained by weighting the fifth sub-score, the sixth sub-score, the seventh sub-score, and the eighth sub-score according to the weight coefficient. The first preset score is set by the staff based on experience. The weight coefficient determination module is used to determine the obtained weight coefficients as the weight coefficients corresponding to multimedia content features, interactive features, text content features, and publisher features, respectively.

9. A device for sorting comment information, characterized in that, The device includes: The comment information pool acquisition module is used to acquire a pool of comment information to be sorted, wherein each comment information in the comment information pool is associated with the same target multimedia content; The comment information sorting module is used to obtain at least one of the following features from the comment information in the comment information pool, input the obtained features into a pre-trained comment sorting model according to any one of claims 1-5, and sort the comment information in the comment information pool to obtain a sorting result: The content characteristics of the target multimedia content; The interactive features of each of the aforementioned comment messages; The text content characteristics of each of the aforementioned comment messages; The publisher characteristics corresponding to each of the aforementioned comment information.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5 or 6-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5 or 6-7.

Images