Risk information determination method and device, equipment and storage medium

Abstract

The invention discloses a risk information determination method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that a first auditing result for a first song is acquired, the first auditing result is obtained through manual auditing, and the auditing result is used for indicating risk information of the first song; auditing the first song through a risk auditing model to obtain a second auditing result, the second auditing result being an auditing result obtained by model auditing, and the risk auditing model being a neural network model used for predicting risk information of the first song; and determining risk information of the first song based on the first audit result and the second audit result. According to the method, the risk information determination efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a method, device, equipment and storage medium for determining risk information. Background Art

[0002] With the development of information technology, the content and form of songs are becoming more and more diverse, which means that relevant music platforms need to review the songs to avoid the appearance of some illegal or inappropriate songs.

[0003] In the related art, songs are generally reviewed manually. For example, a three-person review method is used to review songs. Specifically, a three-person review means that if the review results of the first two people on the song are inconsistent, the review result of the third person shall prevail. If the review results of the first two people on the song are consistent, the review result shall be adopted. The review result here refers to the risk information of the song (including whether there is a risk, or the risk level of the song).

[0004] In the related art, the three-person review method requires at least two manual reviews, and may even require three manual reviews, so the review cost is high, resulting in low efficiency in determining song risk information. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and storage medium for determining risk information, which can improve the efficiency of determining risk information. The technical solutions proposed in this application are as follows:

[0006] According to one aspect of an embodiment of the present application, a method for determining risk information is provided, the method comprising:

[0007] Obtain a first review result for a first song, where the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song;

[0008] Reviewing the first song through a risk review model to obtain a second review result, where the second review result is a review result obtained by reviewing the model, and the risk review model is a neural network model for predicting risk information of the first song;

[0009] Based on the first review result and the second review result, risk information of the first song is determined.

[0010] According to one aspect of an embodiment of the present application, a device for determining risk information is provided, the device comprising:

[0011] A result acquisition module, used to acquire a first review result for a first song, where the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song;

[0012] A model review module, used for reviewing the first song through a risk review model to obtain a second review result, wherein the second review result is a review result obtained by reviewing the model, and the risk review model is a neural network model for predicting risk information of the first song;

[0013] A risk determination module is used to determine the risk information of the first song based on the first review result and the second review result.

[0014] According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned method for determining risk information.

[0015] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned method for determining risk information.

[0016] According to one aspect of an embodiment of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being loaded and executed by a processor to implement the above-mentioned method for determining risk information.

[0017] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0018] This application introduces a neural network model to replace one of the first two manual reviews in the three-person review of the related technology, using the model to replace manual work, reducing the number of manual reviews, thereby reducing the review cost and improving the efficiency of risk information determination. In addition, the model review + manual review method can take into account the accuracy and efficiency of risk information determination. While not losing the accuracy brought by manual review, it reduces the time for risk information determination and improves the efficiency of risk information determination. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of a computer system provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of a single person review provided by the relevant technology;

[0021] Figure 3 It is a schematic diagram of a three-person review provided by the related technology;

[0022] Figure 4 is a flow chart of a method for determining risk information provided by an embodiment of the present application;

[0023] Figure 5 is a flow chart of a method for determining risk information provided by another embodiment of the present application;

[0024] Figure 6 is a flow chart of a method for determining risk information provided by another embodiment of the present application;

[0025] Figure 7 is a schematic diagram of a method for determining risk information provided by an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of a risk audit model and a risk perception model provided by an embodiment of the present application;

[0027] Fig. 9 is a schematic diagram of a random forest classification algorithm provided by an embodiment of the present application;

[0028] Fig.10 is a block diagram of a device for determining risk information provided by an embodiment of the present application;

[0029] Fig.11 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0031] Please refer to Figure 1 , which shows a schematic diagram of a computer system provided by an embodiment of the present application. The computer system can be implemented as a risk information determination system. The computer system may include: a terminal device 10 and a server 20.

[0032] The number of terminal devices 10 can be one or more. The terminal device 10 can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a game console, an e-book reader, a multimedia player, a wearable device, a smart TV, a vehicle-mounted terminal, etc. The client of the target application can be installed in the terminal device 10. Optionally, the target application can be an application that needs to be downloaded and installed, or it can be a point-and-click application, which is not limited in the embodiment of the present application. In the embodiment of the present application, the above-mentioned target application is an application for determining risk information. Of course, there is no limitation on the specific type of the target application, and the type of the target application can be a music application, a video application, an animation application, a social application, a question-and-answer application, a virtual reality application, an augmented reality application, etc. The embodiment of the present application does not limit the specific category of the target application. In other embodiments, the target application can be considered as a separate functional module, such as being implemented as one of the functional modules in the music application for implementing risk information determination. Exemplarily, the client of the above-mentioned target application is running in the terminal device 10.

[0033] The server 20 is used to provide background services for the client of the target application installed and running in the terminal device 10. For example, the server 20 can be the background server of the above-mentioned target application. The server 20 can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. Optionally, the server 20 provides background services for the clients of the target application in multiple terminal devices 10 at the same time. The terminal device 10 and the server 20 can communicate with each other through the network.

[0034] When the above-mentioned target application is a music application, the music application obtains a first review result for the first song, the first review result is a review result obtained by manual review, and the review result is used to indicate the risk information of the first song; the first song is reviewed through a risk review model to obtain a second review result, and the second review result is a review result obtained by model review, and the risk review model is a neural network model for predicting the risk information of the first song; based on the first review result and the second review result, the risk information of the first song is determined.

[0035] The method provided in the embodiment of the present application may be performed by a computer device. The computer device may be any electronic device that has the ability to store and process data. For example, the computer device may be Figure 1 The terminal device 10 may also be a server 20.

[0036] In the related art, there are two review methods, including single-person review and three-person review.

[0037] The first type is a single person trial. Figure 2 As shown, the single-person review solution is that a song classification review task is only reviewed once (single-person review 210). Since the song classification task is relatively complex, the reviewers have different understandings of the review standards, and the review quality is difficult to guarantee.

[0038] The second type is three-person review. The three-person review plan means that a song will be reviewed by at least two and at most three times, and the review result is generated by voting on multiple review results. Figure 3 As shown, when the review results of the first instance 310 and the second instance 320 are consistent, there is no need for a third review; when the review results of the first instance 310 and the second instance 320 are different, the third instance 330 will conduct arbitration and ultimately adopt the result of the third instance 330 as the final result.

[0039] In the related technologies, the single-person review solution only conducts one human review for one review task, and its review quality fluctuates due to factors such as the reviewer's proficiency, psychological state, and subjectivity, and the review quality is generally poor. In the related technologies, the three-person review solution requires 2 to 3 reviews for one review task, and its review quality is improved, but the review cost will also increase by 2 to 3 times. In addition, the risk of songs changes in real time with dynamic factors such as changes in song information, hot events, and public opinion. The review technology in the related technologies cannot automatically identify which songs have high dynamic risks, and cannot push human review to high dynamic risk songs.

[0040] This application introduces a neural network model to replace one of the first two manual reviews in the three-person review of the related technology, using the model to replace manual work, reducing the number of manual reviews, thereby reducing the review cost and improving the efficiency of risk information determination. In addition, the model review + manual review method can take into account the accuracy and efficiency of risk information determination. While not losing the accuracy brought by manual review, it reduces the time for risk information determination and improves the efficiency of risk information determination.

[0041] Furthermore, the present application uses a risk perception model to predict the risk score of a song, and the risk perception model can be used to preliminarily determine whether the song needs to be reviewed again. The risk perception model can greatly reduce the amount of human review while quickly identifying the dynamic risk of the song.

[0042] See the following embodiments for details.

[0043] Please refer to Figure 4 , which shows a flow chart of a method for determining risk information provided by an embodiment of the present application. The execution subject of each step of the method may be the computer device introduced above. The method may include at least one of the following steps (410-430).

[0044] Step 410, obtaining a first review result for the first song, the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song.

[0045] In some embodiments, the first song is a song that needs to be reviewed. Exemplarily, the first song is a song uploaded by a user. Exemplarily, the first song is a song generated by AI. Exemplarily, the first song is a song written by the user himself, or a song edited by the user. This application does not limit the type, duration, content, etc. of the first song. In other embodiments, the first song is a complete song, or a part of a complete song, such as a song clip of a set duration.

[0046] In some embodiments, the first review result is the result of manually reviewing the first song. Exemplarily, the embodiments of the present application do not limit the reviewer of the manual review. Exemplarily, multiple reviewers are preset. Exemplarily, after the target application obtains the first song, it is sent to one of the multiple reviewers for review. Exemplarily, after the reviewer reviews the first song, the reviewer sends the first review result to the first song.

[0047] In some embodiments, the audit result is used to indicate risk information of the first song. Exemplarily, the risk information is information related to the risk of the song, such as at least one of the risk level, risk category, whether there is risk, etc.

[0048] Exemplarily, the audit result includes a risk category. Exemplarily, the risk category includes at least one of the following: blacklist, graylist, no problem (or whitelist). Exemplarily, the blacklist indicates that the song has a higher risk, also known as high risk, medium-high risk. Exemplarily, the graylist indicates that the song has a certain risk, but the risk is not very high, also known as low risk. Exemplarily, no problem indicates that the song does not have a risk, also known as no risk.

[0049] Exemplarily, the audit result includes whether the song has risk, such as risk or no risk. Exemplarily, when the song has risk, regardless of the degree of risk, the audit result is risk. Exemplarily, when the song does not have risk, the audit result is no risk.

[0050] Step 420, review the first song through the risk review model to obtain a second review result, where the second review result is the review result obtained by model review, and the risk review model is a neural network model for predicting risk information of the first song.

[0051] In some embodiments, the risk audit model is a neural network model for predicting risk information of the first song. In some embodiments, the risk audit model is a trained neural network model. Exemplarily, the risk audit model is a pre-trained model, or one or more modules in the risk audit model are pre-trained models, which is not limited in this application.

[0052] In some embodiments, the input of the risk audit model is the first song, and the output is the second audit result. Exemplarily, the input of the risk audit model is song information related to the first song (for specific content, see the explanation of the following embodiment), and the output is the second audit result.

[0053] In some embodiments, the risk audit model includes at least one of a feature extraction layer, a feature fusion layer, a classification layer, etc. Exemplarily, the feature extraction layer is used to extract features of the input information of the risk audit model. Exemplarily, the feature fusion layer is used to fuse the features extracted by the feature extraction layer to obtain fused features. The classification layer is used to classify according to the fused features to obtain a second audit result. The specific structures of the feature extraction layer, feature fusion layer, and classification layer are not limited in this application.

[0054] Step 430, determining risk information of the first song based on the first review result and the second review result.

[0055] In some embodiments, the first audit result and the second audit result may be the same or different.

[0056] In some embodiments, when the first audit result and the second audit result are the same, the risk information of the first song is determined based on the first audit result or the second audit result. Figure 7 As shown, when the first review result and the second review result of the model review obtained through the second review 710 are the same, the risk information of the first song is determined based on the first review result or the second review result.

[0057] In some embodiments, when the first audit result and the second audit result are different, a human audit is performed again, which may be referred to as a re-examination. Exemplarily, based on the audit result of the re-examination, the risk information of the first song is determined. Here, reference is made to the explanation of the following embodiment. In some embodiments, Figure 7 As shown, when the first review result and the second review result of the model review obtained through the second review 710 are different, a human review is performed again, that is, the third review.

[0058] In some embodiments, when the first audit result and the second audit result are different, the first audit result is used as the main factor and the second audit result is used as the auxiliary factor to determine the risk information of the first song. Exemplarily, when the first audit result indicates that the first song is on the blacklist and the second audit result indicates that the first song is on the graylist, the risk information of the first song is determined to be on the blacklist. Exemplarily, when the first audit result indicates that the first song is on the graylist and the second audit result indicates that there is no problem with the first song, the risk information of the first song is determined to be on the graylist. Exemplarily, when the first audit result indicates that the first song is on the blacklist and the second audit result indicates that there is no problem with the first song, the risk information of the first song is determined to be on the graylist.

[0059] This application introduces a neural network model to replace one of the first two manual reviews in the three-person review of the related technology, using the model to replace manual work, reducing the number of manual reviews, thereby reducing the review cost and improving the efficiency of risk information determination. In addition, the model review + manual review method can take into account the accuracy and efficiency of risk information determination. While not losing the accuracy brought by manual review, it reduces the time for risk information determination and improves the efficiency of risk information determination.

[0060] Please refer to Figure 5 , which shows a flow chart of a method for determining risk information provided by an embodiment of the present application. The execution subject of each step of the method may be the computer device introduced above. The method may include at least one of the following steps (510-550).

[0061] Step 510, obtaining a first review result for the first song, where the first review result is a review result obtained through manual review, and the review result is used to indicate risk information of the first song.

[0062] Step 520, obtaining multiple pieces of song information of the first song, where the type of each piece of song information includes at least one of the following: basic information of the first song, interactive information of the first song, portrait information of the first song, and security review information of the first song.

[0063] In some embodiments, when reviewing a song, it is necessary not only to review the content of the song itself, but also to review the content brought by the song, such as comments on the song. Exemplarily, a first song corresponds to multiple pieces of song information. Exemplarily, the type of song information includes at least one of the following: basic information of the first song, interactive information of the first song, portrait information of the first song, and security review information of the first song.

[0064] Exemplarily, basic information refers to information related to the attributes of the song itself. Exemplarily, the basic information of the first song includes at least one of the following: the genre of the first song, the language of the first song, the duration of the first song, whether the first song contains lyrics, etc.

[0065] Exemplarily, the interactive information refers to information about the interaction between the song and the user. Exemplarily, the interactive information of the first song includes at least one of the following: the number of plays of the first song, the number of comments on the first song, the number of collections of the first song, etc.

[0066] Exemplarily, the portrait information refers to information about a person or company associated with a song. Exemplarily, the portrait information of the first song includes at least one of the following: the proportion of songs that violate the law by the singer of the first song, and the proportion of songs that violate the law by the company to which the copyright of the first song belongs.

[0067] Exemplarily, the security audit information refers to information related to the specific content of the song that needs to be audited. Exemplarily, the security audit information of the first song includes at least one of the following: whether the lyrics of the first song, the song title of the first song, the song cover of the first song, etc. contain illegal content and the type of violation.

[0068] Step 530, filter multiple song information according to the importance values ​​corresponding to the multiple song information, and obtain at least one target song information, the target song information is the song information whose importance value is greater than or equal to the first threshold, and the importance value is used to indicate the influence of the song information on the second review result.

[0069] In some embodiments, each piece of song information corresponds to an importance value. Exemplarily, the importance value of each piece of song information is determined when training the risk audit model described below.

[0070] Exemplarily, before training the risk audit model, it is necessary to determine the training samples. Exemplarily, each training sample includes multiple song information of a song (also referred to as the original features of the song). Exemplarily, when using the training samples to train songs, it is necessary to screen multiple song information. Exemplarily, song information with higher importance values ​​is selected to form the training samples to train the risk audit model.

[0071] For example, multiple pieces of song information are sorted into boxes, WOE (Weight of Evidence (weight of evidence) coding, IV (Information Value) calculation, etc. Exemplarily, the importance values ​​corresponding to each piece of song information are calculated. Exemplarily, the IV values ​​corresponding to multiple pieces of song information are used as importance values. The specific binning, WOE coding, and IV values ​​are not described in detail. Among them, the IV value is used to characterize the contribution of the feature (song information) to the prediction target. Generally, the higher the IV value, the stronger the prediction ability of the feature.

[0072] Exemplarily, for different songs, the multiple pieces of song information are different. Exemplarily, the multiple pieces of song information here can also be considered as multiple variables. Exemplarily, some variables are selected from the multiple variables and put into the risk review model for training to form a list of model input variables. Exemplarily, song information with an importance value greater than or equal to a first threshold is selected to obtain at least one target song information. At least one target song information forms a list of model input variables.

[0073] Exemplarily, the importance value is used to indicate the degree of influence of the song information on the second audit result output by the risk audit model. When a piece of song information has a greater degree of influence on the second audit result, it is considered that the song information is a variable that should be selected to enter the risk audit model.

[0074] In some embodiments, after the risk audit model is trained, when the risk audit model is used to audit the first song, at least one target song information of the first song is directly obtained. Exemplarily, the risk audit model is used to audit at least one target song information of the first song to obtain a second audit result.

[0075] In some embodiments, when there are at least two pieces of song information satisfying a combination condition in at least one piece of target song information, the at least two pieces of song information are combined to obtain combined song information, wherein the remaining song information and the combined song information are used to obtain a second review result through a risk review model, and the remaining song information is the song information other than the at least two pieces of song information in the at least one piece of target song information.

[0076] Exemplarily, when at least two pieces of song information satisfying the combination condition exist in at least one piece of target song information, the at least two pieces of song information are combined to obtain the combined song information. Exemplarily, the song information can be combined to obtain the combined song information in either the model training phase or the model use phase.

[0077] Exemplarily, the combination condition includes song information belonging to a preset category. Exemplarily, a combination list is preset, and the combination list records the categories of all song information that can be combined. Exemplarily, when at least one target song information contains at least two song information belonging to a list of song information recorded in the combination list, the at least two song information are combined. Exemplarily, when the song information belongs to security audit information, multiple song information belonging to the security audit information are combined to obtain the combined song information. The combined song information is the constructed new song information.

[0078] In some embodiments, when there is missing content in the first song information in at least one target song information, the missing content in the first song information is determined by an automatic speech recognition (ASR) algorithm or a language classification algorithm based on song information in which content is not missing in at least one target song information.

[0079] Exemplarily, in the case where the content of the first song information is missing, if the first song information is the lyrics of the first song, the song audio of the first song can be identified by the ASR algorithm to obtain the lyrics of the first song.

[0080] Exemplarily, in the case where the content of the first song information is missing, if the first song information is the song language of the first song, the song audio of the first song can be identified through a language classification algorithm to obtain the song language of the first song.

[0081] The technical solution provided in the embodiment of the present application obtains new song information by combining song information, or supplements the missing content in the song information by ASR algorithm or language recognition algorithm. It not only expands the combination method of song information, but also helps to improve the content of song information and the accuracy of song risk review.

[0082] Step 540, obtaining a second audit result based on at least one target song information through a risk audit model.

[0083] The technical solution provided in the embodiment of the present application obtains song information with higher importance values ​​by screening and inputting it into the risk audit model to obtain the audit result. This avoids obtaining or processing unimportant song information, which results in unnecessary information acquisition and processing costs. Furthermore, by streamlining the input song information, the processing efficiency and accuracy of the model can also be improved.

[0084] In some embodiments, a risk audit model is used to obtain a predicted audit result and a first factor corresponding to a first song based on at least one target song information. The first factor is a factor predicted by the risk audit model that leads to the predicted audit result.

[0085] Exemplarily, the first factor is a factor predicted by the risk audit model that leads to the predicted audit result. Exemplarily, the first factor is the leading cause of the predicted audit result predicted by the risk audit model. Exemplarily, the first factor includes at least one of the following: at least one of the violation of the song content, the violation of the song review, the violation of the singer corresponding to the song, etc.

[0086] Exemplarily, the risk audit model outputs the first factor while outputting the predicted audit result.

[0087] In some embodiments, the predicted audit result is used as the second audit result.

[0088] Of course, the predicted audit results can also be adjusted based on the first factor.

[0089] In some embodiments, the predicted audit result is adjusted according to the category of the first factor to obtain a second audit result.

[0090] Exemplarily, a preset adjustment list is provided. Exemplarily, the adjustment list includes different factors and the adjustment range of the factors to the audit results. Exemplarily, the adjustment includes three categories, one is to increase the risk, one is to reduce the risk, and the other is no adjustment (ie, the adjustment range is 0).

[0091] For example, if the predicted audit result is a gray list and the first factor is that the song content violates the rules, the risk category is raised and the gray list is adjusted to a black list.

[0092] For example, if the predicted audit result is a gray list and the first factor is that the song review is illegal, the risk category will not be adjusted and the gray list will remain unchanged.

[0093] For example, if the predicted audit result is a blacklist and the first factor is that the song cover violates the rules, the risk category is downgraded and the blacklist is adjusted to a graylist.

[0094] The technical solution provided by the embodiment of the present application enhances the interpretability of the risk audit model by outputting the first factor through the risk audit model. At the same time, flexible adjustment of the audit result is achieved based on the first factor, reflecting the flexibility of risk information determination.

[0095] Step 550, determining risk information of the first song based on the first review result and the second review result.

[0096] In some embodiments, when the first review result and the second review result are the same, the risk information of the first song indicated by the first review result is determined as the risk information of the first song.

[0097] Exemplarily, if the first audit result and the second audit result are the same, the risk information of the first song indicated by the first audit result is determined as the risk information of the first song. Exemplarily, if the first audit result and the second audit result are the same, the risk information of the first song indicated by the second audit result is determined as the risk information of the first song.

[0098] In some embodiments, when the first review result and the second review result are different, a third review result for the first song is obtained, the third review result is a review result obtained by manual review, and the third review result and the first review result correspond to a different reviewer; the risk information of the first song indicated by the third review result is determined as the risk information of the first song.

[0099] Exemplarily, if the first and second audit results are different, they are re-sent for human review, that is, a third audit result is obtained. Exemplarily, the third audit result is also a manual review result, and the third audit result and the first audit result have different reviewers.

[0100] Exemplarily, the risk information of the first song indicated by the third review result is determined as the risk information of the first song.

[0101] The technical solution provided by the embodiment of the present application, when the audit results corresponding to the manual review and the model review are different, the manual review is pushed again, and the risk information of the first song is determined based on the result of the last manual review. If the audit results corresponding to the manual review and the model review are the same, the manual review is no longer pushed. This is conducive to reducing the number of manual reviews while maintaining the accuracy of manual reviews.

[0102] Please refer to Figure 6 , which shows a flow chart of a method for determining risk information provided by an embodiment of the present application. The execution subject of each step of the method may be the computer device introduced above. The method may include at least one of the following steps (610-650).

[0103] Step 610, obtaining a first review result for the first song, where the first review result is a review result obtained through manual review, and the review result is used to indicate risk information of the first song.

[0104] Step 620, review the first song through the risk review model to obtain a second review result, where the second review result is the review result obtained by model review, and the risk review model is a neural network model for predicting risk information of the first song.

[0105] Step 630, obtaining a risk score for the first song based on the first song through a risk perception model, where the risk perception model is a neural network model for predicting the risk score for the first song.

[0106] In some embodiments, the risk perception model is different from the risk review model, and the risk perception model determines the risk score of the song based on dynamic song information. Exemplarily, the risk perception model is used to identify the dynamic risk of a song. Exemplarily, dynamic song information can be understood as song information that has changed, such as changes in song information, new comments added to the song, and sudden hot events related to the song. Exemplarily, the risk score is used to indicate the risk level of the song. Exemplarily, the higher the risk score, the higher the risk level of the first song, and the greater the possibility of risk. Exemplarily, the lower the risk score, the lower the risk level of the first song, and the smaller the possibility of risk.

[0107] In some embodiments, the risk perception model is a neural network model for predicting the risk score of the first song. In some embodiments, the risk perception model is a trained neural network model. Exemplarily, the risk perception model is a pre-trained model, or one or more modules in the risk perception model are pre-trained models, which is not limited in this application.

[0108] In some embodiments, the input of the risk perception model is the first song, and the output is the risk score. Exemplarily, the input of the risk perception model is dynamic song information related to the first song (for specific content, see the explanation of the following embodiment), and the output is the risk score.

[0109] In some embodiments, the risk perception model includes at least one of a feature extraction layer, a feature fusion layer, a normalization layer, etc. Exemplarily, the feature extraction layer is used to extract features of the input information of the risk perception model. Exemplarily, the feature fusion layer is used to fuse the features extracted by the feature extraction layer to obtain fused features. The normalization layer is used to obtain a risk score based on the fused features. The specific structures of the feature extraction layer, feature fusion layer, and classification layer are not limited in this application.

[0110] In some embodiments, Figure 7 As shown, the risk perception model 720 is used to dynamically evaluate the risks of existing songs, and when the risk score is high, they are re-pushed for human review.

[0111] In some embodiments, the risk perception model includes a first risk perception sub-model, a second risk perception sub-model, and a third risk perception sub-model.

[0112] In some embodiments, a first risk score of the first song is determined based on basic information of the first song through a first risk perception sub-model, and the first risk perception sub-model is a neural network model for predicting the risk score of the first song in terms of basic song information.

[0113] Exemplarily, the first risk perception submodel is a neural network model for predicting the risk score of the first song in terms of basic song information. In some embodiments, the first risk perception submodel is a trained neural network model. Exemplarily, the first risk perception submodel is a pre-trained model, or one or more modules in the first risk perception submodel are pre-trained models, which is not limited in this application.

[0114] Exemplarily, the input of the first risk perception sub-model is the basic information of the first song (e.g., the dynamic part of the basic information, such as the change of the basic information), and the output is the first risk score. Exemplarily, the input of the first risk perception sub-model is the dynamic basic song information related to the first song, and the output is the first risk score.

[0115] Exemplarily, the first risk perception sub-model uses NLP (Natural Language Processing) models such as BERT (Bidirectional Encoder Representation from Transformers) to classify song information, identify the harmfulness of the changed content, and calculate a risk score (also known as the first risk score or song information risk score).

[0116] In some embodiments, a second risk score of the first song is determined based on interaction information of the first song by a second risk perception sub-model, where the second risk perception sub-model is a neural network model for predicting a risk score of the first song in terms of song interaction.

[0117] Exemplarily, the second risk perception submodel is a neural network model for predicting the risk score of the second song in terms of interaction. In some embodiments, the second risk perception submodel is a trained neural network model. Exemplarily, the second risk perception submodel is a pre-trained model, or one or more modules in the second risk perception submodel are pre-trained models, which is not limited in this application.

[0118] Exemplarily, the input of the second risk perception sub-model is the interactive information of the second song (e.g., the dynamic part of the interactive information, such as the newly added song comments), and the output is the second risk score. Exemplarily, the input of the second risk perception sub-model is the dynamic interactive information related to the second song, and the output is the second risk score.

[0119] For example, the increase in song comments per unit time is monitored, and cluster analysis such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise) and k-means is performed on the incremental comments, and harmful content is detected using LLM (Large Language Model). The categories and quantities of harmful content are comprehensively counted to calculate the second risk score (also called the interaction risk score).

[0120] In some embodiments, a third risk score of the first song is determined based on the portrait information of the first song through a third risk perception sub-model, and the third risk perception sub-model is a neural network model for predicting the risk score of the first song in terms of the song portrait.

[0121] Exemplarily, the third risk perception submodel is a neural network model for predicting the risk score of the third song in terms of song portrait. In some embodiments, the third risk perception submodel is a trained neural network model. Exemplarily, the third risk perception submodel is a pre-trained model, or one or more modules in the third risk perception submodel are pre-trained models, which is not limited in this application.

[0122] Exemplarily, the input of the third risk perception sub-model is the portrait information of the third song (e.g., the dynamic part of the portrait information, such as the changed portrait information), and the output is the third risk score. Exemplarily, the input of the third risk perception sub-model is the dynamic portrait information related to the third song, and the output is the third risk score. Exemplarily, the dynamic portrait information includes public opinion and hot events. Exemplarily, the third risk perception sub-model calculates the third risk score (also called real-time portrait risk score) based on responding to public opinion and hot events, combined with core portrait information such as the violation rate of singers' songs and the violation rate of copyright company's songs.

[0123] In some embodiments, a risk score for the first song is determined based on the first risk score, the second risk score, and the third risk score.

[0124] Exemplarily, the first risk score, the second risk score, and the third risk score are weighted to obtain a risk score for the first song. Exemplarily, the weights of different risk scores are preset.

[0125] For example, Figure 8As shown, songs are reviewed through the risk review model 810, which includes content such as feature collection, feature engineering, model training, and model analysis. Among them, feature collection can be understood as collecting song information. Feature engineering can be understood as processing the original song information, such as selecting song information with high importance values, combining song information, and filling in missing content in song information. Exemplarily, model training includes model selection, model optimization, and other parts. Exemplarily, model analysis includes SHAP (SHapley Additive exPlanations, model interpretability) analysis, multiple evaluation indicators, offline verification, and online verification, etc.

[0126] For example, Figure 8 As shown, the risk score is predicted by the risk perception model 820, which includes the song information risk score, the interaction risk score and the portrait risk score.

[0127] The technical solution provided in the embodiment of the present application predicts the risk scores of the first song in different aspects through three sub-models respectively, and weights them to obtain the final risk score to determine whether the first song has dynamic risks, which reflects the accuracy and flexibility of risk score determination and the diversity of song review.

[0128] Step 640, when the risk score of the first song is greater than or equal to the second threshold, obtain a fourth review result for the first song, the fourth review result is a review result obtained by manual review, and the fourth review result and the first review result correspond to a different reviewer.

[0129] Exemplarily, the second threshold is a preset value. Exemplarily, when the risk score of the first song is greater than or equal to the second threshold, it indicates that the first song has a greater dynamic risk, and needs to be re-sent for human review.

[0130] Step 650, determining risk information of the first song based on the first review result, the second review result, and the fourth review result.

[0131] Exemplarily, when the first review result, the second review result, and the fourth review result are all the same, the risk information of the first song is determined.

[0132] Exemplarily, when the first audit result, the second audit result, and the fourth audit result are different, the risk information of the first song is determined based on the first audit result or the fourth audit result. Exemplarily, if the first audit result and the fourth audit result are different, the fourth audit result shall prevail.

[0133] The technical solution provided in the embodiment of the present application helps to improve the perception of dynamic risks and the accuracy of song review by re-pushing songs with risk scores greater than or equal to a second threshold for human review.

[0134] The following introduces the training process of the risk audit model, in which the second audit result includes the risk category.

[0135] In some embodiments, a first pre-trained model is selected from a plurality of pre-trained models as a base model, and the first pre-trained model is a neural network model for implementing classification.

[0136] Exemplarily, the plurality of pre-trained models include at least one of the following: random forest (algorithm process such as Fig. 9 As shown in the figure), decision tree models such as LightGBM (Light Gradient Boosting Machine), scorecards or MLP (Multilayer Perceptron) deep learning models. Exemplarily, the model with the best comprehensive evaluation effect is found from these pre-trained models as the base model.

[0137] In some embodiments, multiple training samples are obtained, each training sample includes song information of a sample song and a label risk category of the sample song, and the label risk category of the sample song is obtained by adjusting the actual risk category of the sample song.

[0138] Exemplarily, multiple training samples are obtained, each of which includes song information of a sample song and a label risk category of the sample song. Exemplarily, the label risk category can be the actual risk category of a song that has been reviewed, or can be obtained by adjusting the actual risk category of the sample song.

[0139] For example, the actual risk category may include more levels. For example, if there are risk levels 1 to 10, some of the levels are merged to obtain several merged levels. For example, risk levels 1 to 3 are merged as a whitelist, risk levels 4 to 6 are merged as a graylist, and risk levels 7 to 10 are merged as a blacklist. The whitelist, graylist, and blacklist are different risk categories after merging.

[0140] In some embodiments, the hyperparameters of the base model are determined by a hyperparameter search technique, where the hyperparameters of the base model are parameters that remain unchanged during the training process of the base model.

[0141] During the training process, the hyperparameter search technology is used to automatically select the hyperparameters with the best performance, and the early stopping, regularization method and other technologies are used to prevent the risk audit model from overfitting. Exemplarily, the hyperparameter search technology, hyperparameters, early stopping, regularization method and the like are not described in detail. Exemplarily, the hyperparameters include at least one of the following: learning rate, batch size and the like.

[0142] In some embodiments, the base model is trained using multiple training samples and hyperparameters of the base model to obtain a risk audit model.

[0143] Exemplarily, after the hyperparameters are determined, the base model can be trained based on the hyperparameters and multiple training samples to obtain a risk audit model. Exemplarily, with the goal of minimizing the loss function value, the parameters in the base model (other learnable parameters other than the hyperparameters) are adjusted to obtain a trained base model (i.e., a risk audit model).

[0144] The technical solution provided in the embodiment of the present application reduces the classification pressure of the model by adjusting the actual risk category of the sample songs, which is conducive to improving the classification accuracy, thereby improving the accuracy of the model review of the song review.

[0145] Furthermore, the optimal hyperparameters can be searched through hyperparameter search technology, and the model overfitting can be prevented, which is conducive to improving the effect of model training and thus improving the model accuracy of the risk audit model.

[0146] The following is an introduction to the focused training of samples in the case of songs that have undergone historical review and have different first and second reviews, leading to a third review. This is also the further optimization process of the model.

[0147] In some embodiments, the different data multiple training samples include a first training sample.

[0148] Exemplarily, the first training sample is a training sample formed by different first and third review results in a scenario of three reviewers in the related art. Exemplarily, since the third review result is the final review result, the training sample is constructed with the third review result as the review result.

[0149] In some embodiments, when the first-instance result and the second-instance result for the second song are different, the third-instance result of the second song is obtained as the actual risk category of the second song. The second song is a song that has been reviewed. The first-instance result is the review result obtained by the first manual review of the second song, the second-instance result is the review result obtained by the second manual review of the second song, and the third-instance result is the review result obtained by the third manual review of the second song. The reviewer for each manual review is different.

[0150] In some embodiments, the first training sample is determined based on the song information of the second song and the actual risk category of the second song.

[0151] Exemplarily, first training samples are constructed, each of which includes song information of the second song and the actual risk category of the second song. Exemplarily, each first training sample includes song information of the second song and the adjusted risk category of the second song adjusted based on the actual risk category.

[0152] In some embodiments, in the process of training the base model using multiple training samples, the sample weight of the first training sample is greater than the sample weights of other training samples in the multiple training samples except the first training sample.

[0153] Exemplarily, the sample weight is used to indicate the importance of the sample. Exemplarily, the number of samples is changed by the sample weight. Exemplarily, when the sample weight is larger, the number of the sample in the total training samples is larger. Exemplarily, for example, the model is trained 5 times using the same training sample.

[0154] The technical solution provided in the embodiment of the present application uses the first training sample to train the model, which is conducive to strengthening the learning of the first-instance error data and allowing the model to better correct the first-instance errors. Furthermore, by increasing the sample weight of the first training sample, the model error correction efficiency is improved.

[0155] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0156] Please refer to Fig.10 , which shows a block diagram of a device for determining risk information provided by an embodiment of the present application. The device has the function of implementing the above-mentioned method for determining risk information, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the computer device described above, or it can be set in a computer device. Fig.10 As shown, the device 1000 may include: a result acquisition module 1010 , a model review module 1020 , and a risk determination module 1030 .

[0157] The result acquisition module 1010 is used to obtain a first review result for a first song, where the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song.

[0158] The model review module 1020 is used to review the first song through a risk review model to obtain a second review result, where the second review result is a review result obtained by reviewing the model. The risk review model is a neural network model for predicting risk information of the first song.

[0159] The risk determination module 1030 is used to determine the risk information of the first song based on the first review result and the second review result.

[0160] In some embodiments, the model review module 1020 is also used to obtain multiple pieces of song information of the first song, and the type of each piece of song information includes at least one of the following: basic information of the first song, interactive information of the first song, portrait information of the first song, and security review information of the first song; according to the importance values ​​corresponding to the multiple pieces of song information, the multiple pieces of song information are screened to obtain at least one target song information, and the target song information is song information having an importance value greater than or equal to a first threshold, and the importance value is used to indicate the degree of influence of the song information on the second review result.

[0161] In some embodiments, the model review module 1020 is further used to obtain the second review result according to the at least one target song information through the risk review model.

[0162] In some embodiments, the model review module 1020 is also used to combine the at least two pieces of song information to obtain combined song information when there are at least two pieces of song information satisfying the combination condition in the at least one piece of target song information, wherein the remaining song information and the combined song information are used to obtain the second review result through the risk review model, and the remaining song information is the song information in the at least one piece of target song information other than the at least two pieces of song information; when there is missing content in the first song information in the at least one piece of target song information, the missing content in the first song information is determined based on the song information in the at least one piece of target song information that is not missing content through an ASR algorithm or a language classification algorithm.

[0163] In some embodiments, the model review module 1020 is also used to obtain the predicted review result and the first factor corresponding to the first song according to the at least one target song information through the risk review model, wherein the first factor is the factor predicted by the risk review model that leads to the predicted review result; according to the category of the first factor, the predicted review result is adjusted to obtain the second review result.

[0164] In some embodiments, the risk determination module 1030 is used to, when the first review result and the second review result are the same, determine the risk information of the first song indicated by the first review result as the risk information of the first song; when the first review result and the second review result are different, obtain a third review result for the first song, the third review result is a review result obtained by manual review, and the third review result is different from the reviewer corresponding to the first review result; determine the risk information of the first song indicated by the third review result as the risk information of the first song.

[0165] In some embodiments, the risk determination module 1030 is used to obtain a risk score for the first song based on the first song through a risk perception model, where the risk perception model is a neural network model for predicting the risk score of the first song; when the risk score of the first song is greater than or equal to a second threshold, a fourth review result for the first song is obtained, where the fourth review result is a review result obtained by manual review, and the fourth review result is different from the reviewer corresponding to the first review result, and the fourth review result is used to determine the risk information of the first song in combination with the first review result and the second review result.

[0166] In some embodiments, the risk perception model includes a first risk perception sub-model, a second risk perception sub-model, and a third risk perception sub-model.

[0167] In some embodiments, the risk determination module 1030 is used to determine a first risk score of the first song based on basic information of the first song through the first risk perception submodel, where the first risk perception submodel is a neural network model for predicting the risk score of the first song in terms of basic song information; determine a second risk score of the first song based on interaction information of the first song through the second risk perception submodel, where the second risk perception submodel is a neural network model for predicting the risk score of the first song in terms of song interaction; determine a third risk score of the first song based on portrait information of the first song through the third risk perception submodel, where the third risk perception submodel is a neural network model for predicting the risk score of the first song in terms of song portrait; determine the risk score of the first song based on the first risk score, the second risk score and the third risk score.

[0168] In some embodiments, the second audit result includes a risk category. In some embodiments, the model audit module 1020 is also used to execute the steps of the training process of the risk audit model: selecting a first pre-trained model from a plurality of pre-trained models as a base model, wherein the first pre-trained model is a neural network model for implementing classification; obtaining a plurality of training samples, each of which includes song information of a sample song and a label risk category of the sample song, wherein the label risk category of the sample song is obtained after adjusting the actual risk category of the sample song; determining the hyperparameters of the base model through a hyperparameter search technique, wherein the hyperparameters of the base model are parameters that remain unchanged during the training process of the base model; training the base model through the plurality of training samples and the hyperparameters of the base model to obtain the risk audit model.

[0169] In some embodiments, the plurality of training samples includes a first training sample.

[0170] In some embodiments, the risk determination module 1030 is also used to obtain the third review result of the second song as the actual risk category of the second song when the first review result and the second review result for the second song are different, wherein the second song is a song that has been reviewed, and the first review result is the review result obtained by the first manual review of the second song, the second review result is the review result obtained by the second manual review of the second song, and the third review result is the review result obtained by the third manual review of the second song, and the reviewer for each manual review is different; the first training sample is determined based on the song information of the second song and the actual risk category of the second song.

[0171] In some embodiments, in the process of training the base model using the multiple training samples, the sample weight of the first training sample is greater than the sample weights of other training samples in the multiple training samples except the first training sample.

[0172] This application introduces a neural network model to replace one of the first two manual reviews in the three-person review of the related technology, using the model to replace manual work, reducing the number of manual reviews, thereby reducing the review cost and improving the efficiency of risk information determination. In addition, the model review + manual review method can take into account the accuracy and efficiency of risk information determination. While not losing the accuracy brought by manual review, it reduces the time for risk information determination and improves the efficiency of risk information determination.

[0173] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0174] Please refer to Fig.11 , which shows a block diagram of a computer device 1100 provided in one embodiment of the present application. The computer device 1100 may be any electronic device with data computing, processing and storage capabilities. The computer device 1100 may be used to implement the risk information determination method provided in the above embodiment.

[0175] Typically, the computer device 1100 includes a processor 1101 and a memory 1102 .

[0176] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI processor for processing computing operations related to machine learning.

[0177] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned risk information determination method.

[0178] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the computer device 1100, and the computer device 1100 may include more or less components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0179] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned method for determining risk information is implemented. Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random Access Memory), SSD (Solid State Drives) or optical disks, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0180] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned method for determining risk information.

[0181] It should be noted that the collection and processing of relevant data (including songs, song information, review results, etc.) in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in instances, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0182] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0183] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining risk information, characterized in that: The method comprises: Obtain a first review result for a first song, where the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song; Reviewing the first song through a risk review model to obtain a second review result, where the second review result is a review result obtained by reviewing the model, and the risk review model is a neural network model for predicting risk information of the first song; Based on the first review result and the second review result, risk information of the first song is determined.

2. The method according to claim 1, characterized in that The method further comprises: Acquire multiple pieces of song information of the first song, where the type of each piece of song information includes at least one of the following: basic information of the first song, interactive information of the first song, portrait information of the first song, and security review information of the first song; According to the importance values ​​respectively corresponding to the plurality of song information, the plurality of song information are screened to obtain at least one target song information, wherein the target song information is song information having an importance value greater than or equal to a first threshold, and the importance value is used to indicate the degree of influence of the song information on the second review result; The step of reviewing the first song through the risk review model to obtain a second review result includes: The second audit result is obtained through the risk audit model according to the at least one target song information.

3. The method according to claim 2, characterized in that The method further comprises: In the case where there are at least two pieces of song information satisfying the combination condition in the at least one piece of target song information, the at least two pieces of song information are combined to obtain combined song information, wherein the remaining song information and the combined song information are used to obtain the second audit result through the risk audit model, and the remaining song information is the song information in the at least one piece of target song information other than the at least two pieces of song information; In the case that there is missing content in the first song information in the at least one target song information, the missing content in the first song information is determined by an automatic speech recognition ASR algorithm or a language classification algorithm based on song information with non-missing content in the at least one target song information.

4. The method according to claim 2 or 3, characterized in that: The step of obtaining the second audit result according to the at least one target song information by using the risk audit model includes: Obtaining, by means of the risk audit model, a predicted audit result and a first factor corresponding to the first song according to the at least one target song information, wherein the first factor is a factor predicted by the risk audit model to cause the predicted audit result; According to the category of the first factor, the predicted audit result is adjusted to obtain the second audit result.

5. The method according to claim 1, characterized in that The determining the risk information of the first song based on the first review result and the second review result includes: In a case where the first review result and the second review result are the same, determining the risk information of the first song indicated by the first review result as the risk information of the first song; When the first review result and the second review result are different, obtain a third review result for the first song, where the third review result is a review result obtained by manual review, and the third review result is different from the reviewer corresponding to the first review result; determine the risk information of the first song indicated by the third review result as the risk information of the first song.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining a risk score for the first song according to the first song through a risk perception model, wherein the risk perception model is a neural network model for predicting the risk score for the first song; When the risk score of the first song is greater than or equal to a second threshold, a fourth review result for the first song is obtained, where the fourth review result is a review result obtained through manual review, and the fourth review result is different from the reviewer corresponding to the first review result. The fourth review result is used to determine the risk information of the first song in combination with the first review result and the second review result.

7. The method according to claim 6, characterized in that The risk perception model includes a first risk perception sub-model, a second risk perception sub-model and a third risk perception sub-model; The step of obtaining a risk score of the first song according to the first song through the risk perception model includes: Determining a first risk score of the first song based on the basic information of the first song through the first risk perception sub-model, wherein the first risk perception sub-model is a neural network model for predicting the risk score of the first song in terms of the basic information of the song; determining a second risk score of the first song based on the interaction information of the first song through the second risk perception sub-model, wherein the second risk perception sub-model is a neural network model for predicting the risk score of the first song in terms of song interaction; Determining a third risk score of the first song based on the portrait information of the first song through the third risk perception sub-model, wherein the third risk perception sub-model is a neural network model for predicting the risk score of the first song in terms of the song portrait; A risk score for the first song is determined based on the first risk score, the second risk score, and the third risk score.

8. The method according to claim 1, characterized in that The second audit result includes a risk category, and the training process of the risk audit model includes the following steps: Selecting a first pre-trained model from a plurality of pre-trained models as a base model, wherein the first pre-trained model is a neural network model for implementing classification; Acquire multiple training samples, each training sample including song information of a sample song and a label risk category of the sample song, where the label risk category of the sample song is obtained by adjusting an actual risk category of the sample song; Determine the hyperparameters of the base model by using a hyperparameter search technique, where the hyperparameters of the base model are parameters that remain unchanged during the training process of the base model; The risk audit model is obtained by training the base model using the multiple training samples and the hyperparameters of the base model.

9. The method according to claim 8, characterized in that The plurality of training samples include a first training sample, and the method further includes: In the case where the first-instance result and the second-instance result for the second song are different, obtaining the third-instance result of the second song as the actual risk category of the second song, the second song is a song that has been reviewed, the first-instance result is the review result obtained by the first manual review of the second song, the second-instance result is the review result obtained by the second manual review of the second song, and the third-instance result is the review result obtained by the third manual review of the second song, and the reviewer of each manual review is different; The first training sample is determined based on the song information of the second song and the actual risk category of the second song.

10. The method according to claim 9, characterized in that In the process of training the base model using the multiple training samples, the sample weight of the first training sample is greater than the sample weights of other training samples among the multiple training samples except the first training sample.

11. A device for determining risk information, characterized in that: The device comprises: A result acquisition module, used to acquire a first review result for a first song, where the first review result is a review result obtained by manual review, and the review result is used to indicate risk information of the first song; A model review module, used for reviewing the first song through a risk review model to obtain a second review result, wherein the second review result is a review result obtained by reviewing the model, and the risk review model is a neural network model for predicting risk information of the first song; A risk determination module is used to determine the risk information of the first song based on the first review result and the second review result.

12. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the method for determining risk information as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method for determining risk information as described in any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. A processor reads and executes the computer program from the computer-readable storage medium to implement the method for determining risk information as described in any one of claims 1 to 10.

Images