Risk content identification model updating method, computer device and storage medium

Abstract

The application relates to a risk content identification model updating method, a computer device and a storage medium. The method comprises the following steps: a server inputs content data to be identified into a pre-trained initial risk content identification model, and obtains first content data identified as not containing risk content by the initial risk content identification model; user behavior features associated with the first content data are obtained, and candidate first content data in the first content data is obtained by screening according to the user behavior features; the candidate first content data is first content data containing potential risk content; target first content data containing risk content in the candidate first content data is obtained based on risk auditing of the candidate first content data; the initial risk content identification model is trained by using the target first content data, model parameters of the initial risk content identification model are updated, and a target risk content identification model is obtained. The method can improve the identification accuracy of risk content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for updating a risky content identification model, a computer device, and a storage medium. Background Technology

[0002] With the development of computer technology, a method has emerged to identify risks in content published by users on the Internet. This method can input the content published by users into a pre-trained risk identification model, and then use the model to identify whether the content published by users contains risky content.

[0003] However, the aforementioned methods for identifying risky content using risky content identification models are often only applicable to identifying some known or relatively common risky content. For new types of risky content, such as risky content that has been modified and edited specifically to circumvent risk identification systems, these risky content identification models cannot accurately identify them. Therefore, the existing risky content identification models have a low accuracy rate in identifying risky content. Summary of the Invention

[0004] Therefore, it is necessary to provide a risk content identification model update method, computer equipment, and computer-readable storage medium that can improve the accuracy of risk content identification in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for updating a risky content identification model, applied to a server, the method comprising:

[0006] The content data to be identified is input into a pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content;

[0007] Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0008] Based on the risk assessment of the candidate first content data, target first content data containing risky content is obtained from the candidate first content data;

[0009] The initial risk content identification model is trained using the target first content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

[0010] In one embodiment, obtaining user behavior features associated with the first content data and filtering candidate first content data from the first content data based on the user behavior features includes: obtaining a probability ranking of the first content data containing risky content based on the user behavior features; obtaining the number of candidate first content data to be filtered; and filtering out the number of candidate first content data from the first content data according to the probability ranking.

[0011] In one embodiment, obtaining the probability ranking of the first content data containing risky content based on the user behavior features includes: obtaining sample user behavior features associated with the first sample content data, and risky sample data that actually contains risky content in the first sample content data; obtaining the similarity between the first content data and the risky sample data based on the user behavior features and the sample user behavior features associated with the risky sample data, and obtaining the predicted probability that the first content data contains risky content based on the similarity; and sorting the first content data according to the predicted probability to obtain the probability ranking.

[0012] In one embodiment, the step of obtaining the similarity between the first content data and the risk sample data based on the user behavior features associated with the risk sample data, and obtaining the predicted probability that the first content data contains risky content based on the similarity, includes: obtaining current first content data and current user behavior features associated with the current first content data; obtaining a similarity ranking between the current first content data and the first sample content data based on the current user behavior features and the sample user behavior features associated with the first sample content data; obtaining a first preset number of first sample content data from the similarity ranking; and obtaining the predicted probability that the current first content data contains risky content if the first preset number of first sample content data contains at least a second preset number of risky sample data; wherein the second preset number is less than or equal to the first preset number.

[0013] In one embodiment, after inputting the content data to be identified into a pre-trained initial risk content identification model, the method further includes: obtaining second content data identified by the initial risk content identification model as containing potentially risky content; obtaining the number of candidate first content data to be screened includes: obtaining a risk review quantity threshold and the quantity of the second content data; obtaining the number of screenings based on the risk review quantity threshold and the quantity of the second content data.

[0014] In one embodiment, obtaining the user behavior features associated with the first content data includes: obtaining the publishing user behavior features of the publishing user corresponding to the first content data, and publishing the first content data within a preset range; obtaining the user feedback behavior features after the first content data is published within the preset range; and using the publishing user behavior features and the user feedback behavior features as the user behavior features associated with the first content data.

[0015] In one embodiment, after inputting the content data to be identified into a pre-trained initial risk content identification model, the method further includes: obtaining second content data identified by the initial risk content identification model as containing potentially risky content; based on risk assessment of the second content data, obtaining target second content data that does not contain risky content; and training the initial risk content identification model using the target first content data and updating the model parameters of the initial risk content identification model to obtain a target risk content identification model, which includes: training the initial risk content identification model using the target first content data and the target second content data, and updating the model parameters of the initial risk content identification model to obtain a target risk content identification model.

[0016] In one embodiment, the initial risk content identification model is trained using second sample content data; the step of training the initial risk content identification model using the target first content data and the target second content data, and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model, includes: inputting the second sample content data, the target first content data, and the target second content data into the initial risk content identification model; obtaining content features of the second sample content data, the target first content data, and the target second content data through the initial risk content identification model; training the initial risk content identification model based on the content features; and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0017] In one embodiment, after training the initial risk content identification model using the target first content data and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model, the method further includes: obtaining model verification results of the initial risk content identification model and the target risk content identification model; if the model verification results indicate that the model performance of the target risk content identification model is better than that of the initial risk content identification model, using the target risk content identification model as the updated initial risk content identification model; and / or if the model verification results indicate that the model performance of the initial risk content identification model is better than that of the target risk content identification model, stopping the updating of the initial risk content identification model.

[0018] Secondly, this application also provides a risk content identification model updating device, applied to a server, the device comprising:

[0019] The first content acquisition module is used to input the content data to be identified into a pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content.

[0020] The candidate content filtering module is used to obtain user behavior features associated with the first content data, and filter candidate first content data in the first content data according to the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0021] The target data acquisition module is used to obtain target first content data containing risky content from the candidate first content data based on the risk assessment of the candidate first content data.

[0022] The identification model update module is used to train the initial risk content identification model using the target first content data and update the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0023] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0024] The content data to be identified is input into a pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content;

[0025] Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0026] Based on the risk assessment of the candidate first content data, target first content data containing risky content is obtained from the candidate first content data;

[0027] The initial risk content identification model is trained using the target first content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

[0028] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0029] The content data to be identified is input into a pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content;

[0030] Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0031] Based on the risk assessment of the candidate first content data, target first content data containing risky content is obtained from the candidate first content data;

[0032] The initial risk content identification model is trained using the target first content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

[0033] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0034] The content data to be identified is input into a pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content;

[0035] Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0036] Based on the risk assessment of the candidate first content data, target first content data containing risky content is obtained from the candidate first content data;

[0037] The initial risk content identification model is trained using the target first content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

[0038] The aforementioned risk content identification model update method, apparatus, computer equipment, storage medium, and computer program product involve the server inputting the content data to be identified into a pre-trained initial risk content identification model to obtain first content data that is identified by the initial risk content identification model as not containing risky content; obtaining user behavior features associated with the first content data, and filtering candidate first content data from the first content data based on the user behavior features; the candidate first content data being first content data containing potentially risky content; obtaining target first content data containing risky content from the candidate first content data based on risk assessment of the candidate first content data; and using the target first content data to train the initial risk content identification model and update the model parameters of the initial risk content identification model to obtain the target risk content identification model. This application's server can identify first content data that does not contain risky content through an initial risky content identification model. Then, it can utilize user behavior features associated with the first content data to filter out candidate first content data that may contain risky content. This candidate first content data undergoes risk review, resulting in target first content data that, after review, contains risky content. The target first content data can then be used to automatically iteratively train the initial risky content identification model without human intervention. This method utilizes content data that the initial risky content identification model missed in its initial model to automatically iteratively update and train the model, making it more effective at identifying new types of risky content and improving the accuracy of risky content identification. Attached Figure Description

[0039] Figure 1 This is an application environment diagram of the risk content identification model update method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a risk content identification model update method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating the process of filtering candidate first content data in one embodiment;

[0042] Figure 4This is a flowchart illustrating the process of obtaining a probability ranking of content containing risks in one embodiment.

[0043] Figure 5 This is a flowchart illustrating the process of obtaining the predicted probability of risk content in one embodiment;

[0044] Figure 6 This is a flowchart illustrating a method for proactively identifying novel risky content in one embodiment.

[0045] Figure 7 This is a flowchart illustrating the intelligent sampling process in one embodiment;

[0046] Figure 8 This is a schematic diagram illustrating the principle of the K-nearest neighbor algorithm in one embodiment;

[0047] Figure 9 This is an example diagram illustrating the distribution of a fixed sample size for review in one embodiment;

[0048] Figure 10 This is an example diagram illustrating the dynamic adjustment of the review volume distribution in one embodiment;

[0049] Figure 11 This is a schematic diagram of the process of automatically updating the model through machine learning in one embodiment;

[0050] Figure 12 This is a structural block diagram of a risk content recognition model update device in one embodiment;

[0051] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The risk content identification model update method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 101 communicates with server 102 via a network. Specifically, a user can publish content data to server 102 through their terminal 101. After receiving the content data, server 102 can input the content data into a pre-trained initial risk content identification model for identifying risky content. The initial risk content identification model first identifies first content data that does not contain risky content. Then, based on the user behavior characteristics associated with the first content data, it can filter out candidate first content data that may contain risky content from the first content data, and perform risk review on the aforementioned candidate first content data to determine the target first content data that actually contains risky content. Finally, the target first content data can be used to train the initial risk content identification model to obtain the target risk content identification model, thereby updating the risk content identification model. Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. Server 102 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0054] In one embodiment, such as Figure 2 As shown, a method for updating a risk content identification model is provided, which can be applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps:

[0055] Step S201: Input the content data to be identified into the pre-trained initial risk content identification model to obtain the first content data that is identified by the initial risk content identification model as not containing risk content.

[0056] The content data to be identified refers to the content data that needs to be identified as risky content. This content data can be content data published by a user to the server 102 through their terminal 101, such as images, text, audio, and video data published by the user. The initial risky content identification model is a pre-trained identification model used to identify risky content. This model can determine whether the content data contains risky content by extracting content features related to the content data to be identified. The first content data refers to the content data to be identified that does not contain risky content, as identified by the initial risky content identification model.

[0057] Specifically, when a user publishes content data to the server 102 through their terminal 101, the server 102 can first take the above-mentioned content data published by the user as the content data to be identified and input it into the pre-trained initial risk content identification model. The initial risk content identification model can then identify whether the content data contains risky content, and the content data that does not contain risky content is taken as the first content data.

[0058] Step S202: Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content.

[0059] User behavior characteristics associated with the first content data refer to user behavior characteristics related to the first content data. For example, this could include user behavior characteristics related to the user who published the first content data, or user feedback behavior after other users viewed the first content data. Candidate first content data, on the other hand, refers to content data that may contain risky content, i.e., content data containing potentially risky content. Since the initial risky content identification model may not accurately identify all content data as potentially risky, there is a possibility of missing risky content. That is, a content data package may contain risky content, but the initial risky content identification model may not identify it and mistakenly classify it as first content data. Candidate first content data refers to the content data among the aforementioned first content data that is most likely to have missed identifying risky content.

[0060] Specifically, after obtaining the first content data, the server 102 can further determine the user behavior characteristics associated with the first content data, such as the proportion of risky content and the ratio of punitive behaviors of the user who published the first content data. In this way, the server can use the above-mentioned user behavior characteristics to filter out candidate first content data that may contain risky content, i.e., contain potential risky content.

[0061] Step S203: Based on the risk assessment of the candidate first content data, the target first content data containing risky content is obtained from the candidate first content data.

[0062] Risk review refers to the review of content data for potential risks. This review can be conducted manually. The target primary content data is the candidate primary content data that, through risk review, contains risky content—that is, the portion of the primary content data for which risky content was missed. After obtaining the candidate primary content data, server 102 can send it for risk review and obtain the risk review results. From this, it can select the candidate primary content data that, according to the risk review results, contains risky content and use it as the target primary content data.

[0063] Step S204: Train the initial risk content identification model using the target first content data, and update the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0064] The target risk content identification model refers to the risk content identification model obtained after updating the initial risk content identification model. After determining the target first content data of the missed risk content in step S203, the server 102 can also use the target first content data to iteratively train the initial risk content identification model to update the model parameters of the initial risk content identification model, thereby obtaining the parameter-updated initial risk content identification model, i.e., the target risk content identification model, thus realizing the update of the risk content identification model.

[0065] In the aforementioned risk content identification model update method, server 102 inputs the content data to be identified into a pre-trained initial risk content identification model to obtain first content data that is identified by the initial risk content identification model as not containing risk content; obtains user behavior features associated with the first content data, and filters candidate first content data in the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content; based on the risk review of the candidate first content data, a target first content data containing risky content is obtained; the initial risk content identification model is trained using the target first content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model. In this application, server 102 can identify first content data that does not contain risky content through an initial risky content identification model. Then, it can use the user behavior features associated with the first content data to filter out candidate first content data that may contain risky content. The candidate first content data is then subject to risk review, and the target first content data that is reviewed and found to contain risky content is obtained. Thus, the initial risky content identification model can be automatically iteratively trained using the target first content data without human intervention. In this way, the initial risky content identification model can be automatically iteratively updated and trained using content data that the initial risky content identification model has missed identifying. This makes it easier for the risky content identification model to identify new types of risky content and improves the accuracy of risky content identification.

[0066] In one embodiment, such as Figure 3 As shown, step S202 may further include:

[0067] Step S301: Based on user behavior characteristics, obtain the probability ranking of the first content data containing risky content.

[0068] Probability ranking refers to the ranking of the probability that the first content data contains risky content. Specifically, after obtaining the user behavior characteristics associated with each piece of first content data, server 102 can predict the probability that each piece of first content data contains risky content based on the aforementioned user behavior characteristics, and rank the first content data according to the magnitude of the aforementioned probability.

[0069] Step S302: Obtain the number of candidate first content data to be filtered;

[0070] Step S303: Sort by probability and select candidate first content data of a certain number from the first content data.

[0071] The number of screenings refers to the number of candidate first content data to be screened. In order to reduce the workload required for risk review, the number of candidate first content data to be screened in this embodiment is limited. Therefore, in this embodiment, the server 102 can sort the candidate first content data according to probability to screen out candidate first content data that is appropriate to the number of screenings mentioned above, so as to ensure that the screened candidate first content data is the content data that is most likely to be missed by the initial risk content identification model. Therefore, the hit probability of the target first content data that has missed the identification of risk content can be increased.

[0072] For example, the first content data may include content data 1, content data 2 and content data 3, and the number of candidate first content data is 1. Then the server 102 can first obtain the probability ranking of the first content data containing risky content, which can be that the probability of content data 2 containing risky content > the probability of content data 1 containing risky content > the probability of content data 3 containing risky content. At this time, the server 102 can use content data 2 as the candidate first content data.

[0073] In this embodiment, the server 102 can sort the first content data according to the probability of containing risky content and the number of candidate first content data to be screened, and then select candidate first content data that is appropriate for the number of screenings. This reduces the workload required for risk review and further increases the probability of hitting the target first content data.

[0074] Furthermore, such as Figure 4 As shown, step S301 may further include:

[0075] Step S401: Obtain the sample user behavior characteristics associated with the first sample content data, and the risk sample data that actually contains risky content in the first sample content data.

[0076] The first sample content data refers to pre-collected sample content data, such as historically published content data. Sample user behavior characteristics refer to user behavior characteristics associated with the first sample content data, such as the behavior characteristics of the users who published the first sample content data, and the behavior characteristics of user feedback on the first sample content data. Risk sample data is the sample content data within the first sample content data that actually contains risky content; for example, it could be first sample content data carrying tags indicating the presence of risky content.

[0077] Specifically, the first sample content data may carry labels indicating whether it contains risky content. A label value of 0 indicates the absence of risky content, while a label value of 1 indicates the presence of risky content. After collecting the first sample content data, server 102 can also obtain the user behavior features associated with each piece of first sample content data, using these as sample user behavior features. Furthermore, it can filter out the first sample content data carrying label value 1, designating them as risky sample data.

[0078] Step S402: Based on the user behavior characteristics associated with the risk sample data, obtain the similarity between the first content data and the risk sample data, and obtain the predicted probability that the first content data contains risky content based on the similarity.

[0079] The similarity between the first content data and the risk sample data can be represented by the distance between them. Generally, the shorter the distance, the higher the similarity, and the greater the probability that the first content data contains risky content. Server 102 can calculate the distance between the first content data and each risk sample data based on the user behavior features associated with the first content data and the user behavior features of each sample. The average of these distances is then used as the similarity between the first content data and the risk data, thereby obtaining the predicted probability that the first content data contains risky content.

[0080] Step S403: Sort the first content data according to the predicted probability to obtain the probability sort.

[0081] Finally, server 102 can sort the first content data according to the order of predicted probabilities from largest to smallest, or it can sort the average distance between the first content data and each risk sample data in ascending order after obtaining the average distance, thereby obtaining the prediction sort.

[0082] In this embodiment, the server 102 can obtain a probability ranking of the first content data containing risky content based on the similarity between the first content data and the risk sample data, thereby further improving the accuracy of the probability ranking acquisition.

[0083] Furthermore, such as Figure 5 As shown, step S402 may further include:

[0084] Step S501: Obtain the current first content data and the current user behavior characteristics associated with the current first content data.

[0085] The current first content data refers to any one of the first content data, while the current user behavior feature refers to the user behavior feature associated with the current first content data. Server 102 can determine one of the multiple first content data as the current first content data, and use the user behavior feature associated with the current first content data as the current user behavior feature.

[0086] Step S502: Based on the current user behavior characteristics and the sample user behavior characteristics associated with the first sample content data, obtain the similarity ranking between the current first content data and the first sample content data, and obtain a first preset number of first sample content data from the similarity ranking.

[0087] Similarity ranking refers to the ranking of the similarity between the current first content data and the first sample content data, while the first preset number is a pre-set number of first sample content data obtained according to the similarity ranking, for example, this number can be 3. Specifically, after determining the current user behavior features associated with the current first content data, server 102 can use the aforementioned current user behavior features, as well as the sample user behavior features associated with each first sample content data, to calculate the similarity between the current first content data and the first sample content data. This process can be achieved by calculating the distance between the current first content data and the first sample content data, and can also obtain the first preset number of first sample content data with the highest similarity.

[0088] Step S503: If the first preset number of first sample content data contains at least the second preset number of risk sample data, obtain the predicted probability that the current first content data contains risk content; the second preset number is less than or equal to the first preset number.

[0089] The second preset quantity is a pre-set condition for determining whether to calculate the predicted probability that the current first content data contains risky content. Only when the first sample content data contains at least the second preset quantity of risky sample data, the probability that the current first content data contains risky content is relatively high, will the step of obtaining the predicted probability that the current first content data contains risky content be executed.

[0090] For example, the first preset quantity can be 3, and the second preset quantity can be 2. After obtaining the similarity between the current first content data and each first sample content data, the server 102 can sort them according to the similarity and determine the top 3 first sample content data with the highest similarity to the current first content data. Then, if at least 2 of the above 3 first sample content data contain risky sample data, it indicates that the probability of the current first content data containing risky content is relatively high. In this case, the server 102 will obtain the predicted probability that the current first content data contains risky content. However, if only 1 or no risky sample data is contained among the above 3 first sample content data, it indicates that the probability of the current first content data containing risky content is relatively low. In this case, the server 102 will not obtain the predicted probability that the current first content data contains risky content.

[0091] In this embodiment, the server 102 can first determine the first preset number of first sample content data that are most similar to each first content data, and determine whether the first sample content data contains at least a second preset number of risk sample data. Only when the above conditions are met will the process of obtaining the prediction probability that the current first content data contains risk content be executed. This method can reduce the number of prediction probabilities obtained and further improve the efficiency of prediction ranking.

[0092] In one embodiment, after step S201, the method may further include: acquiring second content data identified by the initial risk content identification model as containing potential risk content; step S302 may further include: acquiring a risk review quantity threshold and the quantity of second content data; and obtaining a screening quantity based on the risk review quantity threshold and the quantity of second content data.

[0093] The second content data refers to the content data to be identified that may contain risky content, i.e., content data containing potentially risky content, as identified by the initial risky content identification model. In this embodiment, the risky content identification result obtained by the initial risky content identification model can include the following three types: content data that does not contain risky content, i.e., the first content data; content data that contains potentially risky content, i.e., the second content data; and content data that is determined to contain risky content.

[0094] For the first set of content data, server 102 selects a portion for risk review, while for the second set of content data, all of it needs to undergo risk review. Since risk review is conducted manually, the number of pieces of content data subject to risk review is often limited; there is a risk review threshold. Therefore, to review as many candidate first-content data as possible, server 102 can determine the number of candidate first-content data to be selected based on the aforementioned risk review threshold and the identified second-content data.

[0095] For example, if the risk review threshold is 100, and the number of second content data identified in a certain time period is 30, then the number of candidate first content data to be screened can be 70. If the number of second content data identified in another time period is 90, then the number of candidate first content data to be screened can be 10.

[0096] In this way, compared to setting a fixed number of filters, server 102 can adaptively adjust the number of candidate first content data to be filtered based on the risk review threshold and the number of second content data, thereby further improving the intelligence of obtaining the number of candidate first content data to be filtered.

[0097] In one embodiment, step S202 may further include: obtaining the publishing user behavior characteristics of the publishing user corresponding to the first content data, and publishing the first content data within a preset range; obtaining the user feedback behavior characteristics after the first content data is published within the preset range; and using the publishing user behavior characteristics and the user feedback behavior characteristics as user behavior characteristics associated with the first content data.

[0098] The characteristics of publishing user behavior refer to the characteristics of the publishing behavior of the users who publish the first content data. For example, it may include the proportion of risky content published by the publishing user, the time of publication and the address of publication, etc. The characteristics of user feedback behavior refer to the characteristics of user feedback behavior after the first content data is published. For example, the proportion of reports and negative comments from other users regarding the published first content data, etc.

[0099] Meanwhile, since the first content data may be content data that the initial risk content identification model missed identifying as risky content, and server 102 needs to collect user feedback behavior characteristics regarding the first content data, server 102 can publish the first content data only within a pre-defined small scope. For example, it can display the published first content data only in a specific module interface, instead of displaying it in all module interfaces, or it can display the first content data only within a specific time period, instead of publishing and displaying it throughout all time periods. Furthermore, server 102 can also collect user feedback behavior characteristics after publication within the preset scope, thereby using the published user behavior characteristics and the user feedback behavior characteristics after publication within the preset scope as user behavior characteristics associated with the first content data.

[0100] In this embodiment, after obtaining the first content data, the server 102 can publish the first content data only within a preset range and collect the corresponding user feedback behavior characteristics. The aforementioned user feedback behavior characteristics and the publishing user behavior characteristics are used as user behavior characteristics associated with the first content data, thereby further improving the completeness of the user behavior characteristics associated with the first content data.

[0101] In one embodiment, after step S201, the method may further include: acquiring second content data identified by the initial risk content identification model as containing potentially risky content; obtaining target second content data that does not contain risky content based on risk review of the second content data; step S204 may further include: training the initial risk content identification model using the target first content data and the target second content data, and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0102] The second content data refers to the content data to be identified that may contain risky content, as identified by the initial risk content identification model; that is, content data containing potentially risky content. The target second content data, on the other hand, is the second content data that, through risk review, is determined not to contain risky content; that is, the portion of the second content data where risky content was mistakenly detected. In this embodiment, after identifying potentially risky second content data, the server 102 can send all the second content data for risk review and obtain the risk review results. From these results, the server can select the second content data whose risk review result indicates it does not contain risky content as the target second content data.

[0103] Subsequently, in order to further improve the recognition accuracy of the risk content recognition model, in addition to utilizing the target first content data that the initial risk content recognition model missed detecting, the server 102 can also use the target second content data that the initial risk content recognition model erroneously detected to train the risk content recognition model. That is, the initial risk content recognition model is trained simultaneously using both the target first content data and the target second content data to update the model parameters of the initial risk content recognition model, so as to obtain the target risk content recognition model, thereby further improving the risk content recognition accuracy of the obtained target risk content recognition model.

[0104] In this embodiment, the server 102 can simultaneously use the target first content data that the initial risk content recognition model missed detecting, and the target second content data that the initial risk content recognition model erroneously detected, to train the risk content recognition model, thereby further improving the risk content recognition accuracy of the obtained target risk content recognition model.

[0105] Furthermore, the initial risk content identification model is trained using the second sample content data; training the initial risk content identification model using the target first content data and the target second content data, and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model, may further include: inputting the second sample content data, the target first content data, and the target second content data into the initial risk content identification model, obtaining the content features of the second sample content data, the target first content data, and the target second content data through the initial risk content identification model; training the initial risk content identification model based on the content features, and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0106] The second sample content data is the sample content data used to train the initial risk content recognition model. In this embodiment, the server 102 can collect the second sample content data in advance and use the second sample content data to train the initial risk content recognition model. For example, it can extract the content features related to the second sample content data, so that the training of the initial risk content recognition model can be completed based on the content features.

[0107] Subsequently, when updating the initial risk content identification model, the second sample content data, the first target content data, and the second target content data can be simultaneously input into the initial risk content identification model. The initial risk content identification model extracts content features from the second sample content data, the first target content data, and the second target content data. Based on these extracted content features, the initial risk content identification model can be trained and its parameters updated to obtain the target risk content identification model. Compared to training the initial risk content identification model using only the first and second target content data, this embodiment can further improve the model training effect, thereby further improving the risk content identification accuracy of the target risk content identification model.

[0108] In this embodiment, the training of the initial risk content recognition model can be achieved by using the second sample content data, the first target content data, and the second target content data used to train the initial risk content recognition model, which can further improve the model training effect and thus further improve the risk content recognition accuracy of the target risk content recognition model.

[0109] In one embodiment, after step S204, the method may further include: obtaining model verification results of the initial risk content identification model and the target risk content identification model; if the model verification results indicate that the model performance of the target risk content identification model is better than that of the initial risk content identification model, using the target risk content identification model as the updated initial risk content identification model; and / or if the model verification results indicate that the model performance of the initial risk content identification model is better than that of the target risk content identification model, stopping the updating of the initial risk content identification model.

[0110] The model validation result refers to the validation result obtained by validating the performance of the initial risk content identification model and the target risk content identification model. In this embodiment, after obtaining the target risk content identification model, the server 102 can also perform model evaluation and validation on the target risk content identification model to determine whether its performance is superior to that of the initial risk content identification model. This performance validation can be a comparison of the accuracy of the target risk content identification model and the initial risk content identification model in identifying risk content. If the performance of the target risk content identification model is superior to that of the initial risk content identification model, then the server 102 can use the target risk content identification model as the updated initial risk content identification model and perform the next iteration update. If the performance of the initial risk content identification model is superior to that of the target risk content identification model, then the server 102 can discard the target risk content identification model, that is, stop the update process of the initial risk content identification model.

[0111] In this embodiment, after updating the risk content identification model, the updated target risk content identification model and the initial risk content identification model before the update can be validated, and the model performance of the initial risk content identification model and the target risk content identification model can be compared to determine whether to retain the target risk content identification model as the updated risk content identification model, thereby further ensuring the accuracy of the updated risk content identification model in identifying risk content.

[0112] In one embodiment, a method for proactively identifying novel risky content is also provided, such as... Figure 6 As shown, the method may include the following steps:

[0113] 1. Machine Review - Content Recognition: The first step is to use an artificial intelligence model to identify images, text, audio, video, and other content to be recognized. The content to be recognized is divided into three categories: no risk found, risky, and suspected risky. Content data identified as risky is directly deleted.

[0114] 2. Limited release: Content for which the model has not identified any risks will not be released directly to the entire site, but will be released to a limited number of users, who will only be able to see this content first.

[0115] 3. Intelligent sampling inspection: Collect user behavior information published in a small range, such as author behavior attributes and audience behavior attributes, assess the potential risks of the content, sort and sample the risks, and finally enter the manual risk review system.

[0116] 4. Risk Review: In addition to manually verifying content deemed potentially risky by the automated review process, content identified through intelligent random checks is also reviewed. Content deemed risky is deleted directly, while risk-free content is published normally.

[0117] 5. Automated Machine Learning: Collect data on missed recall risks and potentially risky false recalls discovered during sampling, update and iterate the content recognition model, and replace the old model with the new model when the performance of the new model surpasses that of the old model.

[0118] 6. Quality Inspection and Evaluation: Sampling and secondary verification of manually reviewed data to promptly identify any oversights in the risk assessment.

[0119] Among them, intelligent sampling can improve the efficiency of sampling, increase the probability of selecting risky content, and accelerate the accumulation of new risky samples. Its working principle is as follows: Figure 7 As shown, the core process can include the following two points:

[0120] (1) By collecting audience behavior attributes and author behavior attributes, a machine learning model is established to assess the potential risk coefficient of published content and rank them. By assessing the risk coefficient, the proportion of risky content selected is increased.

[0121] (2) Adjust the number of random inspections dynamically according to the workload of risk auditing, and increase the number of random inspections as much as possible without increasing the number of auditing personnel.

[0122] In summary, without increasing the cost of random sampling and review, we can increase the number of random samplings and the proportion of harmful content detected, thereby improving the speed and quantity of identifying new types of risky content. This will provide as many data samples as possible for subsequent automated machine learning.

[0123] The ranking of potential risk coefficients can be achieved by building a ranking model. The primary consideration in this process is the model input parameters. Since the content recognition model has already been used to set labels for the content to be identified in the first step of the review process (machine review - content recognition), the ranking model is no longer built from the content recognition dimension. Instead, the ranking model is built based on user behavior data, so that the machine review model and the ranking model can complement each other.

[0124] For example, the input parameters of the ranking model include user behavior attributes collected after the content to be identified is published on a small scale. These can include: the user report ratio (number of reports / number of views); the negative comment ratio (number of negative comments / total number of comments). The system can pre-train a model to judge comment trends, labeling each comment with three categories: positive, negative, and general comments; high-frequency periods (the periods with the most frequent user interaction); and the proportion of active users from overseas addresses (number of active users from overseas addresses / total number of active users).

[0125] In addition, the ranking model will also take into account the attributes of the authors of the content to be identified, which may include: the number of times they have been penalized for publishing risky content in the past; the proportion of risky content published, which is the amount of risky content published in the past / the total number of publications by the author; whether they are real-name authenticated, whether they are authenticated users, and whether they are overseas users, etc.

[0126] After collecting the above data, it needs to be normalized and then uniformly denoted as the input feature vector V. Training samples are then compiled from historical review records and organized into...<V,label> The data pairs are defined as follows: V is a feature vector composed of the above attributes, and label is an identifier indicating whether the content to be identified contains risky content, where 1 indicates that it contains risky content and 0 indicates that it does not contain risky content.

[0127] exist<V,label> On the training samples in the specified format, various machine learning models can be built and trained. In this embodiment, the K-Nearest Neighbor (KNN) algorithm is used as an example to build a ranking model.

[0128] Inference Phase: Let K be 3. Calculate the Euclidean distance between the unsorted data V and each sample point in the training set. Select the three nearest neighbor data points. Determine whether the unsorted data V is risky based on the labels of these three data points. If two or three points have a label of 1, then the unsorted data V is considered risky data; otherwise, it is considered normal data. If it is risky data, calculate the average distance d from the unsorted data V to each risky sample point. For example... Figure 8 As shown, the circle represents the data V to be sorted, the triangle represents the risk data points, and the square represents the non-risk data points. It can be seen that among the three nearest data points, there are two risk data points. Therefore, the data V to be sorted is risk data.

[0129] Output: For all unsorted data, sort them according to the average distance d calculated during the inference process. The smaller the average distance d, the higher the risk coefficient.

[0130] Regarding dynamically adjusting the number of samples for random checks, generally speaking, evenings are peak user activity times, and the number of works to be identified is also high during this period, resulting in a large workload for review. Conversely, midnight and early morning are off-peak periods with fewer active users and less content being posted, leading to a lower workload for review. The system's review workload fluctuates over time, but since the number of risk reviews is fixed, simply setting a sampling ratio or number of samples during random checks could easily lead to... Figure 9 This situation arises because during peak periods, the system exceeds its maximum capacity, causing delays in the review process and hindering the timely review of user-submitted content, thus impacting the user experience. Conversely, during off-peak periods, there are fewer review tasks, leaving review resources idle.

[0131] Therefore, this embodiment can obtain the number of suspected cases submitted for review during the machine review process, such as... Figure 10 As shown, more spot checks are conducted during off-peak periods, and fewer or no checks are conducted during peak periods, dynamically adjusting the number of checks to ensure that the total number of reviews is roughly the same across different time periods. This ensures both timely review and maximizes the utilization of review resources. It significantly increases the number of spot checks, thereby improving the speed and quantity of risky content detection.

[0132] For the automated machine learning component, the current model used for content recognition in machine-based content review is trained from a large number of samples using deep learning methods. This model can use the risky content features of the training samples to predict unseen samples and has a strong ability to identify similar samples. However, it is weak in identifying novel and optimized risky samples, often missing new harmful content that the model judges as risk-free, only to discover it during the risk review process. After intelligent sampling and risk review steps, the system can discover and accumulate a small number of missed risky samples every day. Therefore, these newly discovered risky samples can be used to quickly iterate and update the deep learning recognition model, improving the new model's ability to identify and process new risky samples.

[0133] like Figure 11 As shown, the process of automatically updating a model through machine learning may include the following steps:

[0134] Step 1: Offline collection of training set and training of the first version of the model: First, collect and organize the training sample set. This sample set is widely available and large in scale. It can be collected from the Internet or from historical business data. Generally, tens of thousands or even hundreds of thousands of image training samples can be accumulated. Train and optimize the image classification model on the training sample set. After reaching the optimal state, release the first version of Model 1 for machine review - content recognition.

[0135] Step 2: Collect daily incremental risk data: Collect daily review data from the risk review platform, organize it and use it for incremental learning. The data here includes: data from the sampling system and risk review confirmed the risk, which is also the risk data missed by Model 1; and data from machine review suspected of being risky but risk review confirmed that there was no risk, which is the data wrongly recalled by Model 1.

[0136] Step 3: Based on Model 1, incremental learning is performed using the accumulated new sample data to train Model X.

[0137] Step 4: Evaluate the recognition performance of Model X. If it surpasses Model 1, release it immediately and replace Model 1; if it is worse than Model 1, abandon Model X.

[0138] Step 1 only needs to be performed once; steps 2-4 are performed once a day, and the cycle repeats.

[0139] Incremental learning is a method that continuously learns new knowledge from new data. It can train iterative models in continuous data streams without constraints such as data storage and privacy protection, making it particularly suitable for iterative improvement of content recognition models in content moderation systems. Furthermore, incremental learning does not require adding output categories or modifying the model structure; it only needs to train and update Model 1 with newly collected risky and harmful data to enhance its ability to identify new types of risky content. However, it is undesirable for the model to lose or weaken its ability to identify older risky content. Based on the scale and source of the training data, the incremental learning methods used in the system can be simply divided into the following three categories:

[0140] Fine-tuning: Model 1 is fine-tuned using only accumulated daily incremental data. After fine-tuning the model parameters, model X is obtained, which significantly improves the ability of model X to identify new types of harmful content. This type of method uses less training data, trains quickly, and has low cost. It has a good ability to identify new types of risky and harmful content, but its ability to identify old risky content may deteriorate, resulting in mediocre overall performance.

[0141] Joint Training: This method uses incremental data and all existing training data to train and adjust the parameters of Model 1. This method has more training data and can take into account the identification of both new and old risk content. The overall performance of the model is better, but the training time is long and the cost is high.

[0142] Replay learning: This method trains the model using incremental data and a portion of existing data. It's relatively fast to train and the model performs well in both identifying new and old risk content, resulting in good overall performance. The key to this approach is selecting high-quality samples from a large set of existing data for iterative model training.

[0143] The characteristics of the above incremental learning methods are shown in Table 1. This embodiment adopts a joint training approach to achieve incremental learning in order to maximize the overall effect of the model.

[0144]

[0145] Table 1 Characteristics of Incremental Learning

[0146] In this embodiment, more new types of risky content can be quickly and cost-effectively discovered through intelligent sampling and risk ranking models. Furthermore, by using automated machine learning methods, the content recognition model can be rapidly iterated to quickly improve the detection capability for new types of risky content.

[0147] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0148] Based on the same inventive concept, this application also provides a risk content identification model updating device for implementing the risk content identification model updating method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more risk content identification model updating device embodiments provided below can be found in the limitations of the risk content identification model updating method described above, and will not be repeated here.

[0149] In one embodiment, such as Figure 12 As shown, a risk content identification model update device is provided, comprising: a first content acquisition module 1201, a candidate content filtering module 1202, a target data acquisition module 1203, and an identification model update module 1204, wherein:

[0150] The first content acquisition module 1201 is used to input the content data to be identified into the pre-trained initial risk content identification model and obtain the first content data that is identified by the initial risk content identification model as not containing risk content.

[0151] The candidate content filtering module 1202 is used to obtain user behavior characteristics associated with the first content data, and filter candidate first content data in the first content data according to the user behavior characteristics; the candidate first content data is first content data containing potentially risky content.

[0152] The target data acquisition module 1203 is used to obtain target first content data containing risky content from the candidate first content data based on the risk assessment of the candidate first content data.

[0153] The identification model update module 1204 is used to train the initial risk content identification model using the target first content data, and update the model parameters of the initial risk content identification model to obtain the target risk content identification model.

[0154] Each module in the aforementioned risk content identification model update device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0155] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores content data to be identified. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a risk content identification model update method.

[0156] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0158] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0159] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0161] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for updating a risk content identification model, characterized in that, Applied to a server, the method includes: The content data to be identified is input into a pre-trained initial risk content identification model to obtain first content data that is identified by the initial risk content identification model as not containing risk content, and second content data that is identified by the initial risk content identification model as containing potential risk content. Obtain user behavior features associated with the first content data, and filter candidate first content data from the first content data based on the user behavior features; the candidate first content data is first content data containing potentially risky content; the number of candidate first content data to be filtered is obtained based on the risk review threshold and the number of second content data, and the candidate first content data is obtained based on the probability that the first content data contains risky content; Based on the risk assessment of the candidate first content data and the second content data, target first content data containing risky content is obtained, and target second content data not containing risky content is obtained; the target first content data is the content data in the first content data that omits the detection of risky content, and the target second content data is the content data in the second content data that incorrectly detects risky content. The initial risk content identification model is trained using the target first content data and the target second content data, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

2. The method according to claim 1, characterized in that, The step of obtaining user behavior features associated with the first content data and filtering candidate first content data from the first content data based on the user behavior features includes: Based on the user behavior characteristics, obtain the probability ranking of the first content data containing risky content; Obtain the number of candidate first content data to be filtered; According to the probability sorting, the number of candidate first content data to be selected is selected from the first content data.

3. The method according to claim 2, characterized in that, The step of obtaining the probability ranking of the first content data containing risky content based on the user behavior characteristics includes: Obtain the sample user behavior characteristics associated with the first sample content data, and the risk sample data that actually contains risky content in the first sample content data; Based on the user behavior characteristics associated with the risk sample data, the similarity between the first content data and the risk sample data is obtained, and the predicted probability that the first content data contains risky content is obtained based on the similarity. The first content data is sorted according to the predicted probabilities to obtain the probability sort.

4. The method according to claim 3, characterized in that, The step of obtaining the similarity between the first content data and the risk sample data based on the sample user behavior features associated with the user behavior features and the risk sample data, and obtaining the predicted probability that the first content data contains risky content based on the similarity, includes: Obtain the current first content data, and the current user behavior characteristics associated with the current first content data; Based on the current user behavior characteristics and the sample user behavior characteristics associated with the first sample content data, obtain the similarity ranking between the current first content data and the first sample content data, and obtain a first preset number of first sample content data from the similarity ranking; If the first preset number of first sample content data contains at least a second preset number of risk sample data, the predicted probability that the current first content data contains risk content is obtained; the second preset number is less than or equal to the first preset number.

5. The method according to claim 2, characterized in that, After inputting the content data to be identified into the pre-trained initial risk content identification model, the method further includes: Obtain second content data that is identified by the initial risk content identification model as containing potentially risky content; The number of times the candidate first content data is obtained for filtering includes: Obtain the threshold for the number of risk audits, and the number of the second content data; The number of samples to be filtered is obtained based on the risk review threshold and the quantity of the second content data.

6. The method according to claim 1, characterized in that, The step of obtaining the user behavior features associated with the first content data includes: Obtain the publishing user behavior characteristics of the publishing user corresponding to the first content data, and publish the first content data within a preset range; Acquire user feedback behavior characteristics after the first content data is published within the preset range; The published user behavior characteristics and the user feedback behavior characteristics are used as the user behavior characteristics associated with the first content data.

7. The method according to claim 1, characterized in that, The initial risk content identification model is trained using the second sample content data; the second sample content data is the sample content data used to train the initial risk content identification model. The step of training the initial risk content identification model using the target first content data and the target second content data, and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model, includes: The second sample content data, the first target content data, and the second target content data are input into the initial risk content identification model, and the content features of the second sample content data, the first target content data, and the second target content data are obtained through the initial risk content identification model. The initial risk content identification model is trained based on the content features, and the model parameters of the initial risk content identification model are updated to obtain the target risk content identification model.

8. The method according to claim 1, characterized in that, After training the initial risk content identification model using the target first content data and updating the model parameters of the initial risk content identification model to obtain the target risk content identification model, the method further includes: Obtain the model validation results of the initial risk content identification model and the target risk content identification model; If the model validation results indicate that the performance of the target risk content identification model is better than that of the initial risk content identification model, the target risk content identification model shall be used as the updated initial risk content identification model. and / or If the model validation results indicate that the performance of the initial risk content identification model is better than that of the target risk content identification model, then the initial risk content identification model shall be stopped from being updated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images