Material auditing method and device

Abstract

The application discloses a material auditing method and device. The method comprises the following steps: periodically determining the type of a target intelligent chat robot; when the target intelligent chat robot is a first type of intelligent chat robot created in a first period, determining a target authentication service partner platform to which the target intelligent chat robot belongs, determining a first rating of the target authentication service partner platform, determining a target rating of the target intelligent chat robot based on the first rating; when the target intelligent chat robot is a second type of intelligent chat robot created before the first period, obtaining first bad behavior statistical data of the target intelligent chat robot, determining the target rating of the target intelligent chat robot based on the existing rating of the target intelligent chat robot and the first bad behavior statistical data; and determining a target auditing mode of a target material based on the target rating. The application solves the technical problem of low material auditing efficiency for intelligent chat robots in the related art.

Description

Technical Field

[0001] This application relates to the field of 5G message management technology, and more specifically, to a method and apparatus for material review. Background Technology

[0002] With the rapid development of technology, operators have launched 5G messaging, which means that we have officially entered the 5G era.

[0003] As more and more users register on the 5G messaging platform, the platform faces a significant challenge in reviewing a large volume of data daily. Furthermore, there is currently no clear standard for judging the credibility of materials provided by chatbots, and the review of such materials is currently limited to human intervention; no clear system review plan exists, resulting in low efficiency in reviewing chatbot-provided materials.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for reviewing submitted materials, which at least solves the technical problem of low review efficiency for materials submitted by intelligent chatbots in related technologies.

[0006] According to one aspect of the embodiments of this application, a material review method is provided, comprising: periodically determining the type of a target intelligent chatbot; when the target intelligent chatbot is a first type of intelligent chatbot created within a first period, determining the target certification service partner platform to which the target intelligent chatbot belongs, and determining a first rating of the target certification service partner platform, and determining a target rating of the target intelligent chatbot based on the first rating; when the target intelligent chatbot is a second type of intelligent chatbot created before the first period, obtaining first misconduct statistics of the target intelligent chatbot, and determining a target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first misconduct statistics; and determining a target review method for target materials based on the target rating, wherein the target materials are materials submitted by the target intelligent chatbot.

[0007] Optionally, determining the first rating of the target certification service partner platform includes: determining the target preset rating of the target certification service partner platform from a preset platform rating table, wherein the platform rating table includes preset ratings of multiple certification service partner platforms; obtaining second misconduct statistics of the target certification service partner platform, wherein the second misconduct statistics include the number of first misconducts of the target certification service partner platform in a second period and the number of second misconducts of the target certification service partner platform in a first time period, the second period being the period preceding the first period, and the first time period including all periods before the second period; and determining the first rating of the target certification service partner platform based on the target preset rating, the number of first misconducts, and the number of second misconducts.

[0008] Optionally, the target preset rating is either Level 1 or Level 2, with Level 1 being higher than Level 2. The first rating of the target certification service partner platform is determined based on the target preset rating, the number of first-type misconducts, and the number of second-type misconducts, including: when the target preset rating is Level 1, if the number of first-type misconducts is greater than a first preset threshold and the number of second-type misconducts is greater than a second preset threshold, the target certification service partner platform's first rating is determined to be Level 2; otherwise, the target certification service partner platform's first rating is determined to be Level 1. When the target preset rating is Level 2, if the number of first-type misconducts is less than a third preset threshold and the number of second-type misconducts is less than a fourth preset threshold, the target certification service partner platform's first rating is determined to be Level 1; otherwise, the target certification service partner platform's first rating is determined to be Level 2.

[0009] Optionally, the target rating may include one of the following: the third, fourth, fifth and sixth levels, which are successively lower in level; the target rating of the target intelligent chatbot may be determined based on the first rating, including: when the first rating of the target certification service partner platform is the first level, the target rating of the target intelligent chatbot may be determined to be the fourth level; when the first rating of the target certification service partner platform is the second level, the target rating of the target intelligent chatbot may be determined to be the fifth level.

[0010] Optionally, misconduct by the target authentication service partner platform includes one of the following: adding a new user to the target authentication service partner platform but failing the review, changing the user's ownership but failing the review, adding a new intelligent chatbot but failing the review, or changing the ownership of an intelligent chatbot but failing the review.

[0011] Optionally, the first set of undesirable behavior statistics includes: the number of the target intelligent chatbot's third undesirable behavior in the second period and the number of the target intelligent chatbot's fourth undesirable behavior in the first time period. The second period is the period preceding the first period, and the first time period includes all periods before the second period and the second period. The target intelligent chatbot's existing rating includes one of the following: the third level, the fourth level, the fifth level, and the sixth level, which are successively lower in level.

[0012] Optionally, the target rating includes one of the following: Level 3, Level 4, Level 5, and Level 6; the target rating of the target intelligent chatbot is determined based on its existing rating and the statistical data of the first undesirable behavior, including: when the existing rating is Level 3, if the number of third undesirable behaviors is greater than the fifth preset threshold and the number of fourth undesirable behaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 4; otherwise, the target rating of the target intelligent chatbot is determined to be Level 3; when the existing rating is Level 4, if the number of third undesirable behaviors is greater than the fifth preset threshold and the number of fourth undesirable behaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 5; if the number of third undesirable behaviors is less than the seventh preset threshold and the number of fourth undesirable behaviors is less than the eighth preset threshold, the target intelligent chatbot is determined to be Level 5. The target rating for the chatbot is Level 3; otherwise, the target rating for the chatbot is Level 4. If the existing rating is Level 5, and the number of third-degree misconduct behaviors exceeds the fifth preset threshold and the number of fourth-degree misconduct behaviors exceeds the sixth preset threshold, the target rating for the chatbot is Level 6. If the number of third-degree misconduct behaviors is less than the seventh preset threshold and the number of fourth-degree misconduct behaviors is less than the eighth preset threshold, the target rating for the chatbot is Level 4; otherwise, the target rating for the chatbot is Level 5. If the existing rating is Level 6, and the number of third-degree misconduct behaviors is less than the seventh preset threshold and the number of fourth-degree misconduct behaviors is less than the eighth preset threshold, the target rating for the chatbot is Level 5; otherwise, the target rating for the chatbot is Level 6.

[0013] Optionally, the misconduct of the target intelligent chatbot includes one of the following: the target intelligent chatbot adds a new menu but fails to pass the review, the target intelligent chatbot submits materials but fails to pass the review, or a complaint is received against the target intelligent chatbot.

[0014] Optionally, the target rating includes one of the following: a third, fourth, fifth, and sixth level, with the levels decreasing sequentially. Based on the target rating, the target review method for the target material is determined, including: when the target rating is the third level, the target review method is determined to be manual review and given priority; when the target rating is the fourth level, the target review method is determined to be manual review; when the target rating is the fifth level, the target review method is determined to be a combination of manual review and artificial intelligence review; when the target rating is the sixth level, the target review method is determined to be a combination of manual review and artificial intelligence review, and an alarm message is generated to indicate low reliability of the material.

[0015] Optionally, if the target intelligent chatbot has a target rating of level six for a consecutive preset number of periods, the target intelligent chatbot will be added to the blacklist.

[0016] According to another aspect of the embodiments of this application, a material review device is also provided, comprising: a first determining module, configured to periodically determine the type of a target intelligent chatbot; a second determining module, configured to, when the target intelligent chatbot is a first type of intelligent chatbot created within a first period, determine the target certification service partner platform to which the target intelligent chatbot belongs, and determine a first rating of the target certification service partner platform, and determine a target rating of the target intelligent chatbot based on the first rating; a third determining module, configured to, when the target intelligent chatbot is a second type of intelligent chatbot created before the first period, obtain first misconduct statistics of the target intelligent chatbot, and determine a target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first misconduct statistics; and a fourth determining module, configured to determine a target review method for target materials based on the target rating, wherein the target materials are materials submitted by the target intelligent chatbot.

[0017] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein the device where the non-volatile storage medium is located executes the above-described material review method by running the program.

[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described material review method through the computer program.

[0019] In this embodiment, the type of the target intelligent chatbot is periodically determined. When the target intelligent chatbot is a first-type intelligent chatbot created within a first period, the target certification service partner platform to which the target intelligent chatbot belongs is determined, and the first rating of the target certification service partner platform is determined. Based on the first rating, the target rating of the target intelligent chatbot is determined. When the target intelligent chatbot is a second-type intelligent chatbot created before the first period, the first misconduct statistics of the target intelligent chatbot are obtained. Based on the existing rating of the target intelligent chatbot and the first misconduct statistics, the target rating of the target intelligent chatbot is determined. Based on the target rating, the target review method for target materials is determined, wherein the target materials are the materials submitted by the target intelligent chatbot. This enables targeted review of materials submitted by intelligent chatbots based on their target rating, thereby ensuring system control over intelligent chatbots and solving the technical problem of low review efficiency for materials submitted by intelligent chatbots in related technologies. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a flowchart illustrating an optional material review method according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of an optional process for determining the first rating of a target authentication service partner platform according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of an optional process for determining the target rating of a target intelligent chatbot according to an embodiment of this application;

[0024] Figure 4 This is a flowchart illustrating an optional method for determining target materials for review, according to an embodiment of this application.

[0025] Figure 5 This is a flowchart illustrating another optional material review method according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of an optional material review device according to an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:

[0030] CSP (Certified Service Partner): refers to a 5G messaging operation platform jointly built between operators and various ecosystem enterprises. It is an operation tool platform used to provide chatbot services. As the center of 5G message sending, operation and management, it is mainly composed of MaaP access module, session adapter, intent recognition engine, business process configuration and other modules.

[0031] Example 1

[0032] As the number of customers joining the 5G messaging operation platform continues to increase, the amount of uploaded materials also increases, resulting in inconsistent quality. Because the intelligent chatbot on the 5G messaging operation platform lacks clear standards for the provided materials, reviewers are required to specifically review materials that violate regulations. This leads to significant time and manpower costs, and the presence of low-quality or non-compliant materials also prevents some high-quality materials from receiving timely review.

[0033] To address the aforementioned issues, this application provides a material review method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 1 This is a flowchart illustrating an optional material review method according to an embodiment of this application, such as... Figure 1 As shown, the method includes at least steps S102-S108, wherein:

[0035] Step S102: Periodically determine the type of the target intelligent chatbot.

[0036] Given the large volume of materials uploaded to the 5G messaging platform, a significant amount of material submitted by intelligent chatbots needs to be reviewed daily by reviewers. To improve the efficiency of message material review, the type of target intelligent chatbot is periodically determined, thereby enabling systemic control over the target intelligent chatbots.

[0037] In this embodiment of the application, the type of intelligent chatbot can be determined periodically, for example, by updating the type of intelligent chatbot daily, so that reviewers can review the materials provided by the intelligent chatbot in a targeted manner according to the type of intelligent chatbot.

[0038] Specifically, high-quality materials submitted by high-level intelligent chatbots can be prioritized for review and the review process can be simplified; while for low-quality materials submitted by low-level intelligent chatbots, the review process will be appropriately lengthened to avoid mis-review of materials.

[0039] Step S104: When the target intelligent chatbot is a first-class intelligent chatbot created in the first cycle, determine the target certification service partner platform to which the target intelligent chatbot belongs, determine the first rating of the target certification service partner platform, and determine the target rating of the target intelligent chatbot based on the first rating.

[0040] Specifically, when Chatbot creates its first type of intelligent chatbot for a user on the 5G messaging platform on the same day, the platform will determine the first rating of the Csp (Certified Service Partner Platform) to which Chatbot belongs, and determine the target rating of Chatbot based on the first rating of the Csp.

[0041] As an optional implementation of this application, Figure 2 A flowchart illustrating an optional method for determining the first rating of a target certification service partner platform is shown, such as... Figure 2As shown, the first rating of Csp can be determined through steps S1041-S1043:

[0042] Step S1041: Determine the target preset rating of the target certification service partner platform from the preset platform rating table, wherein the platform rating table includes preset ratings of multiple certification service partner platforms.

[0043] Typically, a CSP platform is a 5G messaging operation platform jointly built by operators and various ecosystem companies. Therefore, the CSP platform holds crucial commercial value for enterprises, providing a unified message management and operation portal for refined operations. It also allows enterprises to avoid building their own AI capabilities through visual chatbot design tools. Furthermore, it can deploy cloud + private solutions to meet diverse data security needs. Therefore, CSP platforms can be pre-classified into either Level 1 or Level 2, with Level 1 being higher than Level 2. For example, when a CSP platform serves a large enterprise with tens of millions of transactions, such as banks, e-commerce logistics, government and state-owned assets, or operators, it would be rated Level 1. Conversely, when a CSP platform serves small and micro-enterprises, due to the uncertainty of the information they publish, it could be rated Level 2.

[0044] Step S1042: Obtain the second set of misconduct statistics of the target certification service partner platform. The second set of misconduct statistics includes the number of first misconducts of the target certification service partner platform in the second period and the number of second misconducts of the target certification service partner platform in the first time period. The second period is the period before the first period, and the first time period includes all periods before the second period and the second period.

[0045] Among them, the misconduct of the Target Authentication Service Partner Platform includes one of the following: adding new users under the Target Authentication Service Partner Platform but failing the review, changing user ownership but failing the review, adding new intelligent chatbots but failing the review, or changing the ownership of intelligent chatbots but failing the review.

[0046] For example, the CSP platform's misconduct data table is updated daily. Issues such as adding new users but failing the review process, changing user ownership but failing the review process, or adding new intelligent chatbots but failing the review process will all affect the CSP platform's target preset rating. Therefore, we can obtain the number of times the aforementioned misconduct occurred on the CSP platform yesterday (first misconduct) and the number of times it occurred before today (second misconduct). The second misconduct statistics for the CSP platform are determined based on the number of first and second misconduct occurrences.

[0047] Step S1043: Determine the first rating of the target certification service partner platform based on the target preset rating, the number of first misconducts, and the number of second misconducts.

[0048] Optionally, when the target preset rating is Level 1, if the number of first misconducts is greater than the first preset threshold and the number of second misconducts is greater than the second preset threshold, the target certification service partner platform's first rating is determined to be Level 2; otherwise, the target certification service partner platform's first rating is determined to be Level 1. When the target preset rating is Level 2, if the number of first misconducts is less than the third preset threshold and the number of second misconducts is less than the fourth preset threshold, the target certification service partner platform's first rating is determined to be Level 1; otherwise, the target certification service partner platform's first rating is determined to be Level 2.

[0049] Specifically, if the number of first-time misconducts on the Csp platform yesterday falls within the preset threshold range [a, b], and the number of second-time misconducts on the Csp platform before today also falls within the preset threshold range [c, d], the original rating remains unchanged. However, if the number of first-time misconducts on the Csp platform yesterday exceeds the preset threshold b, and the number of second-time misconducts on the Csp platform before today exceeds the preset threshold d, it indicates that the number of misconducts on the Csp platform has decreased, meaning there are fewer materials that fail the review, and the original rating needs to be downgraded.

[0050] For example, if the CSP platform is a large bank or financial enterprise, that is, if the target preset rating of the CSP platform is the first level X, if the number of the first bad behavior that occurred yesterday is greater than the first preset threshold b, and the number of the second bad behavior that occurred before today is greater than the second preset threshold d, then the first rating of the CSP platform is determined to be the second level Y; otherwise, the first rating of the CSP platform is determined to be the first level X.

[0051] If the number of first-time misconduct instances on the CSP platform yesterday is less than the preset threshold 'a' and the number of second-time misconduct instances on the CSP platform before today is less than the preset threshold 'c', it indicates that the number of misconduct instances on the CSP platform has increased, meaning there are many materials that have failed the review and the original level needs to be upgraded.

[0052] For example, when the Csp platform is for some micro and small enterprises, if the number of the first bad behavior that occurred yesterday is less than the third preset threshold a, and the number of the second bad behavior that occurred before today is less than the fourth preset threshold c, the first rating of the Csp platform is determined to be the first level X; otherwise, the first rating of the Csp platform is determined to be the second level Y.

[0053] After determining the first rating of the target authentication service partner platform, the target rating for the next creation of the target intelligent chatbot can be determined based on the first rating of the target authentication service partner platform. Since the first rating of the target authentication service partner platform will upgrade or downgrade the target's preset rating based on the number of different misconduct occurrences, the target rating of the target intelligent chatbot can also include one of the following: a third level, a fourth level, a fifth level, and a sixth level, with levels decreasing sequentially. For ease of distinction, in this embodiment, the third level can be denoted as A, the fourth level as B, the fifth level as C, and the sixth level as D.

[0054] As an optional implementation of this application, when the first rating of the target authentication service partner platform is first level X, the target rating of the target intelligent chatbot is determined to be fourth level B; when the first rating of the target authentication service partner platform is second level Y, the target rating of the target intelligent chatbot is determined to be fifth level C.

[0055] Step S106: When the target intelligent chatbot is a second type of intelligent chatbot created before the first cycle, obtain the first bad behavior statistics of the target intelligent chatbot, and determine the target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first bad behavior statistics.

[0056] The first set of statistics on undesirable behaviors includes: the number of times the target intelligent chatbot exhibited the third undesirable behavior in the second period and the number of times the target intelligent chatbot exhibited the fourth undesirable behavior in the first time period. The second period is the period preceding the first period, and the first time period includes all periods before the second period and the period preceding the second period.

[0057] Generally, the rating of a chatbot is affected by factors such as the review results of its submissions, the number of complaints it receives, and the number of times it is blocked. Additionally, the presence of politically sensitive, gang-related, or pornographic content in the materials submitted by the chatbot increases its weight in the rating criteria. Therefore, the above behaviors can be summarized as follows: Inappropriate behavior by a target chatbot can include any of the following: adding a new menu but failing the review, submitting materials but failing the review, or receiving complaints against the chatbot.

[0058] For example, refresh the Chatbot misbehavior data table daily to obtain the number of third misbehaviors from yesterday, such as adding a new menu item to Chatbot but failing the review, submitting materials to Chatbot but failing the review, or receiving complaints against Chatbot. Also, obtain the number of fourth misbehaviors from today, such as adding a new menu item to Chatbot but failing the review, submitting materials to Chatbot but failing the review, or receiving complaints against Chatbot. Determine the first misbehavior statistics of Chatbot based on the number of third and fourth misbehaviors.

[0059] As another optional implementation of this application, since the Chatbot's misbehavior data table is refreshed daily, it is possible to decide whether to upgrade, maintain, or downgrade the existing rating of the target intelligent chatbot based on the existing rating and the first misbehavior statistics, thereby obtaining the final accurate target rating of the target intelligent chatbot.

[0060] This can be understood as follows: If the number of the third type of misconduct committed by the Chatbot yesterday exceeds the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day exceeds the preset range [p,k], it indicates that the Chatbot has a high number of misconducts, increasing the uncertainty of the quality of submitted materials, and the Chatbot's existing rating needs to be downgraded; if the number of the third type of misconduct committed by the Chatbot yesterday is within the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day is within the preset range [p,k], the Chatbot's existing rating remains unchanged; if the number of the third type of misconduct committed by the Chatbot yesterday is less than the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day is less than the preset range [p,k], it indicates that the Chatbot has a low number of misconducts, the quality of submitted materials is high, and the Chatbot's existing rating needs to be upgraded.

[0061] Specifically, Figure 3 A flowchart illustrating an optional method for determining the target rating of a smart chatbot is shown, such as... Figure 3 As shown, the target rating of the target intelligent chatbot can be determined through steps S1061-S1064:

[0062] Step S1061: If the existing rating is Level 3A, and the number of third-degree misbehaviors is greater than the fifth preset threshold and the number of fourth-degree misbehaviors is greater than the sixth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 4; otherwise, the target rating of the target intelligent chatbot is determined to be Level 3.

[0063] For example, if a Chatbot's current rating is Level 3A, it means the Chatbot is at the highest level. The decision to downgrade the Chatbot's existing Level 3A can only be made based on the number of third-level misconducts from yesterday and the number of fourth-level misconducts from previous days. If the number of third-level misconducts exceeds the fifth preset threshold n and the number of fourth-level misconducts exceeds the sixth preset threshold k, it indicates an increase in the Chatbot's misconduct, meaning the uncertainty of the quality of the submitted materials has increased. In this case, the Chatbot's existing rating needs to be downgraded, and its target rating should be set to Level 4B, which is lower than Level 3A. Otherwise, the Chatbot's existing Level 3A should be maintained.

[0064] Step S1062: If the existing rating is Level 4, and the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 5; if the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 3; otherwise, the target rating of the target intelligent chatbot is determined to be Level 4.

[0065] For example, if a Chatbot's existing rating is Level 4B, the system continues to determine whether to upgrade or downgrade the Chatbot based on the number of third-degree misconducts committed yesterday and the number of fourth-degree misconducts committed before today. If the number of third-degree misconducts exceeds the fifth preset threshold n and the number of fourth-degree misconducts exceeds the sixth preset threshold k, it indicates an increase in the number of misconducts committed by the Chatbot, meaning an increase in the uncertainty of the quality of the submitted materials. Therefore, the Chatbot's existing rating needs to be downgraded, and its target rating should be set to Level 5C, which is lower than Level 4B. If the number of third-degree misconducts is less than the seventh preset threshold n and the number of fourth-degree misconducts is less than the eighth preset threshold p, it indicates a decrease in the number of misconducts committed by the Chatbot, and the quality of the submitted materials needs to be improved. Therefore, the Chatbot's existing rating needs to be upgraded, and its target rating should be set to Level 3B, which is higher than Level 4B. Otherwise, the Chatbot's existing Level 4B should be maintained.

[0066] Step S1063: If the existing rating is Level 5, and the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 6; if the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 4; otherwise, the target rating of the target intelligent chatbot is determined to be Level 5.

[0067] For example, if a Chatbot's existing rating is Level 5, C, the system continues to determine whether to upgrade or downgrade the Chatbot based on the number of third-degree misconducts committed yesterday and the number of fourth-degree misconducts committed before today. If the number of third-degree misconducts exceeds the fifth preset threshold n and the number of fourth-degree misconducts exceeds the sixth preset threshold k, it indicates an increase in the number of misconducts committed by the Chatbot, meaning an increase in the uncertainty of the quality of the submitted materials. Therefore, the Chatbot's existing rating needs to be downgraded, and its target rating should be set to Level 6, D, which is lower than Level 5, C. If the number of third-degree misconducts is less than the seventh preset threshold n and the number of fourth-degree misconducts is less than the eighth preset threshold p, it indicates a decrease in the number of misconducts committed by the Chatbot, and the quality of the submitted materials needs to be improved. Therefore, the Chatbot's existing rating needs to be upgraded, and its target rating should be set to Level 4, B, which is higher than Level 5, C. Otherwise, the Chatbot's existing Level 5, C, should be maintained.

[0068] Step S1064: If the existing rating is Level 6, and the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 5; otherwise, the target rating of the target intelligent chatbot is determined to be Level 6.

[0069] For example, if a Chatbot's current rating is Level 6 (D), it means the Chatbot is at the lowest level. The decision to upgrade the Chatbot to Level 6 (D) can only be made based on the number of third-degree misconducts from yesterday and the number of fourth-degree misconducts from previous days. If the number of third-degree misconducts is greater than the seventh preset threshold m and the number of fourth-degree misconducts is less than the eighth preset threshold p, it means the Chatbot's misconduct count has decreased, the quality of submitted materials has improved, and the Chatbot's current rating needs to be upgraded. The target rating should be set to Level 5 (C), which is higher than Level 6 (D). Otherwise, the Chatbot remains at Level 6 (D).

[0070] Step S108: Determine the target review method for the target material based on the target rating, wherein the target material is the material submitted by the target intelligent chatbot.

[0071] Specifically, Figure 4 A flowchart illustrating an optional method for determining the target content for review is shown, such as... Figure 4 As shown, the target review method for the target material can be determined according to the rules in steps S1081-S1084:

[0072] Step S1081: When the target rating is Level 3A, determine that the target review method is manual review and give it priority.

[0073] In addition, since the materials submitted by the target intelligent chatbot in the third level A are of the highest quality, the target review method can also be single-person review.

[0074] Step S1082: When the target rating is Level 4B, determine the target review method as manual review.

[0075] Step S1083: When the target rating is Level 5 C, determine the target review method as a combination of manual review and artificial intelligence review.

[0076] Step S1084: When the target rating is Level 6D, determine that the target review method is parallel review by human review and artificial intelligence review, and generate alarm information to indicate low reliability of the material.

[0077] Additionally, if a target chatbot consistently receives a Level 6 D rating for a predetermined number of consecutive periods, the chatbot will be added to the blacklist. This prevents Level 6 D chatbots from providing extremely low-quality content, wasting significant review time for moderators and improving review efficiency.

[0078] According to an optional embodiment of this application, Figure 5 A flowchart illustrating another alternative material review method is shown, such as... Figure 5 As shown, the target review method for the target material can be determined through the following steps S1-S11:

[0079] Step S1: Periodically determine the type of Chatbot, and then execute steps S2 and S6;

[0080] Step S2: When Chatbot is the first type of intelligent chatbot created on the same day, determine the Csp platform to which Chatbot belongs, and refresh the bad behavior data table of the Csp platform on a daily schedule.

[0081] Step S3: Determine the first rating of the Csp platform based on the target preset rating and the second misconduct statistics, and then execute steps S4 and S5.

[0082] Step S4: When the first rating on the Csp platform is Level X, determine the target rating for Chatbot as Level 4.

[0083] Step S5: When the first rating on the Csp platform is Level 2 Y, determine the target rating for Chatbot as Level 5.

[0084] Step S6: When Chatbot is a second type of intelligent chatbot created before the first cycle, refresh the Chatbot's bad behavior data table daily to obtain the Chatbot's first bad behavior statistics.

[0085] Step S7: Determine the target rating of Chatbot based on Chatbot's existing ratings and the first set of bad behavior statistics, and then execute steps S8, S9, S10 and S11.

[0086] Step S8: When the target rating is Level 3, determine that the target review method is manual review and give it priority.

[0087] Step S9: When the target rating is Level 4, determine the target review method as manual review;

[0088] Step S10: When the target rating is Level 5, determine the target review method as a combination of manual review and artificial intelligence review.

[0089] Step S11: When the target rating is Level 6, determine that the target review method is parallel review by human review and artificial intelligence review, and generate alarm information to indicate low reliability of the material.

[0090] In this embodiment, the type of the target intelligent chatbot is periodically determined. When the target intelligent chatbot is a first-type intelligent chatbot created within the first period, the target rating of the target intelligent chatbot is determined by the first rating of the target certification service partner platform to which the target intelligent chatbot belongs. Simultaneously, the existing ratings of the target intelligent chatbot in the past and the first set of statistical data on the target intelligent chatbot's misconduct are combined to upgrade or downgrade the existing rating, thereby continuously updating the target rating and achieving system control over the intelligent chatbot. Based on the target rating, the target review method for target materials is determined, allowing reviewers to review submitted materials sequentially according to the target rating, increasing the review efficiency for target intelligent chatbots uploading high-quality materials. At the same time, warnings or blacklisting are performed on low-level target intelligent chatbots, thus solving the technical problem of low review efficiency for materials submitted by intelligent chatbots in related technologies.

[0091] Example 2

[0092] According to an embodiment of this application, a material review apparatus for implementing the material review method in Embodiment 1 is also provided, such as... Figure 6 As shown, the material review device includes at least a first determining module 61, a second determining module 62, a third determining module 63, and a fourth determining module 64, wherein:

[0093] The first determining module 61 is used to periodically determine the type of the target intelligent chatbot.

[0094] Given the large volume of materials uploaded to the 5G messaging operation platform, a significant amount of material submitted by intelligent chatbots needs to be reviewed daily by auditors. To improve the efficiency of message material review, the first determination module 61 periodically determines the type of the target intelligent chatbot, thereby enabling system control over the target intelligent chatbot.

[0095] In this embodiment of the application, the first determining module 61 can periodically determine the type of intelligent chatbot, for example, by updating the type of intelligent chatbot every day, so that the reviewers can review the materials provided by the intelligent chatbot in a targeted manner according to the type of intelligent chatbot.

[0096] Specifically, high-quality materials submitted by high-level intelligent chatbots can be prioritized for review and the review process can be simplified; while for low-quality materials submitted by low-level intelligent chatbots, the review process will be appropriately lengthened to avoid mis-review of materials.

[0097] The second determining module 62 is used to determine the target certification service partner platform to which the target intelligent chatbot belongs when the target intelligent chatbot is a first type of intelligent chatbot created in the first period, and to determine the first rating of the target certification service partner platform, and to determine the target rating of the target intelligent chatbot based on the first rating.

[0098] Specifically, when Chatbot creates the first type of intelligent chatbot for the user on the 5G messaging operation platform on the same day, the second determining module 62 can determine the first rating of the Csp to which Chatbot belongs, and determine the target rating of Chatbot through the first rating of the Csp.

[0099] As an optional implementation of this application, the first rating of Csp can be determined in the following manner:

[0100] First, the target preset rating of the target certification service partner platform is determined from the preset platform rating table, which includes preset ratings of multiple certification service partner platforms.

[0101] Then, obtain the second set of misconduct statistics of the target certification service partner platform. The second set of misconduct statistics includes the number of first misconducts of the target certification service partner platform in the second period and the number of second misconducts of the target certification service partner platform in the first time period. The second period is the period before the first period, and the first time period includes the second period and all periods before the second period.

[0102] Among them, the misconduct of the Target Authentication Service Partner Platform includes one of the following: adding new users under the Target Authentication Service Partner Platform but failing the review, changing user ownership but failing the review, adding new intelligent chatbots but failing the review, or changing the ownership of intelligent chatbots but failing the review.

[0103] Finally, the first rating of the target certification service partner platform is determined based on the target preset rating, the number of first-time misconducts, and the number of second-time misconducts.

[0104] Optionally, when the target preset rating is Level 1, if the number of first misconducts is greater than the first preset threshold and the number of second misconducts is greater than the second preset threshold, the target certification service partner platform's first rating is determined to be Level 2; otherwise, the target certification service partner platform's first rating is determined to be Level 1. When the target preset rating is Level 2, if the number of first misconducts is less than the third preset threshold and the number of second misconducts is less than the fourth preset threshold, the target certification service partner platform's first rating is determined to be Level 1; otherwise, the target certification service partner platform's first rating is determined to be Level 2.

[0105] After determining the first rating of the target certification service partner platform, the target rating for the next creation of the target intelligent chatbot can be determined based on the first rating of the target certification service partner platform. Since the first rating of the target certification service partner platform will upgrade or downgrade the target's preset rating based on the number of different misconducts, the target rating of the target intelligent chatbot can also include one of the following: the third level, the fourth level, the fifth level, and the sixth level, which are successively reduced in level.

[0106] As an optional implementation of this application, when the first rating of the target authentication service partner platform is Level 1, the target rating of the target intelligent chatbot is determined to be Level 4; when the first rating of the target authentication service partner platform is Level 2, the target rating of the target intelligent chatbot is determined to be Level 5.

[0107] The third determining module 63 is used to obtain the first bad behavior statistics of the target intelligent chatbot when the target intelligent chatbot is a second type of intelligent chatbot created before the first cycle, and to determine the target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first bad behavior statistics.

[0108] The first set of statistics on undesirable behaviors includes: the number of times the target intelligent chatbot exhibited the third undesirable behavior in the second period and the number of times the target intelligent chatbot exhibited the fourth undesirable behavior in the first time period. The second period is the period preceding the first period, and the first time period includes all periods before the second period and the period preceding the second period.

[0109] Generally, the rating of a chatbot is affected by factors such as the review results of its submissions, the number of complaints it receives, and the number of times it is blocked. Additionally, the presence of politically sensitive, gang-related, or pornographic content in the materials submitted by the chatbot increases its weight in the rating criteria. Therefore, the above behaviors can be summarized as follows: Inappropriate behavior by a target chatbot can include any of the following: adding a new menu but failing the review, submitting materials but failing the review, or receiving complaints against the chatbot.

[0110] As another optional implementation of this application, since the Chatbot's misbehavior data table is refreshed daily, the third determining module 63 can decide to upgrade, maintain, or downgrade the target intelligent chatbot's existing rating based on the Chatbot's existing rating and the first misbehavior statistics, thereby obtaining the final accurate target rating of the target intelligent chatbot.

[0111] This can be understood as follows: If the number of the third type of misconduct committed by the Chatbot yesterday exceeds the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day exceeds the preset range [p,k], it indicates that the Chatbot has a high number of misconducts, increasing the uncertainty of the quality of submitted materials, and the Chatbot's existing rating needs to be downgraded; if the number of the third type of misconduct committed by the Chatbot yesterday is within the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day is within the preset range [p,k], the Chatbot's existing rating remains unchanged; if the number of the third type of misconduct committed by the Chatbot yesterday is less than the preset range [m,n], and the number of the fourth type of misconduct committed by the Chatbot before the current day is less than the preset range [p,k], it indicates that the Chatbot has a low number of misconducts, the quality of submitted materials is high, and the Chatbot's existing rating needs to be upgraded.

[0112] Specifically, the third determining module 63 can determine the target rating of the target intelligent chatbot using the following rules:

[0113] If the existing rating is Level 3A, and the number of third-degree misbehaviors is greater than the fifth preset threshold and the number of fourth-degree misbehaviors is greater than the sixth preset threshold, then the target rating of the target intelligent chatbot is determined to be Level 4; otherwise, the target rating of the target intelligent chatbot is determined to be Level 3.

[0114] If the existing rating is Level 4, and the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 5; if the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 3; otherwise, the target rating of the target intelligent chatbot is determined to be Level 4.

[0115] If the existing rating is Level 5, and the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 6; if the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 4; otherwise, the target rating of the target intelligent chatbot is determined to be Level 5.

[0116] If the existing rating is Level 6, and the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold, the target rating of the target intelligent chatbot is determined to be Level 5; otherwise, the target rating of the target intelligent chatbot is determined to be Level 6.

[0117] The fourth determination module 64 is used to determine the target review method for target materials based on the target rating, wherein the target materials are the materials submitted by the target intelligent chatbot.

[0118] As an optional implementation of this application, the fourth determining module 64 can determine the target review method of the target material according to the following rules:

[0119] When the target rating is Level 3, the target review method is determined to be manual review and given priority; when the target rating is Level 4, the target review method is determined to be manual review; when the target rating is Level 5, the target review method is determined to be a combination of manual review and AI review; when the target rating is Level 6, the target review method is determined to be a combination of manual review and AI review, and an alarm message is generated to indicate low reliability of the material.

[0120] Additionally, if a target chatbot consistently achieves a Level 6 rating for a predetermined number of consecutive periods, the target chatbot will be added to the blacklist. This prevents Level 6 chatbots from providing extremely low-quality content, wasting significant review time for moderators, and improves review efficiency.

[0121] It should be noted that each module in the material review device in this application embodiment corresponds one-to-one with each implementation step of the material review method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0122] Example 3

[0123] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored program, wherein the device where the non-volatile storage medium is located executes the material review method in Embodiment 1 by running the program.

[0124] Specifically, the device containing the non-volatile storage medium executes the following steps by running the program: periodically determining the type of the target intelligent chatbot; when the target intelligent chatbot is a first-type intelligent chatbot created within the first period, determining the target certification service partner platform to which the target intelligent chatbot belongs, and determining the first rating of the target certification service partner platform, and determining the target rating of the target intelligent chatbot based on the first rating; when the target intelligent chatbot is a second-type intelligent chatbot created before the first period, obtaining the first misconduct statistics of the target intelligent chatbot, and determining the target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first misconduct statistics; determining the target review method for target materials based on the target rating, wherein the target materials are the materials submitted by the target intelligent chatbot.

[0125] According to an embodiment of this application, a processor is also provided for running a program, wherein the program executes the material review method in embodiment 1 during runtime.

[0126] Specifically, the program executes the following steps during runtime: periodically determining the type of the target intelligent chatbot; when the target intelligent chatbot is a first-type intelligent chatbot created within the first period, determining the target certification service partner platform to which the target intelligent chatbot belongs, and determining the first rating of the target certification service partner platform, and determining the target rating of the target intelligent chatbot based on the first rating; when the target intelligent chatbot is a second-type intelligent chatbot created before the first period, obtaining the first misconduct statistics of the target intelligent chatbot, and determining the target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first misconduct statistics; determining the target review method for target materials based on the target rating, wherein the target materials are the materials submitted by the target intelligent chatbot.

[0127] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the material review method of Embodiment 1 through the computer program.

[0128] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0129] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0134] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for reviewing source materials, characterized in that, include: Periodically determine the type of target intelligent chatbot; When the target intelligent chatbot is a first-type intelligent chatbot created within the first period, the target certification service partner platform to which the target intelligent chatbot belongs is determined, and the first rating of the target certification service partner platform is determined. Based on the first rating, the target rating of the target intelligent chatbot is determined. When the target intelligent chatbot is a second type of intelligent chatbot created before the first cycle, the first bad behavior statistics of the target intelligent chatbot are obtained, and the target rating of the target intelligent chatbot is determined based on the existing rating of the target intelligent chatbot and the first bad behavior statistics. The target review method for the target material is determined based on the target rating, wherein the target material is the material submitted by the target intelligent chatbot; The determination of the first rating of the target certification service partner platform includes: determining the target preset rating of the target certification service partner platform from a preset platform rating table, wherein the platform rating table includes preset ratings of multiple certification service partner platforms; obtaining second misconduct statistics of the target certification service partner platform, wherein the second misconduct statistics include the number of first misconducts of the target certification service partner platform in a second period and the number of second misconducts of the target certification service partner platform in a first time period, the second period being the period preceding the first period, and the first time period including the second period and all periods preceding the second period; and determining the first rating of the target certification service partner platform based on the target preset rating, the number of first misconducts, and the number of second misconducts.

2. The method according to claim 1, characterized in that, The target is preset to be either Level 1 or Level 2, with Level 1 being higher than Level 2. The first rating of the target certification service partner platform is determined based on the target preset rating, the number of the first misconduct acts, and the number of the second misconduct acts, including: When the target preset rating is the first level, if the number of the first misconducts is greater than the first preset threshold and the number of the second misconducts is greater than the second preset threshold, the first rating of the target certification service partner platform is determined to be the second level; otherwise, the first rating of the target certification service partner platform is determined to be the first level. When the target preset rating is the second level, if the number of the first misconducts is less than the third preset threshold and the number of the second misconducts is less than the fourth preset threshold, the first rating of the target certification service partner platform is determined to be the first level; otherwise, the first rating of the target certification service partner platform is determined to be the second level.

3. The method according to claim 2, characterized in that, The target rating includes one of the following: the third, fourth, fifth and sixth levels, which are in descending order of level; Determining the target rating of the target intelligent chatbot based on the first rating includes: When the first rating on the target certification service partner platform is the first level, the target rating of the target intelligent chatbot is determined to be the fourth level; When the first rating on the target certification service partner platform is the second level, the target rating of the target intelligent chatbot is determined to be the fifth level.

4. The method according to claim 1, characterized in that, The misconduct of the target authentication service partner platform includes one of the following: adding a new user to the target authentication service partner platform but failing the review, changing the user's ownership but failing the review, adding a new intelligent chatbot but failing the review, or changing the ownership of an intelligent chatbot but failing the review.

5. The method according to claim 1, characterized in that, The first set of statistics on undesirable behaviors includes: the number of times the target intelligent chatbot exhibits the third undesirable behavior in the second period and the number of times the target intelligent chatbot exhibits the fourth undesirable behavior in the first time period. The second period is the period preceding the first period, and the first time period includes the second period and all periods preceding the second period. The existing ratings for the target intelligent chatbot include one of the following: Level 3, Level 4, Level 5, and Level 6, which are ranked in descending order.

6. The method according to claim 5, characterized in that, The target rating includes one of the following: the third level, the fourth level, the fifth level, and the sixth level; Determining the target rating of the target intelligent chatbot based on its existing rating and the first set of misbehavior statistics includes: When the existing rating is the third level, if the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be the fourth level; otherwise, the target rating of the target intelligent chatbot is determined to be the third level. When the existing rating is the fourth level, if the number of third misconduct behaviors is greater than the fifth preset threshold and the number of fourth misconduct behaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be the fifth level; if the number of third misconduct behaviors is less than the seventh preset threshold and the number of fourth misconduct behaviors is less than the eighth preset threshold, the target rating of the target intelligent chatbot is determined to be the third level; otherwise, the target rating of the target intelligent chatbot is determined to be the fourth level. When the existing rating is the fifth level, if the number of third misbehaviors is greater than the fifth preset threshold and the number of fourth misbehaviors is greater than the sixth preset threshold, the target rating of the target intelligent chatbot is determined to be the sixth level. If the number of the third misbehavior is less than the seventh preset threshold and the number of the fourth misbehavior is less than the eighth preset threshold, the target rating of the target intelligent chatbot is determined to be the fourth level; otherwise, the target rating of the target intelligent chatbot is determined to be the fifth level. If the number of third misbehaviors is less than the seventh preset threshold and the number of fourth misbehaviors is less than the eighth preset threshold when the existing rating is the sixth level, then the target rating of the target intelligent chatbot is determined to be the fifth level; otherwise, the target rating of the target intelligent chatbot is determined to be the sixth level.

7. The method according to claim 5, characterized in that, The target intelligent chatbot's misconduct includes one of the following: the target intelligent chatbot adds a new menu but fails to pass the review; the target intelligent chatbot submits materials but fails to pass the review; or a complaint is received against the target intelligent chatbot.

8. The method according to claim 1, characterized in that, The target rating includes one of the following: a descending order of level three, level four, level five, and level six. Based on the target rating, the target review method for the target materials is determined, including: When the target rating is the third level, the target review method is determined to be manual review and given priority. When the target rating is the fourth level, the target review method is determined to be manual review; When the target rating is the fifth level, the target review method is determined to be a combination of manual review and artificial intelligence review. When the target rating is the sixth level, the target review method is determined to be a combination of manual review and artificial intelligence review, and an alarm message is generated to indicate that the material has low reliability.

9. The method according to claim 8, characterized in that, The method further includes: If the target intelligent chatbot's rating is consistently at level six for a predetermined number of consecutive periods, the target intelligent chatbot will be added to the blacklist.

10. A material review device, characterized in that, include: The first determining module is used to periodically determine the type of the target intelligent chatbot; The second determining module is configured to, when the target intelligent chatbot is a first-type intelligent chatbot created within a first period, determine the target authentication service partner platform to which the target intelligent chatbot belongs, and determine the first rating of the target authentication service partner platform, and determine the target rating of the target intelligent chatbot based on the first rating. The method for determining the first rating of the target authentication service partner platform includes: determining the target preset rating of the target authentication service partner platform from a preset platform rating table, wherein the platform rating table includes preset ratings of multiple authentication service partner platforms; obtaining second misconduct statistics of the target authentication service partner platform, wherein the second misconduct statistics include the number of first misconducts of the target authentication service partner platform within a second period and the number of second misconducts of the target authentication service partner platform within a first time period, the second period being the period preceding the first period, and the first time period including the second period and all periods preceding the second period; and determining the first rating of the target authentication service partner platform based on the target preset rating, the number of first misconducts, and the number of second misconducts. The third determining module is used to obtain the first bad behavior statistics of the target intelligent chatbot when the target intelligent chatbot is a second type of intelligent chatbot created before the first period, and determine the target rating of the target intelligent chatbot based on the existing rating of the target intelligent chatbot and the first bad behavior statistics. The fourth determining module is used to determine the target review method for the target material based on the target rating, wherein the target material is the material submitted by the target intelligent chatbot.

11. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein the device containing the non-volatile storage medium executes the material review method according to any one of claims 1 to 9 by running the program.

12. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the material review method according to any one of claims 1 to 9 through the computer program.

Images