Reporting processing method and device of violation data, electronic equipment and storage medium

Abstract

The invention discloses a violation data report processing method and device, electronic equipment and a storage medium, and the method comprises the steps: employing report contents and user data of a reported user for auditing in a machine auditing model and manual auditing, and the report contents comprise report types, reasons, description contents and other data; the user data comprises multi-dimensional data such as historical violation records, reported times, risk control blacklist states, context contents, behavior data and social relation data of reported users, on one hand, violation auditing on the reported target business data is realized through the multi-dimensional data, the target business data can be audited in all directions, and the user experience is improved. On the other hand, the report content needing manual auditing can be screened out and distributed to different manual auditing queues, the number of manual auditing can be reduced, the labor cost can be reduced, data reported in the different manual auditing queues can be audited by a specially-assigned person, and the accuracy and efficiency of manual auditing are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a method and device for reporting processing of illegal data, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of Internet technology, social software has become an indispensable communication tool in people's daily life. Users can share information, express opinions and interact on social platforms through various forms such as text, pictures, audio and video on social software.

[0003] In order to ensure the legality and compliance of the content on the Internet, the data content generated on the Internet needs to be audited. In addition, a reporting channel is provided for users to report and audit illegal data. In the prior art, the subject and content of user reports need to be audited by a large number of manpower, which is costly and difficult for auditors to accurately audit a large number of reports in a short time. The response and processing efficiency of manual auditing is low. On the other hand, relying only on the number of reports and data content to audit reports cannot fully evaluate the reports, resulting in incorrect and missed penalties for illegal subjects. SUMMARY

[0004] The present application provides a method and device for reporting processing of illegal data, electronic equipment and storage medium, which can fully audit the content of the report through multi-dimensional data, improve the accuracy of the report content audit, and filter part of the report content that needs manual audit after machine audit. The report content is classified and distributed to different manual audit queues, which can reduce the number of manual audit of the report content and improve the accuracy and efficiency of manual audit of the report content.

[0005] In a first aspect, the present application provides a method for reporting processing of illegal data, comprising:

[0006] receiving a report content of target business data of a reported user, obtaining user data of the reported user, the report content at least including a reported user identifier, a report type, a report reason and a description content of the target business data;

[0007] inputting the report content and the user data into a machine audit model to obtain a score of the target business data belonging to illegal data;

[0008] when the score is greater than or equal to a preset score threshold, determining a target manual audit queue of the target business data based on the report content and the user data, the target manual audit queue being a queue selected from a plurality of queues;

[0009] distributing the report content and the user data into the target artificial auditing queue for artificial auditing, to obtain an artificial auditing result of the target business data.

[0010] Optionally, the user data of the reported user is obtained, including:

[0011] The at least one data of the reported user is obtained as the user data, including:

[0012] account information, historical violation records, reported times, risk control blacklist status, context data of the target business data, historical behavior data of the reported user, and social relationship data.

[0013] Optionally, after the score that the target business data belongs to violation data is obtained by inputting the report content and the user data into a machine auditing model, the method further includes:

[0014] determining whether the score is greater than or equal to a preset first score threshold;

[0015] if yes, determining the target business data as violation data;

[0016] if no, when the score is greater than or equal to a preset second score threshold, determining the target business data as data to be artificially audited, and performing the step of determining a target artificial auditing queue of the target business data based on the report content and the user data, wherein the second score threshold is less than the first score threshold;

[0017] when the score is less than the second score threshold, determining whether the target business data is violation data based on a preset filtering rule;

[0018] if yes, performing the step of determining a target artificial auditing queue of the target business data based on the report content and the user data;

[0019] if no, not processing the target business data.

[0020] Optionally, after determining that the target business data is data to be artificially audited, the method further includes:

[0021] generating a primary auditing result that the target business data is initially determined as violation data and is in an artificial auditing state;

[0022] feeding back the primary auditing result to the reported user and the reported user.

[0023] Optionally, determining a target artificial auditing queue of the target business data based on the report content and the user data includes:

[0024] inputting the report content and the user data into a preset target business data grading model to obtain a manual review queue to which the target business data belongs.

[0025] Optionally, after the report content and the user data are assigned to the target manual review queue for manual review, the method further comprises:

[0026] receiving a manual review result of the target business data;

[0027] feeding back the manual review result to the report user and the reported user.

[0028] Optionally, the method further comprises:

[0029] receiving feedback data of the review result of the target business data;

[0030] retraining the machine review model and calibrating a review scale of the manual review based on the feedback data.

[0031] In a second aspect, the present application provides a device for processing a report of violation data, comprising:

[0032] a report content and user data acquisition module configured to receive a report content of target business data of a reported user, and acquire user data of the reported user, wherein the report content at least includes a reported user identifier, a report type, a report reason and a report description content;

[0033] a machine review module configured to input the report content and the user data into a machine review model to obtain a score of the target business data belonging to violation data;

[0034] a manual review queue determination module configured to, when the score is greater than or equal to a preset score threshold, determine a target manual review queue of the target business data based on the report content and the user data, wherein the target manual review queue is a queue selected from a plurality of queues;

[0035] a manual review module configured to assign the report content and the user data to the target manual review queue for manual review to obtain a manual review result of the target business data.

[0036] In a third aspect, the present application provides an electronic device, comprising:

[0037] at least one processor; and

[0038] a memory in communication with the at least one processor; wherein

[0039] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the violation data reporting processing method according to any one of the first aspect of the present application.

[0040] In a fourth aspect, the present application provides a computer readable storage medium storing computer instructions for enabling a processor to implement the violation data reporting processing method according to any one of the first aspect of the present application when executed by the processor.

[0041] In the present application, the reporting content and the user data of the reported user are used for auditing in machine auditing model and manual auditing, the reporting content can include reporting type, reporting reason, description of target business data and the like, and the user data can also include historical violation records of the reported user, reporting times, risk control blacklist status, context content of target business data, historical behavior data and social relationship data and the like, on the one hand, the violation auditing of the target business data is realized through multi-dimensional data, and the target business data can be comprehensively audited in machine auditing and manual auditing, thereby improving the accuracy of violation auditing, on the other hand, the reports needing manual auditing can be screened out and distributed to different manual auditing queues, thereby reducing the number of manual auditing reports, reducing labor cost, and improving the accuracy and efficiency of manual auditing through auditing of the reported data in different manual auditing queues by different persons.

[0042] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0044] Figure 1 is a flowchart of a violation data reporting processing method provided by the first embodiment of the present application;

[0045] Figure 2 is a flowchart of a violation data reporting processing method provided by the second embodiment of the present application;

[0046] Figure 3 is a schematic diagram of filling reporting content on a reporting page;

[0047] Figures 4-7 is a flowchart of an example of a report processing of irregular data;

[0048] Figure 8 is a structural schematic diagram of a report processing device of irregular data provided by the third embodiment of the present application;

[0049] Figure 9 is a structural schematic diagram of an electronic device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should belong to the protection scope of the present application.

[0051] Embodiment One

[0052] Figure 1 A flowchart of a report processing method of irregular data provided by the first embodiment of the present application is shown in the figure. The present embodiment can be applied to audit whether the reported business data is irregular. The method can be executed by a report processing device of irregular data. The report processing device of irregular data can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the report processing method of irregular data includes the following steps. Figure 1

[0053] S101, receiving a report content of target business data of a reported user, and obtaining user data of the reported user.

[0054] The target business data can be data generated by the user when performing business operations on each network platform. For example, taking a live platform as an example, the business data can be instant chat messages in a live room, public screen messages in a live room, user nicknames, user avatars, user behaviors and related content, etc. Taking a short video platform as an example, the target business data can be videos, pictures published by the user, and comments on videos and pictures, etc.

[0055] ​The reported user can be a user whose business data is reported as illegal data. The reporting user can report the business data on the network platform through a reporting function module on the network platform. When reporting, the reporting user selects the target business data to be reported, the reporting type, the reporting reason, and the description content of the target business data, and can also submit pictures and other evidence. After the reporting user submits the report, the reporting content including at least the reported user identifier, the reporting type, the reporting reason, the description content of the target business data, and the evidence is generated.

[0056] When the reporting content is subscribed by the server, the reporting content is transmitted in the form of a message queue. The server can receive the reporting content, and can obtain the user data of the reported user according to the reported user identifier in the reporting content. For example, the user data can include at least one of the account information, the historical illegal record, the number of reported times, the risk control blacklist status, the context data of the target business data, the historical behavior data, and the social relationship data of the reported user.

[0057] S102, input the reporting content and the user data into the machine review model to obtain a score of the target business data belonging to illegal data.

[0058] The machine review model can be a neural network model pre-trained to review whether the target business data belongs to illegal data in combination with the reporting content and the user data. The training method of the neural network model can refer to the training method of the neural network model in the prior art, which will not be described in detail here.

[0059] The reporting content (which can include the target business data) and the user data are input into the machine review model to obtain a score of the target business data belonging to illegal data. The greater the score, the greater the possibility that the target business data belongs to illegal data.

[0060] S103, when the score is greater than or equal to a preset score threshold, determining a target artificial review queue of the target business data based on the reporting content and the user data.

[0061] The score threshold can be a score at which the machine review model predicts that the target business data can belong to the violation data but needs manual review. When the score is greater than or equal to the preset score threshold, it can be determined that the reported target business data needs manual review, and the report content and user data can be sent to a manual review queue for manual review. In this embodiment, multiple manual review queues can be set. Different report types, report reasons, and reported users correspond to different manual review queues. The target manual review queue can be determined through the report content and user data. For example, the manual review queue of the reported target business data is determined according to the report type, or the manual review queue of the reported target business data is determined according to the report reason, or the manual review queue of the reported target business data is determined according to the violation record, the number of penalties, the blacklist state, and the like of the reported user. Those skilled in the art can set multiple manual review queues according to the business needs of different scenes, and this embodiment does not limit this.

[0062] S104, the report content and the user data are allocated to the target manual review queue for manual review, and a manual review result of the target business data is obtained.

[0063] In the target manual review queue, the manual review is performed by reading the task from the queue. The manual review personnel obtains the manual review result of the target business data after the review. The manual review result includes whether the target business data is violation data and the corresponding penalty result.

[0064] The present application uses the report content and the user data of the reported user to perform review during machine review and manual review. The report content can include report type, report reason, description of target business data, and the like. The user data can also include historical violation record of the reported user, number of reported times, risk control blacklist state, context content of the target business data, historical behavior data, and social relationship data, and the like. On the one hand, the violation review of the target business data is realized through multi-dimensional data. The target business data can be reviewed in all directions during machine review and manual review, and the accuracy of the violation review is improved. On the other hand, the reports that need manual review can be screened and allocated to different manual review queues. The number of manual review reports can be reduced, the labor cost can be reduced, different data reported in different manual review queues can be reviewed by different persons, and the accuracy and efficiency of manual review can be improved.

[0065] Embodiment two

[0066] Figure 2 A flowchart of a violation data reporting processing method provided by the present application embodiment two is shown in FIG. 8. The violation data reporting processing method includes the following steps. Figure 2 as shown in the figure.

[0067] S201, receiving a report content of target service data of a reported user, and obtaining user data of the reported user.

[0068] As shown in Figure 3 Fig. 1 shows a schematic diagram of a report page. When reporting the target service data, the reporter can select the report type, the report reason, and describe the target service data in the report page. For example, the reporter can input the content of the reported target service data or the description of the content of the target service data, and can add pictures, videos, and other evidence. After submitting the report, the report content including the report type, the report reason, the description of the target service data, and the evidence is generated.

[0069] When the report content is subscribed by the server, the report content is transmitted through the message queue. The server can receive the report content, and can obtain the user data of the reported user according to the reported user identifier in the report content. For example, the user data can include at least one of the account information of the reported user, the historical violation record, the number of reports, the risk control blacklist status, the context data of the target service data, the historical behavior data of the reported user, and the social relationship data. Taking the chat application as an example, the report content can include the identity information (such as the user ID) of the reporter, the report type, the violation reason, the IM chat text and picture content, and the user data can include the attribute data (such as the user ID, the historical violation record, the number of violations, the number of penalties, etc.) of the reported person, the risk control blacklist status, etc.

[0070] S202, inputting the report content and the user data into a machine review model to obtain a score of the target service data belonging to the violation data.

[0071] The machine review model can be a neural network model pre-trained to review whether the target service data belongs to the violation data in combination with the report content and the user data. The training method of the neural network model can refer to the training method of the neural network model in the prior art, which will not be described in detail here.

[0072] The report content (which can include the target service data) and the user data are input into the machine review model to obtain a score of the target service data belonging to the violation data. The larger the score is, the more likely the target service data belongs to the violation data.

[0073] S203, determining whether the score is greater than or equal to a preset first score threshold.

[0074] The first score threshold can be a score capable of determining that the target business data is illegal data. For example, the first score threshold can be 0.9 (total score is 1), when the score is greater than or equal to the first score threshold, S204 can be executed, and when the score is less than the first score threshold, S205 can be executed.

[0075] S204, determining that the target business data is illegal data, generating an audit result that the target business data is illegal data, and feeding back to the reporting user and the reported user.

[0076] When the score is greater than or equal to the first score threshold, it can be determined that there is obvious illegal content in the target business data, the target business data can be determined as illegal data, and corresponding punishment (such as warning, speech ban, account ban, etc.) can be executed, and an audit result that the target business data is illegal data can be generated and fed back to the reporting user and the reported user, so that the reporting user can know the progress and result of the report, and the reported user is prompted to have illegal behavior, without manual auditing of the reported target business data, reducing the number of manual auditing of the reported business data, saving labor cost, and quickly responding to the report feedback audit result.

[0077] S205, determining whether the score is greater than or equal to a preset second score threshold.

[0078] In this embodiment, the second score threshold is less than the first score threshold. For example, the first score threshold is 0.9, and the second score threshold can be 0.6. When the score is greater than or equal to the second score threshold and less than the first score threshold, it means that the machine audit model cannot determine whether the target business data is illegal data, and manual further auditing is needed, S208 can be executed. If the score is less than the second score threshold, the machine audit model preliminarily judges that the target business data is not illegal data, S206-S207 can be executed.

[0079] S206, determining whether the target business data is illegal data based on a preset filtering rule.

[0080] In this embodiment, the preset filtering rule can be a filtering rule irrelevant to the content of the target business data and based on the attribute data of the reported user. For example, the filtering rule can be that the number of illegal times of the user in a historical time period reaches a preset number, the user belongs to a user in a blacklist, the highest level of punishment of the user reaches a preset level, etc. Those skilled in the art can set corresponding filtering rules according to different business scenarios, and the filtering rule is not limited in this embodiment.

[0081] If the score is less than the second score threshold, the reporting content and the user data of the reported user can be filtered by the filtering rule to determine whether the target business data is illegal data. If yes, S208 is executed, and if no, S207 is executed.

[0082] S207, not processing the target service data.

[0083] If it is determined through the filtering rule that the target service data is not illegal data, the report of the target service data for the reported user is ignored, the target service data is not processed, and the result is fed back to the reported user and the reported user. By filtering the reports that are not illegal through the filtering rule, invalid reports are filtered out, the number of reports that need to be manually reviewed is reduced, the intensity of manual review is reduced, human cost is saved, and the efficiency of illegal review is improved.

[0084] S208, determining that the target service data is data to be manually reviewed.

[0085] When the score is greater than or equal to the second score threshold and less than the first score threshold, it means that the machine review model cannot determine whether the target service data belongs to illegal data, and further manual review is needed. The target service data is data to be manually reviewed.

[0086] In an optional embodiment, after determining that the target service data is data to be manually reviewed, a preliminary review result of the target service data being preliminarily judged as illegal data and being in a manual review state can be generated, and the preliminary review result is fed back to the reported user and the reported user. The reported user and the reported user can obtain the current progress and preliminary result of the report.

[0087] S209, determining a target manual review queue of the target service data based on the report content and the user data.

[0088] In the manual review stage, multiple review queues can be divided, for example, a high-risk credit report queue, a normal report queue, a high-risk report queue, an audio high-risk work order queue, and other report queues can be divided. Of course, the queues can also be divided according to the report type, the report reason, etc.

[0089] In an embodiment, the target manual review queue of the target service data can be determined according to the report type, the report reason in the report content, the number of illegal times of the reported user in the user data, the blacklist state, etc.

[0090] In another embodiment, the report content grading model can also be pre-trained, the report content and the user data are input into the preset report content grading model, and the manual review queue to which the target service data belongs is obtained.

[0091] S210, assigning the report content and the user data to the target manual review queue for manual review to obtain a manual review result of the target service data.

[0092] After the report is divided into the target artificial review queue, the reviewer can sign in to review the target business data of the report according to the application scenario, the business type and the queue type, read the task from the queue for artificial review, feed back the artificial review result after the review, obtain the artificial review result of the target business data, and the artificial review result includes whether the target business data is illegal data and the corresponding punishment result.

[0093] In an optional embodiment, after the artificial review result of the target business data is received, the artificial review result can be fed back to the reporting user and the reported user, and feedback data of the user for the review result of the target business data can be received, the machine review model is retrained based on the feedback data, and the review scale of the artificial review is calibrated.

[0094] As shown in Figures 4-7 is a flowchart of an example of report review, as shown in Figure 4 The reporting user enters the report page through the report entry, fills in the report classification, reason, description, evidence, text content, chat content, business data and other content, and submits the report. The report message is sent to Kafka (a distributed stream processing platform), the reporting user is prompted that the report submission is successful, the report service consumes the report message, and the context of the report, the profile of the reported user, the picture data, the behavior data and the like are input into the machine review model. Whether it is illegal is judged by the machine review model.

[0095] As shown in Figure 5 When the machine review model judges that it is illegal, the report is assigned to human review. When the machine review model judges that it is not illegal, the report is filtered by rules. If it is determined to be illegal after filtering, it is assigned to human review. If it is determined to be not illegal after filtering, the report is ignored and the result is fed back to the reporting user. The reporting user evaluates the report processing result and stores it in the database.

[0096] As shown in Figure 6 When assigned to human review, the report is assigned to each queue for artificial review through the report scene, the information of the reporter / reportee, the report type, the report frequency / punishment frequency and the like. After artificial review, disposal is performed.

[0097] As shown in Figure 7 After artificial review, if the rule is not illegal, the report is ignored. If the rule is illegal, the content is cleared and the reported user is punished, and the result is fed back to the reporting user and the reported user. The data of the user evaluation of the report processing is stored in the database.

[0098] The embodiment has the following advantages:

[0099] (1) According to different reporting scenarios, multiple reporting categories and multiple reporting reasons are divided, which helps the reporting user to provide accurate reporting content, so that the audit can be more accurate for different reporting scenarios.

[0100] (2) Based on the multi-dimensional data in the reporting content and the user data of the reported user, the machine audit can filter invalid reports, and can automatically punish the obvious violation reports, reduce the number of manual audit reports, effectively save the labor cost, and improve the response speed and efficiency of the report audit.

[0101] (3) Based on the multi-dimensional data in the reporting content and the user data of the reported user, the report can be divided into different manual audit queues, and different reports are audited in different reporting audit queues to achieve the purpose of special person and special audit, and improve the accuracy and efficiency of manual report audit.

[0102] (3) After the audit, the evaluation data of the user on the report processing result is collected to optimize the machine audit model, and the scale of manual audit is calibrated to achieve the effect of analysis-closed loop.

[0103] Embodiment three

[0104] Figure 8 The structure schematic diagram of a violation data reporting processing device provided by the third embodiment of the application is shown in the figure. Figure 8 As shown in the figure, the violation data reporting processing device comprises:

[0105] The reporting content and user data acquisition module 801 is used for receiving the reporting content of the target business data of the reported user, and acquiring the user data of the reported user, wherein the reporting content at least includes the reported user identifier, the reporting type, the reporting reason and the reporting description content.

[0106] The machine audit module 802 is used for inputting the reporting content and the user data into the machine audit model to obtain the score of the target business data belonging to the violation data.

[0107] The manual audit queue determination module 803 is used for determining the target manual audit queue of the target business data based on the reporting content and the user data when the score is greater than or equal to the preset score threshold, and the target manual audit queue is the queue selected from multiple queues.

[0108] The manual audit module 804 is used for distributing the reporting content and the user data to the target manual audit queue for manual audit to obtain the manual audit result of the target business data.

[0109] Optionally, the report content and user data obtaining module 801 is specifically configured to:

[0110] obtain at least one of the following data of the reported user as user data:

[0111] account information, historical violation records, number of reports, risk control blacklist status, context data of target business data, historical behavior data of the reported user, and social relationship data.

[0112] Optionally, the method further comprises:

[0113] a first judgment module configured to determine whether the score is greater than or equal to a preset first score threshold; if yes, execute a violation determination module, and if no, execute a pending manual review module;

[0114] the violation determination module is configured to determine the target business data as violation data;

[0115] the pending manual review module is configured to determine the target business data as data to be manually reviewed when the score is greater than or equal to a preset second score threshold, and execute a manual review queue determination module 803, wherein the second score threshold is less than the first score threshold;

[0116] the filtering judgment module is configured to determine whether the target business data is violation data based on a preset filtering rule when the score is less than the second score threshold; if yes, execute the manual review queue determination module 803, and if no, execute a report ignoring module;

[0117] the report ignoring module is configured to not process the target business data when it is determined based on the preset filtering rule that the target business data is not violation data.

[0118] Optionally, the method further comprises a first feedback module configured to:

[0119] generate a primary review result that the target business data is initially determined as violation data and is in a manual review state;

[0120] feed back the primary review result to the reported user and the reported user.

[0121] Optionally, the manual review queue determination module 803 is specifically configured to:

[0122] input the report content and the user data into a preset target business data grading model to obtain a manual review queue to which the target business data belongs.

[0123] Optionally, the method further comprises a second feedback module configured to:

[0124] Receiving a manual review result of the target service data;

[0125] Feeding back the manual review result to the reporting user and the reported user.

[0126] Optionally, further comprising a data feedback and optimization module, configured to:

[0127] Receiving feedback data of the review result of the target service data;

[0128] Re-training the machine review model and calibrating the review scale of the manual review based on the feedback data.

[0129] The violation data reporting processing device provided by the embodiments of the present application can execute the violation data reporting processing method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0130] Embodiment four

[0131] Figure 9 A structural schematic diagram of an electronic device 90 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present application described and / or claimed in this document.

[0132] As shown in Figure 9 The electronic device 90 includes at least one processor 91, and a memory, such as a read-only memory (ROM) 92, a random access memory (RAM) 93, etc., which is communicatively connected to the at least one processor 91, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 91 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 92 or the computer program loaded from the storage unit 98 into the random access memory (RAM) 93. In the RAM 93, various programs and data required for the operation of the electronic device 90 can also be stored. The processor 91, the ROM 92, and the RAM 93 are connected to each other through a bus 94. An input / output (I / O) interface 95 is also connected to the bus 94.

[0133] A plurality of components in the electronic device 90 are connected to the I / O interface 95, including: an input unit 96, such as a keyboard, a mouse, etc.; an output unit 97, such as various types of displays, speakers, etc.; a storage unit 98, such as a magnetic disk, an optical disk, etc.; and a communication unit 99, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 99 allows the electronic device 90 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0134] The processor 91 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 91 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 91 performs various methods and processes described above, such as the reporting processing method of violation data.

[0135] In some embodiments, the reporting processing method of violation data can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 98. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 90 via the ROM 92 and / or the communication unit 99. When the computer program is loaded onto the RAM 93 and executed by the processor 91, one or more steps of the reporting processing method of violation data described above can be performed. Alternatively, in other embodiments, the processor 91 can be configured to perform the reporting processing method of violation data by any other appropriate means, such as by means of firmware.

[0136] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0137] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0138] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0140] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0141] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0142] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, and the present disclosure is not limited in this regard.

[0143] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalents, and / or alternatives come within the scope of the present disclosure as recited by the claims.

Claims

1. A method for handling a report of violation data, characterized by, The method comprises the following steps: receiving a report content of target service data of a reported user, obtaining user data of the reported user, wherein the report content at least comprises a reported user identifier, a report type, a report reason, and a description content of the target service data; inputting the report content and the user data into a machine review model to obtain a score of the target service data belonging to illegal data; when the score is greater than or equal to a preset score threshold, determining a target artificial review queue of the target service data based on the report content and the user data, wherein the target artificial review queue is a queue selected from a plurality of queues; allocating the report content and the user data to the target artificial review queue for artificial review to obtain an artificial review result of the target service data.

2. The method of claim 1, wherein, The method for obtaining the user data of the reported user comprises the following steps: obtaining at least one of the following data of the reported user as user data: account information, historical illegal record, number of reported times, risk control blacklist status, context data of the target service data, historical behavior data of the reported user, and social relationship data.

3. The method of claim 1, wherein, After inputting the report content and the user data into the machine review model to obtain the score of the target service data belonging to illegal data, the method further comprises the following steps: determining whether the score is greater than or equal to a preset first score threshold; if yes, determining that the target service data is illegal data; if no, when the score is greater than or equal to a preset second score threshold, determining that the target service data is data to be artificially reviewed, and performing the step of determining the target artificial review queue of the target service data based on the report content and the user data, wherein the second score threshold is less than the first score threshold; when the score is less than the second score threshold, determining whether the target service data is illegal data based on a preset filtering rule; if yes, performing the step of determining the target artificial review queue of the target service data based on the report content and the user data; if no, not processing the target service data.

4. The method of claim 3, wherein, After determining that the target service data is data to be artificially reviewed, the method further comprises the following steps: generating a primary review result that the target service data is preliminarily determined to be illegal data and the state is an artificial review state; feeding back the primary review result to the reported user and the reported user.

5. The method of claim 1, wherein, The method for determining the target artificial review queue of the target service data based on the report content and the user data comprises the following steps: inputting the report content and the user data into a preset target service data grading model to obtain an artificial review queue to which the target service data belongs.

6. The method of claim 1, wherein, After allocating the report content and the user data to the target artificial review queue for artificial review, the method further comprises the following steps: receiving an artificial review result of the target service data; feeding back the artificial review result to the reported user and the reported user.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises the following steps: receiving feedback data of the review result of the target service data; retraining the machine review model based on the feedback data and calibrating a review scale of the artificial review.

8. A violation data reporting processing apparatus characterized by comprising: The method comprises the following steps: The report content and user data obtaining module is configured to receive report content of target service data of a reported user, and obtain user data of the reported user, wherein the report content at least includes a reported user identifier, a report type, a report reason, and report description content; The machine review module is configured to input the report content and the user data into a machine review model to obtain a score of the target service data belonging to the violation data; The artificial review queue determination module is configured to, when the score is greater than or equal to a preset score threshold, determine a target artificial review queue of the target service data based on the report content and the user data, wherein the target artificial review queue is a queue selected from a plurality of queues; The artificial review module is configured to assign the report content and the user data to the target artificial review queue for artificial review to obtain an artificial review result of the target service data.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the violation data reporting processing method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to execute the violation data reporting processing method in any one of claims 1-7 when executed.

Images