Text information intelligent auditing method, device and equipment and storage medium
Through deep learning models combined with variant recognition, semantic analysis and context association technologies, the problem of identifying illegal content in the existing technology is solved, and high accuracy and efficient text review is achieved, especially when facing new threats. ability.
Patent Information
- Application Number
- CN202411036967.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-22
AI Technical Summary
Existing data leakage protection systems are difficult to identify specially modified violations, the false positive rate is high and the new threats cannot be dealt with, and it is especially difficult to identify illicit words and code words in combination with context, context, grammar or semantics.
Using deep learning model combined with variant recognition, semantic analysis and context association technology, through sensitive keyword databases and customized audit models for business scenarios, illegal vocabulary and their variants are identified, reducing manslaughter and missed killing, and improving the accuracy of illegal identification and audit efficiency.
Effectively identifying illegal vocabulary and its variants improves the accuracy and efficiency of text review, avoids misjudgment and misjudgment of a single model, and enhances the ability to respond to new threats.
Smart Images

Figure CN119006005A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for intelligent review of text information. Background Art
[0002] At present, as various social media platforms gradually mature, the threshold for information dissemination is gradually lowered, and users can easily spread information on the Internet. While the scale of user contributions continues to grow, content chaos is becoming increasingly prominent. Some lawless elements use social media to spread malicious promotional content. These harmful contents not only affect the user experience, but also bring great legal risks to the platform and have a very bad impact on the network environment. Therefore, how to identify and filter harmful content from massive information has become an important issue. At present, the bank's publishing terminal mainly uses the data leakage protection system to scan the text data in the marketing materials to be published to prevent the outflow of text data that violates the enterprise's security and compliance policies. The data leakage protection system is mainly used on the internal network boundary, Internet boundary and terminal equipment of the enterprise to detect, record and actively deal with the risks of sensitive data outbound and leakage. However, this method has limited auditing capabilities. The data leakage protection system mainly identifies illegal content through keywords or keyword pair matching, with a high false alarm rate, and lacks the ability to audit content such as mixed simplified and traditional Chinese, mixed uppercase and lowercase, special characters, emoticons, etc.; at the same time, it is also difficult to deal with new threats and cannot identify jargon and code words based on context, grammar or semantics.
[0003] There is an urgent need for an intelligent text information review method to solve the problem that the data leakage protection system in the existing technology is difficult to identify specially modified illegal content and has a high false alarm rate, and is also difficult to deal with new threats. Summary of the invention
[0004] In order to solve the problem that the data leakage protection system in the prior art is based on known sensitive fields and regular expressions for detection and cannot respond to new threats, and cannot identify jargon and secret words in the context, the embodiments of this specification provide a text information intelligent review method, device, equipment and storage medium, which can identify illegal words and their variants based on deep learning models through variant recognition, semantic analysis, context association and other technologies, and use customized review models for different business scenarios in the banking industry to avoid false positives and missed kills of a single model, thereby improving the accuracy of violation identification and the efficiency of review.
[0005] In order to solve the above technical problems, the specific technical solutions of this specification are as follows:
[0006] On the one hand, the embodiments of this specification provide a method for intelligent review of text information, including:
[0007] Acquire the text content to be published from the content to be published of the publishing terminal, and input the text content to be published into the intelligent review system;
[0008] The intelligent audit system pre-processes the text content to be published to generate text content to be audited;
[0009] Building matching recognition rules based on the sensitive keyword library, screening the text content to be reviewed according to the matching recognition rules, and obtaining suspicious text data;
[0010] Establish a data analysis model based on the business scenarios and review strategies involved in the content to be published;
[0011] Identify the suspicious text data according to the data analysis model to obtain illegal text data;
[0012] The illegal text data is analyzed to generate a text review result, and the result is returned to the publishing terminal.
[0013] Furthermore, preprocessing the text content to be published to generate the text content to be reviewed further includes:
[0014] The preprocessing includes at least one of the following: text denoising, normalization of information of the same category, word segmentation and sentence segmentation, and removal of stop words;
[0015] The text denoising is to delete the special symbols that are not helpful for Chinese word segmentation in the text content to be published;
[0016] Normalizing the information of the same category is to merge the number strings and emoticons in the text content to be published and replace them with a unified name;
[0017] The word segmentation is to combine continuous word sequences into word sequences based on a word segmentation algorithm and a word segmentation vocabulary;
[0018] The removal of stop words is to delete words in the suspicious text data that are in a preset stop word list.
[0019] Furthermore, the sensitive keyword library further includes:
[0020] The sensitive keyword library is generated according to sensitive words in the historical audit records of the intelligent audit system.
[0021] Furthermore, establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published further includes:
[0022] Generate target words according to the business scenarios involved in the content to be published;
[0023] The target word is expanded based on the target word and a preset semantic similarity to generate a scene vocabulary library.
[0024] Furthermore, establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published further includes:
[0025] The audit strategy includes at least one of the following: variant identification, semantic analysis, and context association;
[0026] Obtaining a variant recognition model of the scene vocabulary library through a preset variant word algorithm;
[0027] Acquire a semantic recognition model of the scene vocabulary library through a preset semantic algorithm;
[0028] The association recognition model of the scene vocabulary library is obtained through a preset context association algorithm.
[0029] Furthermore, the variant word algorithm further includes:
[0030] Obtain a continuous substring sequence having the same length as the target word;
[0031] Calculating the phonetic similarity and the glyph similarity between the substrings in the continuous substring sequence and the target word;
[0032] The substrings whose pronunciation similarity is greater than a preset pronunciation variant threshold are taken as pronunciation variant words, and the substrings whose shape similarity is greater than a preset shape variant threshold are taken as shape variant words.
[0033] Furthermore, the suspicious text data is identified according to the data analysis model, and obtaining the illegal text data further includes:
[0034] Inputting the suspicious text data into the variant recognition model to obtain a violation variant recognition result corresponding to the suspicious text data;
[0035] Inputting the suspicious text data into the semantic recognition model to obtain a violation semantic recognition result corresponding to the suspicious text data;
[0036] Inputting the suspicious text data into the association recognition model to obtain a violation association recognition result corresponding to the suspicious text data;
[0037] The illegal text data is generated according to the illegal variant recognition result, the illegal semantic recognition result and the illegal association recognition result.
[0038] Furthermore, analyzing the illegal text data to generate text audit results further includes:
[0039] Obtaining the confidence level of the to-be-published text content according to the ratio of the illegal text data to the text length;
[0040] Perform risk classification according to the content of the illegal text data;
[0041] According to the confidence level of the illegal text data and the risk classification, illegal handling opinions and illegal content distribution are generated as the text review results.
[0042] Furthermore, the intelligent audit system further includes synchronous audit and asynchronous audit.
[0043] If the text content to be published is a short text, the synchronous review service is called;
[0044] If the text content to be published is a long text, the asynchronous review service is called.
[0045] Further, analyzing the illegal text data to generate a text review result, and returning it to the publishing terminal further includes:
[0046] If the synchronous audit service is called, the audit result will be returned directly;
[0047] If the asynchronous audit service interface is called, the audit serial number is returned, and the publishing terminal calls the audit result query interface for polling according to the serial number, or the publishing terminal sets a callback address, and the intelligent audit system sends the audit result to the callback address.
[0048] On the other hand, the embodiment of the present specification also provides a text information intelligent review device, including:
[0049] A content acquisition unit, used to acquire text content to be published from the content to be published of the publishing terminal, and input the text content to be published into the intelligent audit system;
[0050] A preprocessing unit, used for the intelligent audit system to preprocess the text content to be published to generate the text content to be audited;
[0051] A suspicious text screening unit, used to construct a matching recognition rule based on a sensitive keyword library, and screen the text content to be reviewed according to the matching recognition rule to obtain suspicious text data;
[0052] A model building unit, used to build a data analysis model according to the business scenarios and review strategies involved in the content to be published;
[0053] An illegal text acquisition unit, used to identify the suspicious text data according to the data analysis model and acquire the illegal text data;
[0054] The result generating unit is used to analyze the illegal text data to generate text review results and return them to the publishing terminal.
[0055] On the other hand, an embodiment of the present specification further provides a computer device, including a memory, a processor, and a computer program stored in the memory, and the processor implements the above method when executing the computer program.
[0056] On the other hand, an embodiment of the present specification further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer device, the above method is executed.
[0057] Finally, the embodiments of this specification also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the above method is performed.
[0058] By using the embodiments of this specification, it is realized that the content to be published in the publishing terminal is reviewed by the intelligent review system. First, the text content to be published is obtained from the content to be published in the publishing terminal and input into the intelligent review system, so as to ensure that the intelligent review system obtains the complete text content to be published. Then, the intelligent review system pre-processes the text content to be published to generate the text content to be reviewed, so as to avoid useless interference information in the text content affecting the review process. Then, matching recognition rules are constructed based on the sensitive keyword library, wherein the sensitive keyword library can be sensitive words extracted from past harmful texts, so as to narrow the scope of texts that need to be screened and obtain suspicious text data; then, a data analysis model is established according to the business scenarios and review strategies involved in the content to be published, so as to improve the pertinence of the data analysis model, avoid the excessive amount of data affecting the review efficiency or the insufficient pertinence of the review leading to too low review efficiency, and then use the data analysis model to identify the suspicious text data and obtain the illegal text data, so as to realize the recognition of multiple variants through the learning model, and finally generate the text review result and return it to the publishing terminal. It realizes the identification of illegal words and their variants through variant recognition, semantic analysis, context association and other technologies, thereby avoiding the misjudgment of harmless text and the omission of harmful text by a single model; improves the accuracy of text review and avoids missed detections caused by users maliciously evading inspections. At the same time, it uses customized audit models for different business scenarios in the banking industry to improve the targetedness of the audit model, thereby improving the efficiency of text data review. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0060] Figure 1 The figure shows a schematic diagram of a system for implementing a text information intelligent review method according to an embodiment of the present specification;
[0061] Figure 2 It is a flowchart of a text information intelligent review method according to an embodiment of this specification;
[0062] Figure 3 The process of establishing a data analysis model according to business scenarios and audit strategies in the embodiments of this specification is shown;
[0063] Figure 4 The process of implementing the variant word algorithm in the embodiment of this specification is shown;
[0064] Figure 5 The figure shows the process of identifying and obtaining illegal text data according to the data analysis model in the embodiment of this specification;
[0065] Figure 6 The text description embodiment shown in the figure analyzes the illegal text data to generate the text audit result;
[0066] Figure 7 The figure shows a schematic diagram of the structure of a text information intelligent review device according to an embodiment of the present specification;
[0067] Figure 8 The figure shows a schematic diagram of the structure of a computer device according to an embodiment of the present specification.
[0068]
Description of reference numerals
[0069] 101. Release terminal;
[0070] 102. Intelligent audit system;
[0071] 701, content acquisition unit;
[0072] 702, pre-processing unit;
[0073] 703. Suspicious text screening unit;
[0074] 704, model building unit;
[0075] 705. Illegal text acquisition unit;
[0076] 706. result generating unit;
[0077] 802. Computer equipment;
[0078] 804. Processing equipment;
[0079] 806. Storage resources;
[0080] 808, driving mechanism;
[0081] 810, input / output module;
[0082] 812. Input device;
[0083] 814. Output device;
[0084] 816. Presentation equipment;
[0085] 818. Graphical user interface;
[0086] 820, network interface;
[0087] 822, communication link;
[0088] 824. Communication bus. DETAILED DESCRIPTION
[0089] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.
[0090] It should be noted that the terms "first", "second", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0091] like Figure 1The figure shows a schematic diagram of a system for implementing a text information intelligent audit method according to an embodiment of the present invention, which may include a publishing terminal 101 and an intelligent audit system 102. A communication connection is established between the publishing terminal 101 and the intelligent audit system 102 to enable data interaction. The publishing terminal 101 may send content to be published to the intelligent audit system 102, which includes text content to be published, and may also include the business scenarios and audit strategies involved, such as bank loan scenarios. In addition, due to the protection of information security, the content to be published is desensitized, and the fields involving account information, contact information, etc. are replaced with special symbols. The intelligent audit system 102 then audits the content to be published, and then returns the audit results to the publishing terminal.
[0092] In the embodiments of this specification, the intelligent audit system 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms.
[0093] In an optional embodiment, the publishing terminal 101 may include, but is not limited to, electronic devices such as smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Optionally, the operating system running on the electronic device may include, but is not limited to, Android, IOS, Linux, and Windows.
[0094] In addition, it should be noted that in practical applications, Figure 1 The application environment shown can be applied to scenarios such as financial industry information release and customer service, and this manual does not impose any restrictions.
[0095] In order to solve the problems existing in the prior art, the embodiments of this specification provide a text information intelligent review method, device, equipment and storage medium. Figure 2 The flowchart of a text information intelligent review method provided by an embodiment of this specification is shown in this figure, which describes the process of reviewing and guiding personnel text information. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2As shown, the method may include:
[0096] Step 201: obtaining text content to be published from the content to be published on the publishing terminal, and inputting the text content to be published into the intelligent review system;
[0097] Step 202: the intelligent audit system pre-processes the text content to be published to generate text content to be audited;
[0098] Step 203: construct matching identification rules based on the sensitive keyword library, and screen the text content to be reviewed according to the matching identification rules to obtain suspicious text data;
[0099] Step 204: Establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published;
[0100] Step 205: Identify the suspicious text data according to the data analysis model to obtain illegal text data;
[0101] Step 206: Analyze the illegal text data to generate text review results, and return them to the publishing terminal.
[0102] Through the method of the embodiment of this specification, it is realized that the content to be published in the publishing terminal is reviewed by the intelligent review system. First, the text content to be published is obtained from the content to be published in the publishing terminal and input into the intelligent review system, so as to ensure that the intelligent review system obtains the complete text content to be published. Then, the intelligent review system pre-processes the text content to be published to generate the text content to be reviewed, so as to avoid useless interference information in the text content affecting the review process. Then, matching recognition rules are constructed based on the sensitive keyword library, wherein the sensitive keyword library can be sensitive words extracted from past harmful texts, so as to narrow the scope of texts that need to be screened and obtain suspicious text data; then, a data analysis model is established according to the business scenarios and review strategies involved in the content to be published, so as to improve the pertinence of the data analysis model, avoid the excessive amount of data affecting the review efficiency or the insufficient pertinence of the review leading to too low review efficiency, and then use the data analysis model to identify the suspicious text data and obtain the illegal text data, so as to realize the recognition of multiple variants through the learning model, and finally generate the text review result and return it to the publishing terminal. It realizes the identification of illegal words and their variants through variant recognition, semantic analysis, context association and other technologies, thereby avoiding the misjudgment of harmless text and the omission of harmful text by a single model; improves the accuracy of text review and avoids missed detections caused by users maliciously evading inspections. At the same time, it uses customized audit models for different business scenarios in the banking industry to improve the targetedness of the audit model, thereby improving the efficiency of text data review.
[0103] In the embodiments of this specification, the publishing terminal can call the intelligent audit system through the API interface, and the intelligent audit system provides synchronous audit services, asynchronous audit services and result query services to the outside world; wherein the intelligent audit system can identify illegal words and their variants based on the deep learning model through variant recognition, semantic analysis, context association and other technologies. After the intelligent audit system obtains the business demand text audit request, it first authenticates according to the application information and the key to determine whether the publishing terminal has the authority to call the intelligent audit service, and then completes the data preprocessing of the text to be audited, builds matching recognition rules based on the sensitive keyword library, and screens the content of the text to be audited according to the matching recognition rules, thereby reducing the scope of text that needs to be audited by the model, and establishes variant recognition, semantic analysis, context association and other data analysis models according to business scenarios and audit strategies, and audits the content to be audited. By customizing the audit model and risk label system according to different business scenarios of the banking industry, it meets the needs of business development, avoids the false positives and omissions of a single model, and has a high accuracy rate of illegal identification.
[0104] In the embodiment of the present specification, in order to improve the efficiency of text review and reduce the influence of useless information in the text content on the review process, the text content to be published is preprocessed, and the generation of the text content to be reviewed further includes:
[0105] The preprocessing includes at least one of the following: text denoising, normalization of information of the same category, word segmentation and sentence segmentation, and removal of stop words;
[0106] The text denoising is to delete the special symbols that are not helpful for Chinese word segmentation in the text content to be published;
[0107] Normalizing the information of the same category is to merge the number strings and emoticons in the text content to be published and replace them with a unified name;
[0108] The word segmentation is to combine continuous word sequences into word sequences based on a word segmentation algorithm and a word segmentation vocabulary;
[0109] The removal of stop words is to delete words in the suspicious text data that are in a preset stop word list.
[0110] Specifically, text denoising is to delete special symbols that are not helpful for Chinese word segmentation in the text content to be published, such as special punctuation marks such as "·", "*", and "#", so as to prevent publishers of harmful information from using such symbols to segment the original keywords and thus circumvent the censorship mechanism; normalization of information of the same category is to use regular expressions to replace the account number, mobile phone number, emoticons and other content in the text content to be published with names, thereby preventing publishers from using numbers and other content to circumvent the censorship mechanism, and at the same time protecting the account login privacy information in the content to be published, thereby avoiding the risk of the auditor obtaining relevant information in the process of generating the audit results.
[0111] In the embodiment of this specification, in order to reduce the text content that needs to be reviewed by the data analysis model, it is necessary to screen the text content to be reviewed to obtain suspicious text data, wherein the sensitive keyword library further includes:
[0112] The sensitive keyword library is generated according to sensitive words in the historical audit records of the intelligent audit system.
[0113] Specifically, each time the intelligent audit system finds illegal text data, it will store the illegal text data and related contextual content and generate keywords. At the same time, after each audit, the sensitive keyword library will be updated according to the new audit results to ensure its accuracy.
[0114] In the embodiment of the present specification, in order to improve the pertinence of the data analysis model, in step 204, establishing the data analysis model according to the business scenario and the review strategy involved in the content to be published further includes:
[0115] Generate target words according to the business scenarios involved in the content to be published;
[0116] The target word is expanded based on the target word and a preset semantic similarity to generate a scene vocabulary library.
[0117] Furthermore, in order to establish the data analysis model, it is necessary to further expand the scene vocabulary library, such as Figure 3 As shown, establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published further includes:
[0118] The audit strategy includes at least one of the following: variant identification, semantic analysis, and context association;
[0119] Step 301: obtaining a variant recognition model of the scene vocabulary library through a preset variant word algorithm;
[0120] Step 302: Obtaining a semantic recognition model of the scene vocabulary library through a preset semantic algorithm;
[0121] Step 303: Acquire the association recognition model of the scene vocabulary library through a preset context association algorithm.
[0122] In step 301, a variant recognition model of the scene vocabulary library is obtained by using a preset variant word algorithm, such as Figure 4 As shown, the variant word algorithm further includes:
[0123] Step 401: Obtain a continuous substring sequence having the same length as the target word;
[0124] Step 402: Calculate the phonetic similarity and the glyph similarity between the substrings in the continuous substring sequence and the target word;
[0125] Step 403: taking the substrings whose pronunciation similarity is greater than a preset pronunciation variant threshold as pronunciation variant words, and taking the substrings whose glyph similarity is greater than a preset glyph variant threshold as glyph variant words.
[0126] Specifically, in one embodiment of the present specification, the identified continuous substring sequence is marked and classified according to the pronunciation similarity and the glyph similarity. It is used as the variant word of the scene vocabulary library, and the semantic similarity is calculated based on this, and the vocabulary scale is further expanded to establish the final vocabulary library. At the same time, in another embodiment of the present specification, a special field sentiment tendency dictionary can also be established, and a vocabulary library of relevant special / special fields can be established by casting a net and accumulating a large amount of data in a specific field.
[0127] In the embodiments of this specification, in order to quickly and accurately identify illegal text data, such as Figure 5 As shown, identifying the suspicious text data according to the data analysis model, obtaining the illegal text data further includes:
[0128] Step 501: inputting the suspicious text data into the variant recognition model to obtain the illegal variant recognition result corresponding to the suspicious text data;
[0129] Step 502: inputting the suspicious text data into the semantic recognition model to obtain a violation semantic recognition result corresponding to the suspicious text data;
[0130] Step 503: inputting the suspicious text data into the association recognition model to obtain the violation association recognition result corresponding to the suspicious text data;
[0131] Step 504: Generate the illegal text data according to the illegal variant recognition result, the illegal semantic recognition result and the illegal association recognition result.
[0132] Specifically, in the process of identifying suspicious text data, cutting-edge natural language processing algorithms are used, and the business scenarios and review strategies involved in the content to be published are combined with a massive sensitive keyword library to build a model, so as to quickly and accurately identify illegal text data. For example, the finite state automaton (FSA) is applied to variant recognition, which can effectively improve the recall rate of online variant black samples while ensuring accuracy; a complex model system built based on NLP technologies such as Fasttext, HMM, CRF, Word2Vec, etc. is used to obtain the illegal semantic recognition results and illegal association recognition results corresponding to the suspicious text data. At the same time, through the joint recognition of multiple models, the misjudgment and omission of a single model can be effectively avoided, and the violation identification is highly accurate.
[0133] In the embodiment of this specification, in order to determine whether the illegal text data affects the release of the content to be released, Figure 6 As shown, analyzing the illegal text data to generate text audit results further includes:
[0134] Step 601: Obtaining the confidence level of the to-be-published text content according to the ratio of the illegal text data to the text length;
[0135] Step 602: performing risk classification according to the content of the illegal text data;
[0136] Step 603: Generate violation handling opinions and violation content distribution as the text review result according to the confidence level of the violation text data and the risk classification.
[0137] 1. Specifically, in one embodiment of the present specification, the confidence level of the content of the text to be published can be obtained according to the ratio of the illegal text data and the length of the text, for example, the confidence level score is performed according to the ratio, and the larger the confidence level score, the greater the risk of the content to be published; at the same time, the risk classification of the illegal content is determined by judging the content of the illegal text data, such as pornography, terrorism, insults, advertisements, etc.; then, according to the confidence level and the risk classification, the violation handling opinion is generated, for example, if the confidence level score is lower than the threshold, and the illegal content is a normal text such as an advertisement, it is marked as approved; if the confidence level score of the illegal text is higher than the threshold or the illegal content is pornography, terrorism, insults, etc., it is marked as rejected for the release request; if a suspicious text is encountered and further manual review is required, it is marked as a suspicious text. At the same time, the distribution of the illegal content is marked in the text review results, such as the paragraph, number of lines, number of columns, etc. to which the illegal text belongs, so as to help the auditors quickly locate it.
[0138] In the embodiment of this specification, in order to improve the audit efficiency of the intelligent audit system, the intelligent audit system further includes synchronous audit and asynchronous audit.
[0139] If the text content to be published is a short text, the synchronous review service is called;
[0140] If the text content to be published is a long text, the asynchronous review service is called.
[0141] Among them, when the intelligent audit system calls different audit services, the way of returning the audit results will also change accordingly, analyzing the illegal text data to generate text audit results, and returning them to the publishing terminal further includes:
[0142] If the synchronous audit service is called, the audit result will be returned directly;
[0143] If the asynchronous audit service interface is called, the audit serial number is returned, and the publishing terminal calls the audit result query interface for polling according to the serial number, or the publishing terminal sets a callback address, and the intelligent audit system sends the audit result to the callback address.
[0144] Specifically, the synchronous review service can be called for short text review needs, such as texts with a length of less than 10,000 Chinese characters, and the review results will be returned in real time; while the asynchronous review service is used for long text review needs, and the asynchronous review service is called. The publishing terminal can obtain the review results by polling the result query interface or setting the callback address. At the same time, depending on the different calling interfaces of the publishing terminal, if the synchronous review service interface is called, the review result will be returned directly; if the asynchronous review service interface is called, the transaction serial number will be returned first, and the business provider can call the audit result query interface to poll according to the serial number, or the publishing terminal can set the callback address, and the intelligent review system will send the review results to the callback address.
[0145] The embodiment of this specification also provides a text information intelligent review device, such as Figure 7 As shown, including
[0146] The content acquisition unit 701 is used to acquire the text content to be published from the content to be published of the publishing terminal, and input the text content to be published into the intelligent audit system;
[0147] A preprocessing unit 702, used for the intelligent audit system to preprocess the text content to be published to generate text content to be audited;
[0148] The suspicious text screening unit 703 is used to construct a matching recognition rule based on the sensitive keyword library, and screen the text content to be reviewed according to the matching recognition rule to obtain suspicious text data;
[0149] A model building unit 704 is used to build a data analysis model according to the business scenario and review strategy involved in the content to be published;
[0150] The illegal text acquisition unit 705 is used to identify the suspicious text data according to the data analysis model to acquire the illegal text data;
[0151] The result generating unit 706 is used to analyze the illegal text data to generate text review results and return them to the publishing terminal.
[0152] Since the principle of solving the problem by the above device is similar to that of the above method, the implementation of the above device can refer to the implementation of the above method, and the repeated parts will not be repeated.
[0153] like Figure 8 The structure diagram of the computer device of the embodiment of the present invention is shown. The apparatus in this specification can be the computer device in this embodiment, and executes the method of the above-mentioned specification. The computer device 802 may include one or more processing devices 804, such as one or more central processing units (CPUs), and each processing unit may implement one or more hardware threads. The computer device 802 may also include any storage resource 806, which is used to store any kind of information such as code, settings, data, etc. Non-limitingly, for example, the storage resource 806 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any storage resource can use any technology to store information. Further, any storage resource can provide volatile or non-volatile retention of information. Further, any storage resource can represent a fixed or removable component of the computer device 802. In one case, when the processing device 804 executes an associated instruction stored in any storage resource or a combination of storage resources, the computer device 802 can perform any operation of the associated instruction. The computer device 802 also includes one or more drive mechanisms 808 for interacting with any storage resources, such as a hard disk drive mechanism, an optical disk drive mechanism, and the like.
[0154] The computer device 802 may also include an input / output module 810 (I / O) for receiving various inputs (via input devices 812) and for providing various outputs (via output devices 814). A specific output mechanism may include a presentation device 816 and an associated graphical user interface (GUI) 818. In other embodiments, the input / output module 810 (I / O), the input device 812, and the output device 814 may not be included, and the computer device 802 may be used as a computer device in a network. The computer device 802 may also include one or more network interfaces 820 for exchanging data with other devices via one or more communication links 822. One or more communication buses 824 couple the components described above together.
[0155] The communication link 822 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 822 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.
[0156] Corresponds to Figures 2 to 6 The method in the embodiment of the present specification also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are executed.
[0157] The embodiment of the present specification also provides a computer-readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the following Figures 2 to 6 The method shown.
[0158] It should be understood that in the various embodiments of this specification, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.
[0159] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this specification generally indicates that the associated objects before and after are in an "or" relationship.
[0160] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.
[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0162] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.
[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this specification.
[0164] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0166] Specific embodiments are used in this specification to illustrate the principles and implementation methods of this specification. The description of the above embodiments is only used to help understand the methods and core ideas of this specification. At the same time, for those skilled in the art, according to the ideas of this specification, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this specification.
Claims
1. A text information intelligent review method, characterized in that: The method comprises, Acquire the text content to be published from the content to be published of the publishing terminal, and input the text content to be published into the intelligent review system; The intelligent audit system pre-processes the text content to be published to generate text content to be audited; Building matching recognition rules based on the sensitive keyword library, screening the text content to be reviewed according to the matching recognition rules, and obtaining suspicious text data; Establish a data analysis model based on the business scenarios and review strategies involved in the content to be published; Identify the suspicious text data according to the data analysis model to obtain illegal text data; The illegal text data is analyzed to generate a text review result, and the result is returned to the publishing terminal.
2. The text information intelligent review method according to claim 1, characterized in that: Preprocessing the text content to be published to generate the text content to be reviewed further includes: The preprocessing includes at least one of the following: text denoising, normalization of information of the same category, word segmentation and sentence segmentation, and removal of stop words; The text denoising is to delete the special symbols that are not helpful for Chinese word segmentation in the text content to be published; Normalizing the information of the same category is to merge the number strings and emoticons in the text content to be published and replace them with a unified name; The word segmentation is to combine continuous word sequences into word sequences based on a word segmentation algorithm and a word segmentation vocabulary; The removal of stop words is to delete words in the suspicious text data that are in a preset stop word list.
3. The text information intelligent review method according to claim 1, characterized in that: The sensitive keyword library further includes: The sensitive keyword library is generated according to sensitive words in the historical audit records of the intelligent audit system.
4. The text information intelligent review method according to claim 1, characterized in that: Establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published further includes: Generate target words according to the business scenarios involved in the content to be published; The target word is expanded based on the target word and a preset semantic similarity to generate a scene vocabulary library.
5. The text information intelligent review method according to claim 4, characterized in that: Establishing a data analysis model according to the business scenarios and review strategies involved in the content to be published further includes: The audit strategy includes at least one of the following: variant identification, semantic analysis, and context association; Obtaining a variant recognition model of the scene vocabulary library through a preset variant word algorithm; Acquire a semantic recognition model of the scene vocabulary library through a preset semantic algorithm; The association recognition model of the scene vocabulary library is obtained through a preset context association algorithm.
6. The text information intelligent review method according to claim 5, characterized in that: The variant word algorithm further includes: Obtain a continuous substring sequence having the same length as the target word; Calculating the phonetic similarity and the glyph similarity between the substrings in the continuous substring sequence and the target word; The substrings whose pronunciation similarity is greater than a preset pronunciation variant threshold are taken as pronunciation variant words, and the substrings whose shape similarity is greater than a preset shape variant threshold are taken as shape variant words.
7. The text information intelligent review method according to claim 5, characterized in that: Identifying the suspicious text data according to the data analysis model, and obtaining the illegal text data further includes: Inputting the suspicious text data into the variant recognition model to obtain a violation variant recognition result corresponding to the suspicious text data; Inputting the suspicious text data into the semantic recognition model to obtain a violation semantic recognition result corresponding to the suspicious text data; Inputting the suspicious text data into the association recognition model to obtain a violation association recognition result corresponding to the suspicious text data; The illegal text data is generated according to the illegal variant recognition result, the illegal semantic recognition result and the illegal association recognition result.
8. The text information intelligent review method according to claim 1, characterized in that: Analyzing the illegal text data to generate text audit results further includes: Obtaining the confidence level of the to-be-published text content according to the ratio of the illegal text data to the text length; Perform risk classification according to the content of the illegal text data; According to the confidence level of the illegal text data and the risk classification, illegal handling opinions and illegal content distribution are generated as the text review results.
9. The text information intelligent review method according to claim 1, characterized in that: The intelligent audit system further includes synchronous audit and asynchronous audit. If the text content to be published is a short text, the synchronous review service is called; If the text content to be published is a long text, the asynchronous review service is called.
10. The text information intelligent review method according to claim 9, characterized in that: Analyzing the illegal text data to generate text review results and returning them to the publishing terminal further includes: If the synchronous audit service is called, the audit result will be returned directly; If the asynchronous audit service interface is called, the audit serial number is returned, and the publishing terminal calls the audit result query interface for polling according to the serial number, or the publishing terminal sets a callback address, and the intelligent audit system sends the audit result to the callback address.
11. A text information intelligent review device, characterized in that: include, A content acquisition unit, used to acquire text content to be published from the content to be published of the publishing terminal, and input the text content to be published into the intelligent audit system; A preprocessing unit, used for the intelligent audit system to preprocess the text content to be published to generate the text content to be audited; A suspicious text screening unit, used to construct a matching recognition rule based on a sensitive keyword library, and screen the text content to be reviewed according to the matching recognition rule to obtain suspicious text data; A model building unit, used to build a data analysis model according to the business scenarios and review strategies involved in the content to be published; An illegal text acquisition unit, used to identify the suspicious text data according to the data analysis model and acquire the illegal text data; The result generating unit is used to analyze the illegal text data to generate text review results and return them to the publishing terminal.
12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.
14. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.
Citation Information
Cited By
Intelligent contract auditing method and device through semantic analysis
CN119477227A
Safety protection method and device based on LLM safety protection system
CN120181065A
We-media content publishing, auditing and verifying method and system
CN120449890A
Text review method and device, storage medium and program product
CN121117859A
