AI-based social platform content filtering management system, method and storage medium

Abstract

The present invention discloses an AI-based social platform content filtering management system, method and storage medium, which relate to the field of social platform content filtering and solve the current problem of being unable to accurately identify the content in a text. The system comprises a verification module, a text splitting module, a word segmentation recognition module and a content filtering module; the verification module is used to verify the terminal status of a sending terminal, and if it is on a blacklist, the subsequent steps are interrupted, and if it is on a whitelist, the subsequent steps are continued; the text splitting module splits the actual original text to obtain a plurality of actual original short sentences corresponding to the actual original text and word segmentation results corresponding to the actual original short sentences; the word segmentation recognition module identifies the violation of the word segmentation results corresponding to the actual original short sentences and obtains a violation score value for each actual original short sentence; the content filtering module performs content filtering on the actual original text, and the present invention realizes accurate recognition and precise filtering of the text.

Description

Technical Field

[0001] The present invention relates to the field of social platform content filtering, and in particular to an AI-based social platform content filtering management system, method, and storage medium. Background Art

[0002] Social platforms are online platforms built through the internet and mobile communication technologies to facilitate social interaction and information sharing among users. These platforms offer features such as user registration, profile creation, content publishing, viewing other users' content, and interactive commenting, aiming to connect people and promote the dissemination and exchange of information. As important media for information dissemination, social platforms must comply with national and regional laws and regulations, including cybersecurity laws and freedom of speech laws, to prevent the dissemination of illegal content. Therefore, they must ensure users feel safe and have a positive experience on the platforms. Content filtering management systems can help social platforms promptly identify and filter out illegal content.

[0003] However, currently, when filtering Chinese text on social platforms, fixed filtering criteria are usually used to filter out relevant text. However, this method does not analyze the overall situation of the text, and thus cannot confirm the text situation of each short sentence in the text, making it impossible to make appropriate responses, resulting in a poor user experience.

[0004] To this end, we propose an AI-based social platform content filtering management system, method and storage medium. Summary of the Invention

[0005] The purpose of the present invention is to propose a method for automatically mapping the topology of an electric grid in an important power supply guaranteeing location, so as to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] AI-based social platform content filtering and management system, including data collection module, verification module, sending terminal, text segmentation module, word segmentation and recognition module, content filtering module and display terminal;

[0008] The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module;

[0009] The verification module is used to verify the terminal status of the sending terminal, and if the sending terminal is on the blacklist, the subsequent steps are interrupted, and if the sending terminal is on the whitelist, the subsequent steps are continued;

[0010] The text splitting module is used to split the actual original text, obtain multiple actual original short sentences corresponding to the actual original text and corresponding word segmentation results and send them to the word segmentation recognition module;

[0011] The word segmentation recognition module is used to identify violations of the word segmentation results corresponding to the actual original short sentences, obtain the violation score value of each actual original short sentence and send it to the content filtering module; the content filtering module is used to perform content filtering on the actual original text, obtain the corresponding actual replacement text and send it to the display terminal or add the sending IP address of the corresponding sending terminal to the blacklist;

[0012] The display terminal is used to display the actual replacement text.

[0013] Furthermore, the working process of the verification module is as follows:

[0014] Get the sending IP address of the sending terminal, and then get the IP address corresponding to the blacklist set;

[0015] Compare the sending IP address of the sending terminal with the IP address corresponding to the blacklist set;

[0016] If the sending IP address of the sending terminal successfully matches the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the blacklist;

[0017] If the sending IP address of the sending terminal fails to match the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the whitelist.

[0018] Furthermore, the text splitting module's splitting process is as follows:

[0019] Obtain the actual original text, and divide the actual original text into n actual original short sentences using punctuation marks as delimiters, where n is the number of the actual original short sentence, and n=1, 2, ..., z, where z is the maximum value of the number;

[0020] The actual original short sentence is segmented by using stop words to obtain m segmentation results corresponding to the actual original short sentence.

[0021] Furthermore, the recognition process of the word segmentation recognition module is as follows:

[0022] Get m word segmentation results corresponding to n actual original short sentences;

[0023] The first segmentation result of the first actual original sentence is recorded as the first recognized segmentation;

[0024] Match the first recognized word with the original training text. If no match is found, select the next recognized word for matching.

[0025] If the match is successful, obtain x original training sentences corresponding to the first recognition word;

[0026] The original training sentence is compared with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences.

[0027] Furthermore, the recognition process of the word segmentation recognition module also includes:

[0028] The first violation rate DYW of the first recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of original training sentences;

[0029] Let the event "the original training sentence corresponding to the first recognition segmentation is an illegal sentence" be recorded as A, then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A)=DYW;

[0030] The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text;

[0031] The event "there are illegal short sentences in the original training text" is recorded as event B, then P(B) represents the probability of there being illegal short sentences in the original training text, that is, P(B)=YSW;

[0032] Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B);

[0033] Calculate P(B|A) as the predicted violation rate of the first identified word;

[0034] Similarly, the predicted violation rates are calculated for the second to the mth recognized segmentation in the first actual original sentence, and the predicted violation rates for the m recognized segmentations corresponding to the first actual original sentence are obtained;

[0035] The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence;

[0036] Similarly, the predicted violation rate YWGn of n actual original sentences in the actual original text is calculated;

[0037] The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence.

[0038] Furthermore, the calculation process of the predicted violation rate corresponding to the first recognition segmentation is equivalent to:

[0039] Known conditional probability formula: P(A|B)=P(AB) / P(B);

[0040] Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A);

[0041] So we get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B);

[0042] Among them, P(B|A) is the probability of event B under the premise that event A occurs; P(A|B) is the probability of event A under the premise that event B occurs; P(AB) is the probability of event AB occurring at the same time.

[0043] Furthermore, the working process of the content filtering module is as follows:

[0044] Get the violation score value corresponding to the actual original sentence;

[0045] Add up the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text;

[0046] If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0047] If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no action is taken.

[0048] Furthermore, the working process of the content filtering module also includes:

[0049] If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text;

[0050] The content filtering module sends the actual replacement text to the text splitting module and recalculates the text violation score value corresponding to the actual replacement text;

[0051] If the text violation score of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display;

[0052] If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal will be added to the blacklist.

[0053] The second aspect is the AI-based social platform content filtering and management method, which includes:

[0054] Step S101: Verify whether the sending terminal is on the blacklist. If it is on the blacklist, terminate the subsequent operations. If it is on the whitelist, execute the subsequent steps.

[0055] Step S102: splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results;

[0056] Step S103: Identify the violation of the word segmentation result corresponding to the actual original short sentence to obtain the violation score value corresponding to the actual original short sentence;

[0057] Step S104 , performing content filtering on the actual original text, obtaining a corresponding actual replacement text and sending it to a display terminal for display or adding the sending IP address of the corresponding sending terminal to a blacklist.

[0058] Furthermore, step S103 includes the following sub-steps:

[0059] Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations;

[0060] Step S1032: Match the recognized segmented word with the original training text. If the match is not successful, select the next recognized segmented word for matching. If the match is successful, obtain multiple original training sentences corresponding to the recognized segmented word.

[0061] Step S1033: Compare the original training sentence with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences.

[0062] Step S1034: The violation rate of the recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of original training sentences. The original violation rate of the original training text is obtained by dividing the number of violation sentences in the original training text by the total number of sentences in the original training text. The predicted violation rates of the recognition segmentation corresponding to the multiple actual original sentences are calculated.

[0063] Step S1035 : Select the predicted violation rate with the largest value as the predicted violation rate of the actual original sentence, and use the predicted violation rate as the violation score value corresponding to the actual original sentence.

[0064] Furthermore, step S104 includes the following sub-steps:

[0065] Step S1041: Obtain the violation score value corresponding to the actual original short sentence, and add the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text;

[0066] Step S1042: If the text violation score of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0067] Step S1043: If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed. If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text.

[0068] In step S1044, the content filtering module sends the actual replacement text to the text splitting module and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

[0069] In a third aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements an AI-based social platform content filtering and management method.

[0070] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0071] The present invention first obtains the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal through a data acquisition module, sends the actual original text corresponding to the sending terminal to a text splitting module and sends the sending IP address of the sending terminal to a verification module, uses the verification module to verify whether the sending terminal is a blacklisted terminal, and if it is a whitelisted terminal, terminates the subsequent steps, and if it is a whitelisted terminal, splits the actual original text through the text splitting module to obtain multiple actual original short sentences and corresponding word segmentation results corresponding to the actual original text, and then uses the word segmentation recognition module to identify the violation of the word segmentation results corresponding to the actual original short sentences to obtain a violation score value for each actual original short sentence, and the content filtering module performs content filtering on the actual original text in combination with the violation score value, obtains the corresponding actual replacement text and sends it to the display terminal for display or adds the sending IP address of the corresponding sending terminal to the blacklist, thereby realizing accurate recognition and precise filtering of text. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0073] Figure 1 This is a block diagram of the overall system of the present invention.

[0074] Figure 2 This is a flow chart showing the actual original text in the present invention.

[0075] Figure 3 Flow chart of the method in the present invention.

[0076] Figure 4 It is a structural diagram of the computer device in the present invention. DETAILED DESCRIPTION

[0077] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0078] Example 1: Please refer to Figure 1-Figure 2 As shown, the technical solution provided by the present invention is: an AI-based social platform content filtering and management system, including a data acquisition module, a verification module, a sending terminal, a database, a text splitting module, a word segmentation and recognition module, a content filtering module and a display terminal;

[0079] The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, and send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module, wherein the actual original text is the text content edited and sent by the user, and the sending IP address is specifically a 128-bit address represented by eight groups of hexadecimal numbers separated by colons;

[0080] The database is used to store a blacklist set, original training text, and a set of illegal words, and send the blacklist set to the verification module, and send the original training text and the set of illegal words to the text splitting module;

[0081] The verification module is used to verify the terminal status of the sending terminal. The working process of the verification module is specifically as follows:

[0082] Get the sending IP address of the sending terminal;

[0083] Then obtain the IP address corresponding to the blacklist set;

[0084] Compare the sending IP address of the sending terminal with the IP addresses corresponding to the blacklist set;

[0085] If the sending IP address of the sending terminal successfully matches the IP addresses corresponding to the blacklist set, then the sending terminal is determined to be on the blacklist;

[0086] If the sending IP address of the sending terminal fails to match the IP addresses corresponding to the blacklist set, then the sending terminal is determined to be on the white list;

[0087] If the verification module analyzes that the sending terminal is on the blacklist, no subsequent operations are performed; if the verification module analyzes that the sending terminal is on the white list, then the subsequent steps are carried out;

[0088] Further, the text splitting module is used to split the actual original text, and the specific splitting process is as follows:

[0089] Obtain the actual original text, and divide the actual original text into n actual original short sentences with punctuation marks as separators, where n is the number of the actual original short sentences, n = 1, 2,..., z, and z is the maximum value of the number;

[0090] Perform word segmentation on the actual original short sentences through stop words and word types to obtain m word segmentation results corresponding to the actual original short sentences;

[0091] Among them, stop words refer to some common words that are automatically filtered out in natural language processing and data mining to save storage space and improve processing efficiency. Stop words are all manually input rather than automatically generated; common stop words include "of", "already", "in", "and", "is", "but", etc.;

[0092] Exemplarily, if a certain actual original short sentence is: "Strengthen the work of educational counseling for school students", then the word segmentation results are: "strengthen for", "strengthen", "for school", "school", "students", "education", "counseling", "work";

[0093] The text splitting module sends the m word segmentation results corresponding to the n actual original short sentences to the word segmentation recognition module;

[0094] The word segmentation recognition module is used to recognize the violation situations of the word segmentation results corresponding to the actual original short sentences, and the specific recognition process is as follows:

[0095] Obtain the m word segmentation results corresponding to the n actual original short sentences;

[0096] Record the first word segmentation result of the first actual original short sentence as the first recognition word segment;

[0097] Compare the first recognition word segment with

[0098] If the match fails, the next identified word is selected for matching.

[0099] If the match is successful, x original training sentences corresponding to the first identified word segment are obtained; word and text comparison is a prior art and is not specifically limited here;

[0100] Compare the original training sentence with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence. The number of illegal sentences is counted and recorded as the number of illegal sentences.

[0101] The first violation rate DYW of the first recognition segmentation is obtained by dividing the number of violation sentences in the original training segmentation by the total number of original training segmentation sentences; the event "the original training segmentation sentence corresponding to the first recognition segmentation is a violation sentence" is recorded as A;

[0102] Then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A)=DYW;

[0103] The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text; the event "there are illegal short sentences in the training original text" is recorded as event B;

[0104] Then P(B) represents the probability of there being illegal short sentences in the original training text, that is, P(B)=YSW;

[0105] Then calculate P(B|A) as the predicted violation rate of the first identified word;

[0106] Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B);

[0107] In the specific calculation, all the illegal sentences in the original training text can be counted; then, an illegal sentence is randomly selected from all the illegal sentences, and the probability of the selected illegal sentence existing in the first recognition segmentation is calculated;

[0108] Known conditional probability formula: P(A|B)=P(AB) / P(B);

[0109] Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A);

[0110] So we can get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B);

[0111] Among them, P(B|A) is the probability of event B under the premise of event A; P(A|B) is the probability of event A under the premise of event B; P(AB) is the probability of event AB occurring simultaneously;

[0112] Therefore, the predicted violation rate of the first identified segment can be obtained;

[0113] Similarly, the predicted violation rates are calculated for the second to the mth recognized segmentation in the first actual original sentence, and the predicted violation rates for the m recognized segmentations corresponding to the first actual original sentence are obtained;

[0114] The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence;

[0115] Similarly, the predicted violation rate YWGn of n actual original sentences in the actual original text is calculated;

[0116] The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence;

[0117] The word segmentation recognition module sends the violation score value corresponding to the actual original short sentence to the content filtering module;

[0118] The content filtering module is used to filter the original text. The filtering process is as follows:

[0119] Get the violation score value corresponding to the actual original sentence;

[0120] Add up the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text;

[0121] If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0122] If the text violation score of the actual original text is greater than the text violation threshold, the filtering operation is performed:

[0123] Compare the sentence violation score of the original sentence with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no action is taken.

[0124] If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text;

[0125] The content filtering module sends the actual replacement text to the text splitting module, and performs the above operation again to calculate the text violation score value corresponding to the actual replacement text;

[0126] If the text violation score of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display;

[0127] If the text violation score of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal will be added to the blacklist;

[0128] The display terminal is used to receive the actual replacement text after being filtered by the content filtering module and display the actual replacement text.

[0129] In this application, if a corresponding calculation formula appears, the above calculation formula is dimensionless and its numerical calculation is performed. The weight coefficient, proportional coefficient and other coefficients in the formula are set to a result value obtained by quantifying each parameter. Regarding the size of the weight coefficient and the proportional coefficient, as long as it does not affect the proportional relationship between the parameter and the result value, it is acceptable.

[0130] Example 2: Please refer to Figure 3 As shown, based on another concept of the same invention, an AI-based social platform content filtering management method is now proposed, including the following steps:

[0131] Step S101: Verify whether the sending terminal is on the blacklist. If it is on the blacklist, terminate the subsequent operations; if it is on the whitelist, execute the subsequent steps.

[0132] Step S102: splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results;

[0133] Step S103: Identify the violation of the word segmentation result corresponding to the actual original short sentence to obtain the violation score value corresponding to the actual original short sentence;

[0134] In this embodiment, step S103 includes the following sub-steps:

[0135] Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations;

[0136] Step S1032: Match the recognized segmented word with the original training text. If the match is not successful, select the next recognized segmented word for matching. If the match is successful, obtain multiple original training sentences corresponding to the recognized segmented word.

[0137] Step S1033: Compare the original training sentence with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences.

[0138] Step S1034: The violation rate of the recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of original training sentences. The original violation rate of the original training text is obtained by dividing the number of violation sentences in the original training text by the total number of sentences in the original training text. The predicted violation rates of the recognition segmentation corresponding to the multiple actual original sentences are calculated.

[0139] Step S1035 , selecting the largest value among the predicted violation rates as the predicted violation rate of the actual original sentence, and using the predicted violation rate as the violation score value corresponding to the actual original sentence;

[0140] Step S104, the actual original text content filtering, get the corresponding actual replacement text is sent to the display terminal for display or the corresponding sending terminal's sending IP address is added to the blacklist;

[0141] In this embodiment, step S104 includes the following sub-steps:

[0142] Step S1041: Obtain the violation score value corresponding to the actual original short sentence, and add the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text;

[0143] Step S1042: If the text violation score of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0144] Step S1043: If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed. If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text.

[0145] In step S1044, the content filtering module sends the actual replacement text to the text splitting module and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

[0146] Example 3: Please refer to Figure 4 As shown, a structural diagram of a computer device is also proposed. The computer device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor can call logic instructions in the memory to execute an AI-based social platform content filtering management method, which includes: verifying whether the sending terminal is on the blacklist; if so, terminating subsequent operations; if on the whitelist, splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results; identifying the violation of the word segmentation results corresponding to the actual original short sentence, obtaining a violation score value corresponding to the actual original short sentence, performing content filtering on the actual original text, obtaining a corresponding actual replacement text, and sending it to a display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0147] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0148] Example 4: The present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the AI-based social platform content filtering management method provided by the above methods, the method including: verifying whether the sending terminal is on the blacklist, if it is on the blacklist, terminating subsequent operations, if it is on the whitelist, splitting the actual original text into multiple actual original sentences, and then splitting each actual original sentence into multiple word segmentation results, identifying the violations of the word segmentation results corresponding to the actual original sentences, obtaining the violation score value corresponding to the actual original sentences, performing content filtering on the actual original text, obtaining the corresponding actual replacement text and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0149] Example 5: The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned AI-based social platform content filtering management method, the method comprising: verifying whether the sending terminal is on the blacklist, terminating subsequent operations if it is on the blacklist, and if it is on the whitelist, splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results, identifying the violations of the word segmentation results corresponding to the actual original short sentences, obtaining the violation score value corresponding to the actual original short sentence, performing content filtering on the actual original text, obtaining the corresponding actual replacement text, and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0151] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An AI-based social platform content filtering management system, characterized by: It includes data collection module, verification module, sending terminal, text splitting module, word segmentation and recognition module, content filtering module and display terminal; The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module; The verification module is used to verify the terminal status of the sending terminal, and if the sending terminal is on the blacklist, the subsequent steps are interrupted, and if the sending terminal is on the whitelist, the subsequent steps are continued; The text splitting module is used to split the actual original text, obtain multiple actual original short sentences corresponding to the actual original text and corresponding word segmentation results and send them to the word segmentation recognition module; The word segmentation recognition module is used to identify violations of the word segmentation results corresponding to the actual original short sentences, obtain the violation score value of each actual original short sentence and send it to the content filtering module; the content filtering module is used to perform content filtering on the actual original text, obtain the corresponding actual replacement text and send it to the display terminal or add the sending IP address of the corresponding sending terminal to the blacklist; The recognition process of the word segmentation recognition module is as follows: Get m word segmentation results corresponding to n actual original short sentences; The first segmentation result of the first actual original sentence is recorded as the first recognized segmentation; Match the first recognized word with the original training text. If no match is found, select the next recognized word for matching. If the match is successful, obtain x original training sentences corresponding to the first recognition word; Compare the original training sentence with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence. The number of illegal sentences is counted and recorded as the number of illegal sentences. The first violation rate DYW of the first recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of original training sentences; Let the event "the original training sentence corresponding to the first recognition segmentation is an illegal sentence" be recorded as A, then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A)=DYW; The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text; Let the event "there are illegal short sentences in the original training text" be recorded as event B, then P(B) represents the probability that there are illegal short sentences in the original training text, that is, P(B) = YSW; Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B); Calculate P(B|A) as the predicted violation rate of the first identified word; Similarly, the predicted violation rates are calculated for the second to the mth recognized segmentation in the first actual original sentence, and the predicted violation rates for the m recognized segmentations corresponding to the first actual original sentence are obtained; The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence; Similarly, the predicted violation rate YWGn of n actual original short sentences in the actual original text is calculated; The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence; The calculation process of the predicted violation rate corresponding to the first recognition segmentation is equivalent to: Known conditional probability formula: P(A|B)=P(AB) / P(B); Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A); So we get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B); Among them, P(B|A) is the probability of event B under the premise of event A; P(A|B) is the probability of event A under the premise of event B; P(AB) is the probability of event AB occurring simultaneously; The display terminal is used to display the actual replacement text.

2. The AI-based social platform content filtering management system according to claim 1 is characterized in that: The working process of the verification module is as follows: Get the sending IP address of the sending terminal, and then get the IP address corresponding to the blacklist set; Compare the sending IP address of the sending terminal with the IP address corresponding to the blacklist set; If the sending IP address of the sending terminal successfully matches the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the blacklist; If the sending IP address of the sending terminal fails to match the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the whitelist.

3. The AI-based social platform content filtering management system according to claim 2, characterized in that: The splitting process of the text splitting module is as follows: Obtain the actual original text, and divide the actual original text into n actual original short sentences using punctuation marks as delimiters, where n is the number of the actual original short sentence, and n=1, 2, ..., z, where z is the maximum value of the number; The actual original short sentence is segmented by using stop words to obtain m segmentation results corresponding to the actual original short sentence.

4. The AI-based social platform content filtering management system according to claim 1, characterized in that: The working process of the content filtering module is as follows: Get the violation score value corresponding to the actual original sentence; Add up the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text; If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display; If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no action is taken.

5. The AI-based social platform content filtering management system according to claim 4 is characterized in that: The working process of the content filtering module also includes: If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text; The content filtering module sends the actual replacement text to the text splitting module and recalculates the text violation score value corresponding to the actual replacement text; If the text violation score of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display; If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal will be added to the blacklist.

6. An AI-based social platform content filtering and management method, characterized in that: Utilizing the AI-based social platform content filtering management system according to any one of claims 1 to 5, the method comprises: Step S101: Verify whether the sending terminal is on the blacklist. If it is on the blacklist, terminate the subsequent operations. If it is on the whitelist, execute the subsequent steps. Step S102: splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results; Step S103: Identify the violation of the word segmentation result corresponding to the actual original short sentence to obtain the violation score value corresponding to the actual original short sentence; Step S104 , performing content filtering on the actual original text, obtaining a corresponding actual replacement text and sending it to a display terminal for display or adding the sending IP address of the corresponding sending terminal to a blacklist.

7. The AI-based social platform content filtering management method according to claim 6, characterized in that: The step S103 includes the following sub-steps: Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations; Step S1032: Match the recognized segmented word with the original training text. If the match is not successful, select the next recognized segmented word for matching. If the match is successful, obtain multiple original training sentences corresponding to the recognized segmented word. Step S1033: Compare the original training sentence with the illegal words in the illegal word set. If the words in the original training sentence do not match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences. Step S1034: The violation rate of the recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of original training sentences. The original violation rate of the original training text is obtained by dividing the number of violation sentences in the original training text by the total number of sentences in the original training text. The predicted violation rates of the recognition segmentation corresponding to the multiple actual original sentences are calculated. Step S1035 : Select the predicted violation rate with the largest value as the predicted violation rate of the actual original sentence, and use the predicted violation rate as the violation score value corresponding to the actual original sentence.

8. The AI-based social platform content filtering management method according to claim 7, characterized in that: The step S104 includes the following sub-steps: Step S1041: Obtain the violation score value corresponding to the actual original short sentence, and add the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text; Step S1042: If the text violation score of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display; Step S1043: If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed. If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text. In step S1044, the content filtering module sends the actual replacement text to the text splitting module and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the AI-based social platform content filtering management method described in any one of claims 6 to 8 is implemented.

Images