Social platform content filtering management system and method based on AI and storage medium

Abstract

The invention discloses an AI-based social platform content filtering management system and method and a storage medium, relates to the field of social platform content filtering, solves the problem that the content condition in a text cannot be accurately recognized currently, and comprises a verification module, a text splitting module, a word segmentation recognition module and a content filtering module. The verification module is used for verifying the terminal condition of the sending terminal, interrupting the subsequent steps if the sending terminal is a black list, and continuing to execute the subsequent steps if the sending terminal is a white list; the text splitting module splits the actual original text to obtain a plurality of actual original short sentences corresponding to the actual original text and word segmentation results corresponding to the actual original short sentences; the word segmentation recognition module is used for recognizing violation conditions of word segmentation results corresponding to the actual original short sentences to obtain violation score values of the actual original short sentences; and the content filtering module is used for filtering the content of the actual original text. According to the invention, accurate recognition and accurate filtering of the text are realized.

Description

Technical Field

[0001] The present invention relates to the field of social platform content filtering, and in particular to an AI-based social platform content filtering management system, method and storage medium. Background Art

[0002] Social platforms refer to online platforms established through the Internet and mobile communication technologies to promote social interaction and information sharing among users. These platforms provide functions such as user registration, creation of personal profiles, content publishing, viewing other users' content, interactive comments, etc., aiming to connect people and promote the dissemination and exchange of information. As an important medium for information dissemination, social platforms need to comply with the laws and regulations of various countries and regions, including cybersecurity laws and freedom of speech laws, to prevent the dissemination of illegal content on the platform. Therefore, it is necessary to ensure users' sense of security and good experience on the platform. Content filtering management systems can help social platforms identify and filter out illegal content in a timely manner, such as violence, pornography, hate speech, etc.

[0003] However, at present, when filtering Chinese characters on social platforms, fixed filtering standards are usually used to filter out relevant texts. However, this method does not analyze the overall situation of the text, and thus cannot confirm the text situation of each short sentence in the text, and cannot make corresponding responses, which brings a poor experience to users.

[0004] To this end, we propose an AI-based social platform content filtering management system, method and storage medium. Summary of the invention

[0005] The purpose of the present invention is to propose a method for automatically mapping the topology of an electric grid in an important power supply guaranteeing place, so as to solve the problems raised in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The AI-based social platform content filtering management system includes a data collection module, a verification module, a sending terminal, a text segmentation module, a word segmentation recognition module, a content filtering module, and a display terminal;

[0008] The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module;

[0009] The verification module is used to verify the terminal status of the sending terminal, and if the sending terminal is on the blacklist, the subsequent steps are interrupted, and if the sending terminal is on the whitelist, the subsequent steps are continued;

[0010] The text splitting module is used to split the actual original text, obtain multiple actual original short sentences corresponding to the actual original text and corresponding word segmentation results and send them to the word segmentation recognition module;

[0011] The word segmentation recognition module is used to identify the violation of the word segmentation result corresponding to the actual original short sentence, obtain the violation score value of each actual original short sentence and send it to the content filtering module; the content filtering module is used to perform content filtering on the actual original text, obtain the corresponding actual replacement text and send it to the display terminal or add the sending IP address of the corresponding sending terminal to the blacklist;

[0012] The display terminal is used to display the actual replacement text.

[0013] Furthermore, the working process of the verification module is as follows:

[0014] Obtain the sending IP address of the sending terminal, and then obtain the IP address corresponding to the blacklist set;

[0015] Compare the sending IP address of the sending terminal with the IP address corresponding to the blacklist set;

[0016] If the sending IP address of the sending terminal successfully matches the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the blacklist;

[0017] If the sending IP address of the sending terminal fails to match the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the whitelist.

[0018] Furthermore, the splitting process of the text splitting module is as follows:

[0019] Obtaining the actual original text, dividing the actual original text into n actual original short sentences using punctuation marks as delimiters, where n is the number of the actual original short sentence, n=1, 2, ..., z, and z is the maximum value of the number;

[0020] The actual original short sentence is segmented by using stop words to obtain m segmentation results corresponding to the actual original short sentence.

[0021] Furthermore, the recognition process of the word segmentation recognition module is as follows:

[0022] Get m word segmentation results corresponding to n actual original short sentences;

[0023] The first segmentation result of the first actual original sentence is recorded as the first recognized segmentation;

[0024] Match the first recognized word segment with the original training text. If the match fails, select the next recognized word segment for matching.

[0025] If the match is successful, obtain x original training sentences corresponding to the first recognized word segmentation;

[0026] The original training sentence is compared with the illegal words in the illegal word set. If the words in the original training sentence match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences.

[0027] Furthermore, the recognition process of the word segmentation recognition module also includes:

[0028] The first violation rate DYW of the first recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of the original training sentences;

[0029] Let the event "the original training sentence corresponding to the first recognition segmentation is an illegal sentence" be recorded as A, then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A) = DYW;

[0030] The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text;

[0031] The event "there are illegal short sentences in the original training text" is recorded as event B, then P(B) represents the probability of there being illegal short sentences in the original training text, that is, P(B) = YSW;

[0032] Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B);

[0033] Calculate P(B|A) as the predicted violation rate of the first recognized word segment;

[0034] Similarly, the predicted violation rates are calculated for the second to the mth recognized segmented words in the first actual original sentence, and the predicted violation rates of the m recognized segmented words corresponding to the first original sentence are obtained;

[0035] The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence;

[0036] Similarly, the predicted violation rate YWGn of n actual original short sentences in the actual original text is calculated;

[0037] The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence.

[0038] Furthermore, the calculation process of the predicted violation rate corresponding to the first recognition segmentation is equivalent to:

[0039] Known conditional probability formula: P(A|B)=P(AB) / P(B);

[0040] Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A);

[0041] So we get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B);

[0042] Among them, P(B|A) is the probability of event B under the premise that event A occurs; P(A|B) is the probability of event A under the premise that event B occurs; P(AB) is the probability of event AB occurring at the same time.

[0043] Furthermore, the working process of the content filtering module is as follows:

[0044] Obtain the violation score value corresponding to the actual original short sentence;

[0045] Add the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text;

[0046] If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0047] If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original short sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed.

[0048] Furthermore, the working process of the content filtering module also includes:

[0049] If the sentence violation score value is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text;

[0050] The content filtering module sends the actual replacement text to the text splitting module, and recalculates the text violation score value corresponding to the actual replacement text;

[0051] If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display;

[0052] If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

[0053] The second aspect is a social platform content filtering management method based on AI, which includes:

[0054] Step S101, verify whether the sending terminal is on the blacklist, if it is on the blacklist, terminate the subsequent operations, if it is on the whitelist, execute the subsequent steps;

[0055] Step S102, splitting the actual original text into a plurality of actual original short sentences, and then splitting each actual original short sentence into a plurality of word segmentation results;

[0056] Step S103, identifying the violation of the word segmentation result corresponding to the actual original short sentence, and obtaining the violation score value corresponding to the actual original short sentence;

[0057] Step S104, content filtering is performed on the actual original text, and the corresponding actual replacement text is obtained and sent to the display terminal for display or the sending IP address of the corresponding sending terminal is added to the blacklist.

[0058] Furthermore, the step S103 includes the following sub-steps:

[0059] Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations;

[0060] Step S1032, matching the recognized segmented word with the original training text, if the match is not successful, selecting the next recognized segmented word for matching, if the match is successful, obtaining multiple original training sentences corresponding to the recognized segmented word;

[0061] Step S1033, comparing the original training sentence with the illegal words in the illegal word set, if the words in the original training sentence match the illegal words in the illegal word set, no operation is performed, if the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences;

[0062] Step S1034, obtaining the violation rate of the recognition segmentation by dividing the number of violation short sentences in the training original short sentences by the total number of the training original short sentences, obtaining the original violation rate of the training original text by dividing the number of violation short sentences in the training original text by the total number of short sentences in the training original text, and calculating the predicted violation rates of the recognition segmentation corresponding to the multiple actual original short sentences;

[0063] Step S1035 , selecting the predicted violation rate with the largest value as the predicted violation rate of the actual original short sentence, and using the predicted violation rate as the violation score value corresponding to the actual original short sentence.

[0064] Furthermore, the step S104 includes the following sub-steps:

[0065] Step S1041, obtaining the violation score value corresponding to the actual original short sentence, and adding the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text;

[0066] Step S1042: if the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0067] Step S1043: if the text violation score value of the actual original text is greater than the text violation threshold, the sentence violation score value of the actual original short sentence is compared with the sentence violation threshold. When the sentence violation score value is less than or equal to the sentence violation threshold, no operation is performed. When the sentence violation score value is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original short sentence is replaced with "blank", and the actual original text that has been replaced is recorded as the actual replacement text.

[0068] In step S1044, the content filtering module sends the actual replacement text to the text splitting module, and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

[0069] In a third aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements an AI-based social platform content filtering management method.

[0070] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0071] The present invention firstly obtains the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal through a data acquisition module, sends the actual original text corresponding to the sending terminal to a text splitting module and sends the sending IP address of the sending terminal to a verification module, uses the verification module to verify whether the sending terminal is a blacklisted terminal, and terminates the subsequent steps if it is a whitelisted terminal, and then splits the actual original text through the text splitting module to obtain a plurality of actual original short sentences and corresponding word segmentation results corresponding to the actual original text, and then uses the word segmentation recognition module to identify the violation of the word segmentation results corresponding to the actual original short sentences to obtain the violation score value of each actual original short sentence, and the content filtering module performs content filtering on the actual original text in combination with the violation score value, obtains the corresponding actual replacement text and sends it to the display terminal for display or adds the corresponding sending IP address of the sending terminal to the blacklist, and the present invention realizes accurate recognition and precise filtering of text. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0073] Figure 1 It is the overall system block diagram of the present invention.

[0074] Figure 2 It is a flow chart showing the actual original text in the present invention.

[0075] Figure 3 It is a flow chart of the method in the present invention.

[0076] Figure 4 It is a schematic diagram of the structure of the computer device in the present invention. DETAILED DESCRIPTION

[0077] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] Example 1: Please refer to Figure 1-Figure 2 As shown, the technical solution provided by the present invention is: an AI-based social platform content filtering management system, including a data collection module, a verification module, a sending terminal, a database, a text splitting module, a word segmentation recognition module, a content filtering module and a display terminal;

[0079] The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, and send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module, wherein the actual original text is the text content edited and sent by the user, and the sending IP address is specifically a 128-bit address, which is represented by eight groups of hexadecimal numbers separated by colons;

[0080] The database is used to store a blacklist set, original training text and a set of illegal words, send the blacklist set to the verification module, and send the original training text and the set of illegal words to the text splitting module;

[0081] The verification module is used to verify the terminal status of the sending terminal. The working process of the verification module is specifically as follows:

[0082] Get the sending IP address of the sending terminal;

[0083] Then obtain the IP address corresponding to the blacklist set;

[0084] Compare the sending IP address of the sending terminal with the IP addresses corresponding to the blacklist set;

[0085] If the sending IP address of the sending terminal successfully matches the IP addresses corresponding to the blacklist set, the sending terminal is identified as being on the blacklist;

[0086] If the sending IP address of the sending terminal fails to match the IP addresses corresponding to the blacklist set, the sending terminal is identified as being on the white list;

[0087] If the verification module analyzes that the sending terminal is on the blacklist, no subsequent operations are performed; if the verification module analyzes that the sending terminal is on the white list, the subsequent steps are carried out;

[0088] Furthermore, the text splitting module is used to split the actual original text, and the splitting process is as follows:

[0089] Obtain the actual original text, and divide the actual original text into n actual original short sentences with punctuation marks as delimiters, where n is the number of the actual original short sentences, n = 1, 2,..., z, and z is the maximum value of the numbers;

[0090] Perform word segmentation on the actual original short sentences through stop words and part-of-speech to obtain m word segmentation results corresponding to the actual original short sentences;

[0091] Among them, stop words refer to some common words that are automatically filtered out in natural language processing and data mining to save storage space and improve processing efficiency. Stop words are all manually input rather than automatically generated; common stop words include "of", "already", "at", "and", "is", "but", etc.;

[0092] Exemplarily, if a certain actual original short sentence is: "Strengthen the work of educational counseling for school students", the word segmentation results are: "strengthen for", "strengthen", "for school", "school", "students", "education", "counseling", "work";

[0093] The text splitting module sends the m word segmentation results corresponding to the n actual original short sentences to the word segmentation recognition module;

[0094] The word segmentation recognition module is used to identify the violation situations of the word segmentation results corresponding to the actual original short sentences, and the recognition process is as follows:

[0095] Obtain the m word segmentation results corresponding to the n actual original short sentences;

[0096] Record the first word segmentation result of the first actual original short sentence as the first recognition word segmentation;

[0097] Compare the first recognition word segmentation with

[0098] If the match fails, the next recognition word is selected for matching;

[0099] If the match is successful, x original training sentences corresponding to the first identified word segment are obtained; word and text comparison is a prior art and is not specifically limited here;

[0100] The original training sentence is compared with the illegal words in the illegal word set. If the words in the original training sentence match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences;

[0101] The first violation rate DYW of the first recognition segment is obtained by dividing the number of violation sentences in the original training segment by the total number of original training segments; the event "the original training segment corresponding to the first recognition segment is a violation sentence" is recorded as A;

[0102] Then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A) = DYW;

[0103] The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text; the event "there are illegal short sentences in the training original text" is recorded as event B;

[0104] Then P(B) represents the probability of the existence of illegal short sentences in the original training text, that is, P(B) = YSW;

[0105] Then the calculated P(B|A) is the predicted violation rate of the first recognized word segment;

[0106] Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B);

[0107] In the specific calculation, all the illegal sentences in the original training text can be counted; then, one illegal sentence is randomly selected from all the illegal sentences, and the probability that the selected illegal sentence exists in the first recognition segmentation is calculated;

[0108] Known conditional probability formula: P(A|B)=P(AB) / P(B);

[0109] Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A);

[0110] So we can get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B);

[0111] Among them, P(B|A) is the probability of event B given the occurrence of event A; P(A|B) is the probability of event A given the occurrence of event B; P(AB) is the probability of event AB occurring simultaneously;

[0112] Therefore, the predicted violation rate of the first identified segment can be obtained;

[0113] Similarly, the predicted violation rates are calculated for the second to the mth recognized segmented words in the first actual original sentence, and the predicted violation rates of the m recognized segmented words corresponding to the first original sentence are obtained;

[0114] The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence;

[0115] Similarly, the predicted violation rate YWGn of n actual original short sentences in the actual original text is calculated;

[0116] The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence;

[0117] The word segmentation recognition module sends the violation score value corresponding to the actual original short sentence to the content filtering module;

[0118] The content filtering module is used to filter the content of the actual original text. The filtering process is as follows:

[0119] Obtain the violation score value corresponding to the actual original short sentence;

[0120] Add the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text;

[0121] If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0122] If the text violation score of the actual original text is greater than the text violation threshold, the filtering operation is performed:

[0123] Compare the sentence violation score of the actual original sentence with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed.

[0124] If the sentence violation score value is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text;

[0125] The content filtering module sends the actual replacement text to the text splitting module, and performs the above operation again to calculate the text violation score value corresponding to the actual replacement text;

[0126] If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display;

[0127] If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist;

[0128] The display terminal is used to receive the actual replacement text after being filtered by the content filtering module, and to display the actual replacement text.

[0129] In this application, if corresponding calculation formulas appear, the above calculation formulas are all dimensionless and take their numerical calculations. The weight coefficients, proportional coefficients and other coefficients in the formulas are set to a result value obtained by quantifying each parameter. The size of the weight coefficient and the proportional coefficient can be determined as long as it does not affect the proportional relationship between the parameter and the result value.

[0130] Example 2: Please refer to Figure 3 As shown, based on another concept of the same invention, a social platform content filtering management method based on AI is now proposed, comprising the following steps:

[0131] Step S101, verify whether the sending terminal is on the blacklist, if it is on the blacklist, terminate the subsequent operations; if it is on the whitelist, execute the subsequent steps;

[0132] Step S102, splitting the actual original text into a plurality of actual original short sentences, and then splitting each actual original short sentence into a plurality of word segmentation results;

[0133] Step S103, identifying the violation of the word segmentation result corresponding to the actual original short sentence, and obtaining the violation score value corresponding to the actual original short sentence;

[0134] In this embodiment, step S103 includes the following sub-steps:

[0135] Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations;

[0136] Step S1032, matching the recognized segmented word with the original training text, if the match is not successful, selecting the next recognized segmented word for matching, if the match is successful, obtaining multiple original training sentences corresponding to the recognized segmented word;

[0137] Step S1033, comparing the original training sentence with the illegal words in the illegal word set, if the words in the original training sentence match the illegal words in the illegal word set, no operation is performed, if the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences;

[0138] Step S1034, obtaining the violation rate of the recognition segmentation by dividing the number of violation short sentences in the training original short sentences by the total number of the training original short sentences, obtaining the original violation rate of the training original text by dividing the number of violation short sentences in the training original text by the total number of short sentences in the training original text, and calculating the predicted violation rates of the recognition segmentation corresponding to the multiple actual original short sentences;

[0139] Step S1035, selecting the largest value among the predicted violation rates as the predicted violation rate of the actual original short sentence, and using the predicted violation rate as the violation score value corresponding to the actual original short sentence;

[0140] Step S104, filtering the actual original text, obtaining the corresponding actual replacement text and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist;

[0141] In this embodiment, step S104 includes the following sub-steps:

[0142] Step S1041, obtaining the violation score value corresponding to the actual original short sentence, and adding the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text;

[0143] Step S1042: if the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display;

[0144] Step S1043: if the text violation score value of the actual original text is greater than the text violation threshold, the sentence violation score value of the actual original short sentence is compared with the sentence violation threshold. When the sentence violation score value is less than or equal to the sentence violation threshold, no operation is performed. When the sentence violation score value is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original short sentence is replaced with "blank", and the actual original text that has been replaced is recorded as the actual replacement text.

[0145] In step S1044, the content filtering module sends the actual replacement text to the text splitting module, and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

[0146] Example 3: Please refer to Figure 4 As shown, a structural schematic diagram of a computer device is also proposed, and the computer device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete mutual communication through the communication bus. The processor can call the logic instructions in the memory to execute the AI-based social platform content filtering management method, the method comprising: verifying whether the sending terminal is on the blacklist, if it is on the blacklist, terminating the subsequent operation, if it is on the whitelist, splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results, identifying the violation of the actual original short sentence corresponding to the word segmentation result, obtaining the violation score value corresponding to the actual original short sentence, filtering the actual original text, obtaining the corresponding actual replacement text and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0147] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0148] Embodiment 4: The present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the AI-based social platform content filtering management method provided by the above methods, the method including: verifying whether the sending terminal is on the blacklist, if it is on the blacklist, terminating subsequent operations, if it is on the whitelist, splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results, identifying the violation of the word segmentation results corresponding to the actual original short sentences, obtaining the violation score value corresponding to the actual original short sentences, performing content filtering on the actual original text, obtaining the corresponding actual replacement text and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0149] Embodiment 5: The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned AI-based social platform content filtering management methods, the methods comprising: verifying whether the sending terminal is on the blacklist, terminating subsequent operations if it is on the blacklist, and if it is on the whitelist, splitting the actual original text into multiple actual original short sentences, and then splitting each actual original short sentence into multiple word segmentation results, identifying the violation of the word segmentation results corresponding to the actual original short sentences, obtaining the violation score value corresponding to the actual original short sentences, performing content filtering on the actual original text, obtaining the corresponding actual replacement text and sending it to the display terminal for display or adding the sending IP address of the corresponding sending terminal to the blacklist.

[0150] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An AI-based social platform content filtering management system, characterized by: It includes data collection module, verification module, sending terminal, text segmentation module, word segmentation recognition module, content filtering module and display terminal; The data acquisition module is used to obtain the actual original text sent by the sending terminal to the server and the sending IP address of the sending terminal, send the actual original text corresponding to the sending terminal to the text splitting module and send the sending IP address of the sending terminal to the verification module; The verification module is used to verify the terminal status of the sending terminal, and if the sending terminal is on the blacklist, the subsequent steps are interrupted, and if the sending terminal is on the whitelist, the subsequent steps are continued; The text splitting module is used to split the actual original text, obtain multiple actual original short sentences corresponding to the actual original text and corresponding word segmentation results and send them to the word segmentation recognition module; The word segmentation recognition module is used to identify the violation of the word segmentation result corresponding to the actual original short sentence, obtain the violation score value of each actual original short sentence and send it to the content filtering module; the content filtering module is used to perform content filtering on the actual original text, obtain the corresponding actual replacement text and send it to the display terminal or add the sending IP address of the corresponding sending terminal to the blacklist; The display terminal is used to display the actual replacement text.

2. The AI-based social platform content filtering management system according to claim 1 is characterized in that: The working process of the verification module is as follows: Obtain the sending IP address of the sending terminal, and then obtain the IP address corresponding to the blacklist set; Compare the sending IP address of the sending terminal with the IP address corresponding to the blacklist set; If the sending IP address of the sending terminal successfully matches the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the blacklist; If the sending IP address of the sending terminal fails to match the IP address corresponding to the blacklist set, the sending terminal is deemed to be on the whitelist.

3. The AI-based social platform content filtering management system according to claim 2 is characterized in that: The splitting process of the text splitting module is as follows: Obtaining the actual original text, dividing the actual original text into n actual original short sentences using punctuation marks as delimiters, where n is the number of the actual original short sentence, n=1, 2, ..., z, and z is the maximum value of the number; The actual original short sentence is segmented by using stop words to obtain m segmentation results corresponding to the actual original short sentence.

4. The AI-based social platform content filtering management system according to claim 3 is characterized in that: The recognition process of the word segmentation recognition module is as follows: Get m word segmentation results corresponding to n actual original short sentences; The first segmentation result of the first actual original sentence is recorded as the first recognized segmentation; Match the first recognized word segment with the original training text. If the match fails, select the next recognized word segment for matching. If the match is successful, obtain x original training sentences corresponding to the first recognized word segmentation; The original training sentence is compared with the illegal words in the illegal word set. If the words in the original training sentence match the illegal words in the illegal word set, no operation is performed. If the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences.

5. The AI-based social platform content filtering management system according to claim 4 is characterized in that: The recognition process of the word segmentation recognition module also includes: The first violation rate DYW of the first recognition segmentation is obtained by dividing the number of violation sentences in the original training sentences by the total number of the original training sentences; Let the event "the original training sentence corresponding to the first recognition segmentation is an illegal sentence" be recorded as A, then P(A) represents the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence, that is, P(A) = DYW; The original violation rate YSW of the training original text is obtained by dividing the number of illegal short sentences in the training original text by the total number of short sentences in the training original text; The event "there are illegal short sentences in the original training text" is recorded as event B, then P(B) represents the probability that there are illegal short sentences in the original training text, that is, P(B) = YSW; Calculate the probability that the original training sentence corresponding to the first recognition segmentation is an illegal sentence under the premise that there are illegal sentences in the original training text and record it as P(A|B); Calculate P(B|A) as the predicted violation rate of the first recognized word segment; Similarly, the predicted violation rates are calculated for the second to the mth recognized segmented words in the first actual original sentence, and the predicted violation rates of the m recognized segmented words corresponding to the first original sentence are obtained; The largest value among the m predicted violation rates is selected as the predicted violation rate of the first actual original short sentence; Similarly, the predicted violation rate YWGn of n actual original short sentences in the actual original text is calculated; The predicted violation rate is taken as the violation score value corresponding to the actual original short sentence.

6. The AI-based social platform content filtering management system according to claim 5, characterized in that: The calculation process of the predicted violation rate corresponding to the first recognition segmentation is equivalent to: Known conditional probability formula: P(A|B)=P(AB) / P(B); Similarly, P(B|A)=P(AB) / P(A); therefore, P(AB)=P(A)×P(B|A); So we get: P(B|A)=P(AB) / P(B)=P(A)×P(A|B) / P(B); Among them, P(B|A) is the probability of event B under the premise that event A occurs; P(A|B) is the probability of event A under the premise that event B occurs; P(AB) is the probability of event AB occurring at the same time.

7. The AI-based social platform content filtering management system according to claim 1, characterized in that: The working process of the content filtering module is as follows: Obtain the violation score value corresponding to the actual original short sentence; Add the violation score values ​​of n actual original short sentences to obtain the text violation score value of the actual original text; If the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display; If the text violation score of the actual original text is greater than the text violation threshold, the sentence violation score of the actual original short sentence is compared with the sentence violation threshold. If the sentence violation score is less than or equal to the sentence violation threshold, no operation is performed.

8. The AI-based social platform content filtering management system according to claim 7, characterized in that: The working process of the content filtering module also includes: If the sentence violation score is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text after the replacement is recorded as the actual replacement text; The content filtering module sends the actual replacement text to the text splitting module, and recalculates the text violation score value corresponding to the actual replacement text; If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display; If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

9. The AI-based social platform content filtering management method is characterized by: Using the AI-based social platform content filtering management system according to any one of claims 1 to 8, the method comprises: Step S101, verify whether the sending terminal is on the blacklist, if it is on the blacklist, terminate the subsequent operations, if it is on the whitelist, execute the subsequent steps; Step S102, splitting the actual original text into a plurality of actual original short sentences, and then splitting each actual original short sentence into a plurality of word segmentation results; Step S103, identifying the violation of the word segmentation result corresponding to the actual original short sentence, and obtaining the violation score value corresponding to the actual original short sentence; Step S104, content filtering is performed on the actual original text, and the corresponding actual replacement text is obtained and sent to the display terminal for display or the sending IP address of the corresponding sending terminal is added to the blacklist.

10. The AI-based social platform content filtering management method according to claim 9, characterized in that: The step S103 includes the following sub-steps: Step S1031, obtaining the word segmentation results corresponding to a plurality of actual original short sentences, and recording the word segmentation results of the actual original short sentences as recognized word segmentations; Step S1032, matching the recognized segmented word with the original training text, if the match is not successful, selecting the next recognized segmented word for matching, if the match is successful, obtaining multiple original training sentences corresponding to the recognized segmented word; Step S1033, comparing the original training sentence with the illegal words in the illegal word set, if the words in the original training sentence match the illegal words in the illegal word set, no operation is performed, if the words in the original training sentence match the illegal words in the illegal word set, the original training sentence is recorded as an illegal sentence, and the number of illegal sentences is counted and recorded as the number of illegal sentences; Step S1034, obtaining the violation rate of the recognition segmentation by dividing the number of violation short sentences in the training original short sentences by the total number of the training original short sentences, obtaining the original violation rate of the training original text by dividing the number of violation short sentences in the training original text by the total number of short sentences in the training original text, and calculating the predicted violation rates of the recognition segmentation corresponding to the multiple actual original short sentences; Step S1035 , selecting the predicted violation rate with the largest value as the predicted violation rate of the actual original short sentence, and using the predicted violation rate as the violation score value corresponding to the actual original short sentence.

11. The AI-based social platform content filtering management method according to claim 9, characterized in that: The step S104 includes the following sub-steps: Step S1041, obtaining the violation score value corresponding to the actual original short sentence, and adding the violation score values ​​of multiple actual original short sentences to obtain the text violation score value of the actual original text; Step S1042: if the text violation score value of the actual original text is less than or equal to the text violation threshold, the actual original text is sent to the display terminal for display; Step S1043, if the text violation score value of the actual original text is greater than the text violation threshold, the sentence violation score value of the actual original sentence is compared with the sentence violation threshold. When the sentence violation score value is less than or equal to the sentence violation threshold, no operation is performed. When the sentence violation score value is greater than the sentence violation threshold, the corresponding violation identification segmentation in the actual original sentence is replaced with "blank", and the actual original text that has been replaced is recorded as the actual replacement text; In step S1044, the content filtering module sends the actual replacement text to the text splitting module, and recalculates the text violation score value corresponding to the actual replacement text. If the text violation score value of the actual replacement text is less than or equal to the text violation threshold, the actual replacement text is sent to the display terminal for display. If the text violation score value of the actual replacement text is greater than the text violation threshold, the sending IP address of the corresponding sending terminal is added to the blacklist.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the AI-based social platform content filtering management method described in claims 9-11 is implemented.

Images