Self-adaptive content security AI retrieval control system and method

Abstract

The invention discloses a self-adaptive content security AI retrieval control system and method, and relates to the technical field of content security, and the method specifically comprises the following steps: step 1, collecting unsafe web text contents, and establishing an AI retrieval comparison database; 2, determining a word structure belonging to the content abnormal text of the to-be-retrieved data, and evaluating the content security of the to-be-retrieved data; 3, analyzing the text content of which the content security is influenced, and determining the correlation degree between the text content and different types of abnormal word structures; according to the method, whether the to-be-processed text content is affected by the content abnormal text or not is judged through comparison, the text content with the content safety affected and needing to be controlled is rapidly and accurately screened out, the content safety of the to-be-processed text content is improved, and the content safety of the to-be-processed text content is improved. And the text quality and the text content security can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of content security, and in particular relates to an adaptive content security AI retrieval control system and method. Background Art

[0002] The background technologies of content security AI retrieval cover multiple fields such as natural language processing, computer vision, machine learning, deep learning, data processing and privacy protection. The comprehensive application of these technologies provides an efficient and intelligent solution for content security supervision, which helps to maintain the health and security of cyberspace.

[0003] However, the quality of data on the Internet varies greatly, with a large amount of noise and redundant information. Data from different sources and of different types have different characteristics and formats, and flexible data processing processes need to be designed to deal with them. How to extract valuable information from these data and conduct effective analysis and processing? The text classification module in natural language processing technology can automatically divide text into different categories, such as normal text, sensitive text, illegal text, etc., which helps to quickly identify potential abnormal text content. Semantic understanding technology enables AI to deeply understand the meaning and context of the text, so as to more accurately judge whether the text contains abnormal content. How to help AI retrieve and identify potential content security risk points in the text by analyzing the keywords, phrases, sentence structures and the logical relationships between them is a problem that needs to be solved at present. Summary of the invention

[0004] The purpose of the present invention is to provide an adaptive content security AI retrieval control system and method to solve the technical problem in the prior art that it is difficult to identify potential content security risk points in text by analyzing words.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: Step 1: Collect unsafe network text content and establish an AI retrieval and comparison database; collect a large amount of network text content data that has been marked as unsafe, record it as abnormal content text, split the text content belonging to abnormal content text, extract the words in the text content, record it as word structure, and divide the word structure into a single word structure and a combined word structure, where a combined word structure refers to a word structure in the form of a word or phrase composed of multiple single word structures and having actual meaning, delete repeated word structures and word structures without actual meaning, and classify the abnormal content text based on the sensitivity, potential adverse impact risk, and negative emotional tendency of the abnormal content text. Define the single word structure or combined word structure that is prohibited from dissemination by organizations, industries or laws as first-level abnormal content text; define the single word structure or combined word structure that has an adverse impact risk as a second-level abnormal content text; define the single word structure or combined word structure that covers up or fabricates facts as a third-level abnormal content text; define the single word structure or combined word structure with disordered word order, mixed Chinese and English, and special characters as a fourth-level abnormal content text; classify and store abnormal content texts of different levels to establish a database for AI retrieval and comparison.

[0006] Step 2: Determine the word structure of the text with abnormal content in the data to be retrieved, and assess the content security of the data to be retrieved; content data includes: text content data, image content data and sound content data. Combined with computer vision technology, speech recognition technology and natural language processing technology in AI technology, the image content data and sound content data are uniformly converted into text content data, and the text content is segmented to ensure that the text content is segmented without destroying the text words and sentence structure. The segmentation method includes: dividing according to the text content structure, dividing according to the number of text words, and dividing according to the text content type, searching and processing the segmented text content data, extracting the word structure in the text content data, and comparing the data in the AI ​​search comparison database to determine the word structure of the text with abnormal content, record it as an abnormal word structure, determine the data to be retrieved, and for the data to be retrieved in non-text format, convert the non-text format data to be retrieved into text content data, perform semantic analysis on the text content, and confirm that the text content structure is complete.

[0007] Perform AI search and comparison on the text content with complete structure, filter out the abnormal text in the data to be retrieved, mark the number of occurrences of each abnormal word structure in the abnormal text, determine the level of each abnormal word structure in the abnormal text, and obtain the abnormal word structure set, which is expressed as , where G(A) represents the set of abnormal word structures contained in the text content A, and x represents the xth abnormal word structure in the set. Represents a set of abnormal word structures belonging to the first-level abnormal content text, Represents a set of abnormal word structures belonging to abnormal text of secondary content, Represents a set of abnormal word structures belonging to the third-level abnormal content text, Represents a set of abnormal word structures belonging to level 4 content abnormal text.

[0008] According to the type of data to be retrieved and the content security requirements, determine the degree of influence of abnormal content texts at different levels on content security. By determining the influence of abnormal content texts on the data to be retrieved, the text content security value of the data to be retrieved is obtained. The specific formula is as follows: ; in, represents the content safety value of text content A, U represents the number of words in text content A, u represents the number of words in abnormal content in text content A, i represents the level of abnormal content, represents the xth abnormal word structure in the abnormal word structure set of level i, Indicates abnormal word structure The number of occurrences, Indicates abnormal word structure The number of words, Indicates the i-th level content abnormal text weight value set according to text content A.

[0009] A content security threshold is set according to the text content type and the text content security requirements. When the content security value of the text content of the data to be retrieved is greater than or equal to the content security threshold, it is determined that the content security of the data to be retrieved is affected by the abnormal content text.

[0010] Step 3: Analyze the text content whose content security is affected, and determine the degree of association between the text content and different types of abnormal word structures; for the text content whose content security is affected by the abnormal content text, extract all the abnormal content texts in the text content, decompose the abnormal content text into texts composed of multiple abnormal word structures, retain each abnormal word structure content, delete repeated abnormal word structure content, and determine the degree of association between the text content and different types of abnormal word structures based on the abnormal word structure set contained in the text content data. The specific calculation formula is as follows: ; in, Indicates abnormal word structure The degree of correlation with the text content is recorded as abnormal correlation. Indicates abnormal word structure The number of occurrences, represents the xth abnormal word structure in the abnormal word structure set of the i-th level, M represents the sum of the number of occurrences of all abnormal word structures, and N represents the total number of paragraphs obtained by segmenting the text content. Indicates that it contains abnormal word structure The number of paragraphs, e represents the base of the logarithm of natural numbers, and the degree of association between each abnormal word structure and the text content is determined by the formula. According to the level of the abnormal word structure, the proportion of the degree of association between each abnormal word structure level and the text content is determined. The specific formula is as follows: ; in, Indicates the number of abnormal word structures at level i, The weight coefficient is determined according to the specific situation and needs, and is usually jointly formulated and confirmed by professionals or relevant stakeholders. , , , The formula is used to determine the correlation ratio between abnormal word structures at each level and text content, and the abnormal word structure level that needs to be controlled is screened according to the demand and the correlation ratio.

[0011] Step 4: Determine the abnormal word structures that need to be controlled to enhance the content security of the text content; determine the abnormal word structures that need to be controlled based on the proportion of the correlation between the abnormal word structure level and the text content, further clarify the specific location and content of the words, phrases, and sentences that need to be modified, and use the existing deep learning text generation model to perform control operations based on actual needs while ensuring content security, and clarify whether to change the overall structure or logical order of the text. The control operations include: deletion, which refers to removing unnecessary modifiers or repeated information; replacement, which refers to replacing the original word with a synonym that is similar or identical to the original meaning; optimization, which refers to replacing nouns, verbs, adjectives, and other parts of speech as needed. Make appropriate adjustments, verify the text content that has been modified by the control operation, ensure that the sentence structure of the modified text content is complete and the grammar is correct, obtain the verified text content, calculate the content safety value of the text content, determine whether the text content is affected by the abnormal content text, output the text content that has been determined to be not affected by the abnormal content text, and for the text content that has been determined to be affected by the abnormal content text, determine the abnormal word structure that needs to be controlled and perform the control operation by calculating the proportion of the correlation between the abnormal word structure level and the text content, and repeat this step until the text content is determined to be not affected by the abnormal content text, and then output the text content.

[0012] The present invention also provides an adaptive content security AI retrieval control system, which specifically includes: The AI ​​retrieval and comparison database module is used to collect network text content data that has been marked as unsafe, record it as abnormal content text, classify the abnormal content text based on its sensitivity, potential adverse impact risk, and negative emotional tendency, and classify and store abnormal content texts of different levels for AI retrieval and comparison; The content security value calculation module converts non-text content data into text content data, performs AI search and comparison on the text content, determines the word structure of the text with abnormal content, records it as abnormal word structure, determines the impact of the abnormal content text on the data to be retrieved through formula calculation, and evaluates the text content security value of the data to be retrieved; A correlation degree determination module determines the correlation degree between the text content and different abnormal word structures based on the abnormal word structure set contained in the text content data, determines the correlation degree ratio of each abnormal word structure level to the text content according to the level of the abnormal word structure, and selects the abnormal word structure level that needs to be controlled according to the demand and the correlation degree ratio; The abnormal word structure control module clarifies the specific location and content of the characters, words, and sentences that need to be modified, performs control operations based on actual needs while ensuring content security, verifies the text content modified by the control operation, and determines whether the text content is affected by the abnormal text content. By repeatedly calculating the proportion of the correlation between the abnormal word structure level and the text content, the abnormal word structure that needs to be controlled is determined and control operations are performed until the text content is determined not to be affected by the abnormal text content and the text content is output.

[0013] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. The present invention clarifies the texts with abnormal content and divides the abnormal content texts into levels. According to the text type, the influence of abnormal texts at each level on the overall data set or specific analysis tasks is determined, and a weight value is assigned to each level. According to the weight and number of abnormal text levels, the overall data set or specific text set is evaluated to see whether it is significantly affected by abnormal content texts, thereby effectively ensuring the quality of text data and the accuracy of analysis results. 2. The present invention converts non-text content into text content for processing, sets a content security value, extracts abnormal word structures in the text content to be processed, calculates the content security value of the text content to be processed according to the number of occurrences, quantity and level of the abnormal word structures, sets a threshold value according to the content security requirements of the text type, and determines whether the text content to be processed is affected by the abnormal text through comparison, so as to quickly and accurately screen out the text content whose content security is affected and needs to be controlled; 3. The present invention determines the degree of association between text content and different abnormal word structures, as well as the proportion of the degree of association between each abnormal word structure level and text content. It screens the abnormal word structure level that needs to be controlled according to the text content type, clarifies the specific position and content of the characters, words, and sentences that need to be modified, facilitates the control and adjustment of the abnormal word structure, and helps to improve the text quality and text content security. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A step diagram of an adaptive content security AI retrieval control method is shown; Figure 2 A flow chart of a method for controlling abnormal text content is shown; Figure 3 A block diagram of an adaptive content security AI retrieval control system is shown. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1, as Figure 1 , Figure 2 As shown, the present embodiment provides an adaptive content security AI retrieval control method, which specifically includes the following steps: Step 1: Collect unsafe online text content and establish an AI retrieval and comparison database; collect a large amount of network text content data that has been marked as unsafe, record it as abnormal content text, split the text content belonging to abnormal content text, extract the words in the text content, record it as word structure, and the word structure is divided into a single word structure and a combined word structure, where the combined word structure refers to a word structure composed of multiple single word structures and has a practical meaning in the form of words or phrases, delete duplicate word structures and word structures without practical meaning, and comprehensively analyze the sensitivity, potential adverse impact risks, and negative emotional tendencies of abnormal content texts. In this classification, single word structures or combined word structures that are prohibited from being disseminated by organizations, industries or laws are defined as first-level abnormal content text; single word structures or combined word structures that have the risk of adverse effects are defined as second-level abnormal content text, such as text content that is low-level, vulgar, or rude; single word structures or combined word structures that cover up or fabricate facts are defined as third-level abnormal content text; single word structures or combined word structures with disordered word order, a mixture of Chinese and English, and the presence of special characters are defined as fourth-level abnormal content text. Abnormal content texts of different levels are classified and stored to establish a database for AI retrieval and comparison.

[0018] Step 2: Determine the word structure of the text with abnormal content in the data to be retrieved, and assess the content security of the data to be retrieved; content data includes: text content data, image content data and sound content data. Combined with computer vision technology, speech recognition technology and natural language processing technology in AI technology, the image content data and sound content data are uniformly converted into text content data, and the text content is segmented to ensure that the text content is segmented without destroying the text words and sentence structure. The segmentation method includes: dividing according to the text content structure, dividing according to the number of text words, and dividing according to the text content type, searching and processing the segmented text content data, extracting the word structure in the text content data, and comparing the data in the AI ​​search comparison database to determine the word structure of the text with abnormal content, record it as an abnormal word structure, determine the data to be retrieved, and for the data to be retrieved in non-text format, convert the non-text format data to be retrieved into text content data, perform semantic analysis on the text content, and confirm that the text content structure is complete.

[0019] Perform AI search and comparison on the text content with complete structure, filter out the abnormal text in the data to be retrieved, mark the number of occurrences of each abnormal word structure in the abnormal text, determine the level of each abnormal word structure in the abnormal text, and obtain the abnormal word structure set, which is expressed as , where G(A) represents the set of abnormal word structures contained in the text content A, and x represents the xth abnormal word structure in the set. Represents a set of abnormal word structures belonging to the first-level abnormal content text, Represents a set of abnormal word structures belonging to abnormal text of secondary content, Represents a set of abnormal word structures belonging to the third-level abnormal content text, Represents a set of abnormal word structures belonging to level 4 content abnormal text.

[0020] According to the type of data to be retrieved and the content security requirements, determine the degree of influence of abnormal content texts at different levels on content security. By determining the influence of abnormal content texts on the data to be retrieved, the text content security value of the data to be retrieved is obtained. The specific formula is as follows: ; in, represents the content safety value of text content A, U represents the number of words in text content A, u represents the number of words in abnormal content in text content A, i represents the level of abnormal content, represents the xth abnormal word structure in the abnormal word structure set of level i, Indicates abnormal word structure The number of occurrences, Indicates abnormal word structure The number of words, It indicates the abnormal text weight value of the i-th level content set according to the text content A. For example, the text content A belongs to the news content type. According to the requirements, the abnormal text weight value of the first level content in the text content is set to 1.8, the abnormal text weight value of the second level content is set to 1.5, the abnormal text weight value of the third level content is set to 1.3, and the abnormal text weight value of the fourth level content is set to 1.2.

[0021] A content security threshold is set according to the text content type and the text content security requirements. When the content security value of the text content of the data to be retrieved is greater than or equal to the content security threshold, it is determined that the content security of the data to be retrieved is affected by the abnormal content text.

[0022] Step 3: Analyze the text content whose content security is affected, and determine the degree of association between the text content and different types of abnormal word structures; for the text content whose content security is affected by the abnormal content text, extract all the abnormal content texts in the text content, decompose the abnormal content text into texts composed of multiple abnormal word structures, retain each abnormal word structure content, delete repeated abnormal word structure content, and determine the degree of association between the text content and different types of abnormal word structures based on the abnormal word structure set contained in the text content data. The specific calculation formula is as follows: ; in, Indicates abnormal word structure The degree of correlation with the text content is recorded as abnormal correlation. Indicates abnormal word structure The number of occurrences, represents the xth abnormal word structure in the abnormal word structure set of the i-th level, M represents the sum of the number of occurrences of all abnormal word structures, and N represents the total number of paragraphs obtained by segmenting the text content. Indicates that it contains abnormal word structure The number of paragraphs, e represents the base of the logarithm of natural numbers, and the degree of association between each abnormal word structure and the text content is determined by the formula. According to the level of the abnormal word structure, the proportion of the degree of association between each abnormal word structure level and the text content is determined. The specific formula is as follows: ; in, Indicates the number of abnormal word structures at level i, The weight coefficient is determined according to the specific situation and needs, and is usually jointly formulated and confirmed by professionals or relevant stakeholders. , , , , the formula is used to determine the correlation ratio between abnormal word structures at each level and text content, and the abnormal word structure level that needs to be controlled is screened according to the demand and correlation ratio. For example: for text content A1, the correlation ratio between abnormal word structures at each level and text content is the same, so it is necessary to control the abnormal word structure at each level. For text content A2, except for the abnormal word structure at the third level, the correlation ratio between abnormal word structures at other levels and text content is less than 5%, so it is only necessary to control the abnormal word structure at the third level.

[0023] Step 4: Determine the abnormal word structures that need to be controlled to enhance the content security of the text content; based on the proportion of the abnormal word structure level and the degree of correlation between the text content, determine the abnormal word structure that needs to be controlled, further clarify the specific location and content of the words, phrases, and sentences that need to be modified, and use existing deep learning text generation models, such as GPT-3 and GPT-4, to perform control operations based on actual needs while ensuring content security, and clarify whether to change the overall structure or logical order of the text. Control operations include: deletion, which refers to removing unnecessary modifiers or repeated information; replacement, which refers to replacing the original word with a synonym that is similar or identical to the original meaning; optimization, which refers to replacing nouns and verbs as needed. Appropriate adjustments are made to parts of speech such as words and adjectives, and the text content modified by the control operation is verified to ensure that the sentence structure and grammar of the modified text content are complete, and the verified text content is obtained, and the content safety value of the text content is calculated to determine whether the text content is affected by the abnormal content text, and the text content that is judged not to be affected by the abnormal content text is output. For the text content that is judged to be affected by the abnormal content text, the abnormal word structure that needs to be controlled is determined by calculating the proportion of the correlation between the abnormal word structure level and the text content, and the control operation is performed, and this step is repeated until the text content is judged not to be affected by the abnormal content text, and the text content is output.

[0024] Embodiment 2, as Figure 3 The embodiment provides an adaptive content security AI retrieval control system, which specifically includes the following contents: AI retrieval and comparison database module: collect a large amount of network text content data that has been marked as unsafe and record it as abnormal content text, split the text content belonging to abnormal content text, extract words in the text content and record them as word structures, which are divided into single word structures and combined word structures, delete repeated word structures and word structures without practical meaning, and classify abnormal content texts based on their sensitivity, potential adverse impact risks, and negative emotional tendencies. Single word structures or combined word structures that are prohibited from dissemination by organizations, industries or laws are defined as first-level abnormal content texts; single word structures or combined word structures that have adverse impact risks are defined as second-level abnormal content texts; single word structures or combined word structures that cover up or fabricate facts are defined as third-level abnormal content texts; single word structures or combined word structures with disordered word order, mixed Chinese and English, and the presence of special characters are defined as fourth-level abnormal content texts, and abnormal content texts of different levels are classified and stored.

[0025] Content security value calculation module: Combining computer vision technology, speech recognition technology and natural language processing technology in AI technology, non-text content data is uniformly converted into text content data, the text content is segmented, the segmented text content data is retrieved, and the word structure in the text content data is extracted. By comparing the data in the AI ​​retrieval comparison database, the word structure of the text with abnormal content is determined and recorded as abnormal word structure, and the data to be retrieved is determined. For the data to be retrieved in non-text format, the non-text format data to be retrieved is uniformly converted into text content data, and the text content is semantically analyzed to confirm that the text content structure is complete.

[0026] Perform AI retrieval and comparison on text content with complete structure, screen out abnormal texts in the data to be retrieved, mark the number of occurrences of each abnormal word structure in the abnormal text, determine the level of each abnormal word structure in the abnormal text, and obtain the abnormal word structure set, which is expressed as , where G(A) represents the set of abnormal word structures contained in the text content A, and x represents the xth abnormal word structure in the set. Represents a set of abnormal word structures belonging to the first-level abnormal content text, Represents a set of abnormal word structures belonging to abnormal text of secondary content, Represents a set of abnormal word structures belonging to the third-level abnormal content text, Represents a set of abnormal word structures belonging to level 4 content abnormal text.

[0027] Filter out the abnormal content text in the data to be retrieved and mark the level. According to the type of data to be retrieved and the content security requirements, determine the impact of abnormal content text at different levels on content security. Determine the impact of abnormal content text on the data to be retrieved through formula calculation and assess the text content security value of the data to be retrieved. The specific formula is as follows: ; in, represents the content safety value of text content A, U represents the number of words in text content A, u represents the number of words in abnormal content in text content A, i represents the level of abnormal content, represents the xth abnormal word structure in the abnormal word structure set of level i, Indicates abnormal word structure The number of occurrences, Indicates abnormal word structure The number of words, It means that a content security threshold is set according to the i-th level content abnormal text weight value set for the text content A. When the content security value of the text content of the data to be retrieved is greater than or equal to the content security threshold, it is judged that the content security of the data to be retrieved is affected by the content abnormal text.

[0028] Association degree determination module: for the text content whose content security is affected by the abnormal content text, all the abnormal content text in the text content is extracted, the abnormal content text is decomposed into text composed of multiple abnormal word structures, each abnormal word structure content is retained, the repeated abnormal word structure content is deleted, and the number of occurrences of each abnormal word structure is marked. Based on the abnormal word structure set contained in the text content data, the association degree between the text content and different abnormal word structures is determined. The specific calculation formula is as follows: ; in, Indicates abnormal word structure The degree of correlation with the text content is recorded as abnormal correlation. Indicates abnormal word structure The number of occurrences, represents the xth abnormal word structure in the abnormal word structure set of the i-th level, M represents the sum of the number of occurrences of all abnormal word structures, and N represents the total number of paragraphs obtained by segmenting the text content. Indicates that it contains abnormal word structure The number of paragraphs, e represents the base of the logarithm of natural numbers, and the degree of association between each abnormal word structure and the text content is determined by the formula. According to the level of the abnormal word structure, the proportion of the degree of association between each abnormal word structure level and the text content is determined. The specific formula is as follows: ; in, Indicates the number of abnormal word structures at level i, The weight coefficient is determined according to the specific situation and needs, and is usually jointly formulated and confirmed by professionals or relevant stakeholders. , , , The formula is used to determine the correlation ratio between abnormal word structures at each level and text content, and the abnormal word structure level that needs to be controlled is screened according to the demand and the correlation ratio.

[0029] Abnormal word structure control module: Based on the correlation between the abnormal word structure level and the text content, determine the abnormal word structure that needs to be controlled, further clarify the specific position and content of the words, phrases, and sentences that need to be modified, use the existing deep learning text generation model, and perform control operations based on actual needs while ensuring content security to clarify whether to change the overall structure or logical order of the text. The control operations include: deletion, which refers to removing unnecessary modifiers or repeated information; replacement, which refers to replacing the original word with a synonym that is similar or identical to the original meaning; optimization, which refers to appropriately adjusting nouns, verbs, adjectives and other parts of speech as needed; and optimization, which refers to appropriately adjusting nouns, verbs, adjectives and other parts of speech that have been modified. Verify the text content modified by the control operation to ensure that the sentence structure and grammar of the modified text content are complete, obtain the verified text content, calculate the content safety value of the text content, determine whether the text content is affected by the abnormal content text, output the text content that is judged not to be affected by the abnormal content text, and for the text content that is judged to be affected by the abnormal content text, determine the abnormal word structure that needs to be controlled and perform the control operation by calculating the proportion of the correlation between the abnormal word structure level and the text content, and repeat this step until the text content is judged not to be affected by the abnormal content text, and then output the text content.

[0030] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0031] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An adaptive content security AI retrieval control method, characterized in that: include: Step 1: Collect unsafe online text content and establish an AI retrieval and comparison database; Step 2: determine the word structure of the abnormal text of the data to be retrieved, and evaluate the content security of the data to be retrieved; determine the impact of the abnormal text of different levels on the content security, and obtain the text content security value of the data to be retrieved; Step 3: Analyze the text content that is affected by content security, determine the degree of correlation between the text content and different types of abnormal word structures; determine the proportion of the correlation between abnormal word structures at each level and the text content, and select the abnormal word structure level that needs to be controlled; Step 4: Determine the abnormal word structure that needs to be controlled to enhance the content security of the text content.

2. According to claim 1, an adaptive content security AI retrieval control method is characterized in that: Collect unsafe online text content and build an AI retrieval and comparison database. The specific method is as follows: A large amount of network text content data that has been marked as unsafe is collected and recorded as abnormal content text. The abnormal content text is classified according to its sensitivity, potential risk of adverse impact, and negative emotional tendency. The text that is prohibited from dissemination by organizations, industries or laws is defined as level one abnormal content text; the text that poses a risk of adverse impact is defined as level two abnormal content text; the text that involves covering up or fabricating facts is defined as level three abnormal content text; the text that has disordered word order, a mixture of Chinese and English, and the presence of special characters is defined as level four abnormal content text. Different levels of abnormal content text are classified and stored, and a database for AI retrieval and comparison is established.

3. According to claim 1, the adaptive content security AI retrieval control method is characterized in that: Determine the word structure of the text belonging to the abnormal content of the data to be retrieved, the specific method is as follows: Perform AI search and comparison on the text content with complete structure, filter out the abnormal text in the data to be retrieved, mark the number of occurrences of each abnormal word structure in the abnormal text, determine the level of each abnormal word structure in the abnormal text, and obtain the abnormal word structure set, which is expressed as , where G(A) represents the set of abnormal word structures contained in the text content A, and x represents the xth abnormal word structure in the set. Represents a set of abnormal word structures belonging to the first-level abnormal content text, Represents a set of abnormal word structures belonging to abnormal text of secondary content, Represents a set of abnormal word structures belonging to the third-level abnormal content text, Represents a set of abnormal word structures belonging to level 4 content abnormal text.

4. According to claim 1, the adaptive content security AI retrieval control method is characterized in that: Get the text content security value of the data to be retrieved. The specific method is: Using the formula Calculate the text content security value of the data to be retrieved, where: represents the content safety value of text content A, U represents the number of words in text content A, u represents the number of words in abnormal content in text content A, i represents the level of abnormal content, represents the xth abnormal word structure in the abnormal word structure set of level i, Indicates abnormal word structure The number of occurrences, Indicates abnormal word structure The number of words, Indicates the i-th level content abnormal text weight value set according to text content A.

5. According to claim 1, the adaptive content security AI retrieval control method is characterized in that: The content security of the data to be retrieved is assessed by: A content security threshold is set according to the text content type and the text content security requirements. When the content security value of the text content of the data to be retrieved is greater than or equal to the content security threshold, it is determined that the content security of the data to be retrieved is affected by the abnormal content text.

6. The adaptive content security AI retrieval control method according to claim 1, characterized in that: Analyze the text content that is affected by content security and determine the degree of correlation between the text content and different types of abnormal word structures. The specific method is as follows: Using the formula Determine the degree of correlation between text content and different types of abnormal word structures, where Indicates abnormal word structure The degree of correlation with the text content is recorded as abnormal correlation. Indicates abnormal word structure The number of occurrences, represents the xth abnormal word structure in the abnormal word structure set of the i-th level, M represents the sum of the number of occurrences of all abnormal word structures, and N represents the total number of paragraphs obtained by segmenting the text content. Indicates that it contains abnormal word structure The number of paragraphs, e represents the base of the logarithm of natural numbers.

7. The adaptive content security AI retrieval control method according to claim 1 is characterized in that: Determine the correlation between each level of abnormal word structure and text content. The specific method is as follows: Using the formula Determine the proportion of the correlation between each abnormal word structure level and the text content, where: Indicates the number of abnormal word structures at level i, Represents the weight coefficient of the i-th level.

8. The adaptive content security AI retrieval control method according to claim 1, characterized in that: Determine the abnormal word structure that needs to be controlled to enhance the content security of the text content. The specific methods are as follows: Based on the proportion of the correlation between the abnormal word structure level and the text content, determine the abnormal word structure that needs to be controlled, further clarify the specific position and content of the characters, words and sentences that need to be modified, use the existing deep learning text generation model, perform control operations on the premise of ensuring content security according to actual needs, verify the text content modified by the control operation, calculate the content safety value of the verified text content, determine whether the text content is affected by the abnormal content text, output the text content that is not affected by the abnormal content text, and for the text content affected by the abnormal content text, repeatedly determine the abnormal word structure that needs to be controlled and perform control operations until the text content is determined not to be affected by the abnormal content text and then output the text content.

9. An adaptive content security AI retrieval control system, applied to an adaptive content security AI retrieval control method according to any one of claims 1 to 8, characterized in that: include: The AI ​​retrieval and comparison database module is used to collect network text content data that has been marked as unsafe, record it as abnormal content text, classify the abnormal content text based on its sensitivity, potential adverse impact risk, and negative emotional tendency, and classify and store abnormal content texts of different levels for AI retrieval and comparison; The content security value calculation module converts non-text content data into text content data, performs AI search and comparison on the text content, determines the word structure of the text with abnormal content, records it as abnormal word structure, determines the impact of the abnormal content text on the data to be retrieved through formula calculation, and evaluates the text content security value of the data to be retrieved; A correlation degree determination module determines the correlation degree between the text content and different abnormal word structures based on the abnormal word structure set contained in the text content data, determines the correlation degree ratio of each abnormal word structure level to the text content according to the level of the abnormal word structure, and selects the abnormal word structure level that needs to be controlled according to the demand and the correlation degree ratio; The abnormal word structure control module clarifies the specific location and content of the characters, words, and sentences that need to be modified, performs control operations based on actual needs while ensuring content security, verifies the text content modified by the control operation, and determines whether the text content is affected by the abnormal text content. By repeatedly calculating the proportion of the correlation between the abnormal word structure level and the text content, the abnormal word structure that needs to be controlled is determined and control operations are performed until the text content is determined not to be affected by the abnormal text content and the text content is output.

Images