An artificial intelligence recognition method based on big data

Through artificial intelligence recognition methods based on big data, sensitive words on the network platform are identified and filtered, and the problem of user improper speech content recognition is solved, and network information is purified and good atmosphere is maintained.

CN118821760BActive Publication Date: 2024-11-29四川易景智能终端有限公司
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411302989.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-11-29
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

On the online platform, it is difficult to effectively identify and filter the content of improper speech or sensitive words of users, affecting the quality of the network atmosphere.

Method used

Using artificial intelligence recognition method based on big data, we use the feature parameters of the user's output text to build a disassembled function and semantic expression matrix, extract the content of doubtful expression, and compare it with the sensitive word database to identify and filter sensitive words.

Benefits of technology

It realizes the evaluation and filtering of sensitive words for user output text to ensure the purification of network information and a good network atmosphere.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118821760B_ABST
    Figure CN118821760B_ABST
Patent Text Reader

Abstract

The present invention discloses an artificial intelligence recognition method based on big data, which belongs to the field of semantic recognition technology and includes the following steps: S1, obtaining the output text of the user, extracting the characteristic parameters of the output text, and constructing a disassembly function for each sentence; S2, constructing a disassembly characteristic value using the disassembly function of each sentence, and extracting the questionable expression content of the output text; S3, comparing the questionable expression content with a sensitive word database, and identifying whether the output text contains sensitive words. The present invention can screen and compare the output text of the user, ensure that the output text does not contain illegal content, and purify network information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of semantic recognition, and in particular relates to an artificial intelligence recognition method based on big data. Background Art

[0002] With the development of mobile Internet, users can publish information on various network platforms and comment on media resources on various network platforms; however, all good interactions are based on high-quality media resources and a good network atmosphere; when users output text content on the network, there may be some inappropriate remarks or content, and some sensitive words will inevitably appear, which will affect the good network atmosphere. To build a good network atmosphere, it is mainly necessary for each network platform to search and verify the content, and after determining that it is bad information, intercept and filter the bad information to purify the network information. Summary of the invention

[0003] In order to solve the above problems, the present invention proposes an artificial intelligence recognition method based on big data.

[0004] The technical solution of the present invention is: an artificial intelligence recognition method based on big data comprises the following steps:

[0005] S1. Obtain the user's output text, extract the feature parameters of the output text, and construct a decomposition function for each sentence;

[0006] S2, using the decomposition function of each sentence to construct a decomposition feature value and extract the questionable expression content of the output text;

[0007] S3. Compare the questionable expression content with the sensitive word database to identify whether the output text contains sensitive words.

[0008] Furthermore, S1 includes the following sub-steps:

[0009] S11, obtaining the user's output text, and extracting the query vector, key vector, and value vector of the output text;

[0010] S12, taking the mean between the query vector value and the key vector value as the first linear coefficient, taking the mean between the query vector value and the value vector value as the second linear coefficient, and taking the mean between the key vector value and the value vector value as the third linear coefficient;

[0011] S13. Construct a decomposition function for each sentence of the output text based on the first linear coefficient, the second linear coefficient and the third linear coefficient.

[0012] The beneficial effect of the above further scheme is: in the present invention, the query vector, key vector and value vector corresponding to the output text can reflect the dependency relationship of the text. Therefore, the three vector values ​​of the output text are combined with the word vectors of each word to generate a decomposition function for the sentence, which can provide basic parameters for the extraction of questionable expression content.

[0013] Furthermore, in S13, the expression of the sentence decomposition function Part is:

[0014] ; In the formula, a1 represents the first linear coefficient, a2 represents the second linear coefficient, a3 represents the third linear coefficient, f1 represents the frequency of the first word in the sentence, c1 represents the word vector of the first word in the sentence, and f m represents the frequency of the mth word in the sentence, c m Represents the word vector of the mth word in the sentence, f M represents the frequency of the Mth word in the sentence, c M represents the word vector of the Mth word in the sentence, log(·) represents the logarithmic function, and max(·) represents the maximum value function.

[0015] Furthermore, S2 includes the following sub-steps:

[0016] S21, taking the decomposition function values ​​of all sentences as decomposition value sequences;

[0017] S22, determining a disassembly characteristic value according to the disassembly value sequence;

[0018] S23, constructing a semantic expression matrix for each sentence;

[0019] S24, determining the semantic expression value of each sentence according to the maximum singular value of the semantic expression matrix;

[0020] S24. The sentences whose semantic expression value is less than the decomposition feature value are regarded as questionable expression contents.

[0021] The beneficial effect of the above further scheme is that: in the present invention, the maximum value and median of the decomposition function values ​​of all sentences are used as eigenvalues ​​to determine the decomposition eigenvalues ​​to screen the questionable expression content. A semantic expression matrix with the same number of rows and columns is constructed for each sentence, and the word vector values ​​of the words in the sentence are filled into the matrix, so that the matrix is ​​comprehensive and accurate.

[0022] Furthermore, in S22, the calculation formula of the disassembly characteristic value x is:

[0023] ; In the formula, max(·) represents the maximum value function, and media(·) represents the median function.

[0024] Furthermore, in S23, the method for constructing the semantic expression matrix is ​​as follows: according to the number of words contained in the sentence, the number of rows and columns of the semantic expression matrix is ​​determined, and the words of the sentence are sorted in descending order of word frequency, and the word vector values ​​are filled into the semantic expression matrix in sequence, and the empty positions are filled with 1.

[0025] The number of rows and columns of the semantic expression matrix is ​​the same. For example, if the sentence contains 8 words, the number of words is padded to 9. The number of rows and columns of the semantic expression matrix is ​​3. The word with the highest frequency is the element of the 1st row and 1st column, and the word with the second highest frequency is the element of the 1st row and 2nd column, and so on. If there is no corresponding word for the element in the 3rd row and 3rd column, it is padded with 1.

[0026] Furthermore, in S24, the calculation formula of the semantic expression value B of the sentence is:

[0027] ; where μ represents the maximum singular value of the semantic expression matrix, λ represents the regularization parameter, Z represents the semantic expression matrix of the sentence, T represents the matrix transpose, and tanh(·) represents the hyperbolic tangent function.

[0028] Furthermore, S3 includes the following sub-steps:

[0029] S31, removing stop words from the questionable expression content to obtain standard questionable expression content;

[0030] S32, taking the words contained in all sentences in the standard questionable expression content as a questionable expression word set;

[0031] S33. Calculate the similarity between each word in the questionable expression word set and the words in the sensitive word database. If the similarity is greater than or equal to the similarity threshold, the output text contains sensitive words.

[0032] The beneficial effects of the present invention are as follows: the present invention discloses an artificial intelligence recognition method based on big data, which performs sensitive word evaluation on the content that a user needs to publish and output, constructs a decomposition function for each sentence of the user's output text, thereby screening out questionable expressions that may contain sensitive words, and then compares the questionable expressions with an existing sensitive word database to generate a final result; the present invention can screen and compare the user's output text to ensure that the output text does not contain any illegal content, thereby purifying network information. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Flowchart of the artificial intelligence recognition method based on big data. DETAILED DESCRIPTION

[0034] The embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, the present invention provides an artificial intelligence recognition method based on big data, comprising the following steps:

[0036] S1. Obtain the user's output text, extract the feature parameters of the output text, and construct a decomposition function for each sentence;

[0037] S2, using the decomposition function of each sentence to construct a decomposition feature value and extract the questionable expression content of the output text;

[0038] S3. Compare the questionable expression content with the sensitive word database to identify whether the output text contains sensitive words.

[0039] In this embodiment of the present invention, S1 includes the following sub-steps:

[0040] S11, obtaining the user's output text, and extracting the query vector, key vector, and value vector of the output text;

[0041] S12, taking the mean between the query vector value and the key vector value as the first linear coefficient, taking the mean between the query vector value and the value vector value as the second linear coefficient, and taking the mean between the key vector value and the value vector value as the third linear coefficient;

[0042] S13. Construct a decomposition function for each sentence of the output text based on the first linear coefficient, the second linear coefficient and the third linear coefficient.

[0043] In the present invention, the query vector, key vector and value vector corresponding to the output text can reflect the dependency relationship of the text. Therefore, the three vector values ​​of the output text are combined with the word vectors of each word to generate a decomposition function for the sentence, which can provide basic parameters for the extraction of questionable expression content.

[0044] In the embodiment of the present invention, in S13, the expression of the sentence decomposition function Part is:

[0045] ; In the formula, a1 represents the first linear coefficient, a2 represents the second linear coefficient, a3 represents the third linear coefficient, f1 represents the frequency of the first word in the sentence, c1 represents the word vector of the first word in the sentence, and f m represents the frequency of the mth word in the sentence, c m Represents the word vector of the mth word in the sentence, f M represents the frequency of the Mth word in the sentence, c M represents the word vector of the Mth word in the sentence, log(·) represents the logarithmic function, and max(·) represents the maximum value function.

[0046] In this embodiment of the present invention, S2 includes the following sub-steps:

[0047] S21, taking the decomposition function values ​​of all sentences as decomposition value sequences;

[0048] S22, determining a disassembly characteristic value according to the disassembly value sequence;

[0049] S23, constructing a semantic expression matrix for each sentence;

[0050] S24, determining the semantic expression value of each sentence according to the maximum singular value of the semantic expression matrix;

[0051] S24. The sentences whose semantic expression value is less than the decomposition feature value are regarded as questionable expression contents.

[0052] In the present invention, the maximum and median of the decomposition function values ​​of all sentences are used as eigenvalues ​​to determine the decomposition eigenvalues ​​to screen out questionable expressions. A semantic expression matrix with the same number of rows and columns is constructed for each sentence, and the word vector values ​​of the words in the sentence are filled into the matrix to make the matrix comprehensive and accurate.

[0053] In the embodiment of the present invention, in S22, the calculation formula of the disassembly characteristic value x is:

[0054] ; In the formula, max(·) represents the maximum value function, and media(·) represents the median function.

[0055] In an embodiment of the present invention, in S23, the method for constructing the semantic expression matrix is: according to the number of words contained in the sentence, the number of rows and columns of the semantic expression matrix is ​​determined, and the words of the sentence are sorted in descending order of word frequency, and the word vector values ​​are filled into the semantic expression matrix in sequence, and the empty positions are filled with 1.

[0056] In the present invention, the number of rows and columns of the semantic expression matrix is ​​the same. For example, if the sentence contains 8 words, the number of words is padded to 9. The number of rows and columns of the semantic expression matrix are both 3. The word with the highest frequency is used as the element of the 1st row and 1st column, and the word with the second highest frequency is used as the element of the 1st row and 2nd column, and so on. If there is no corresponding word for the element in the 3rd row and 3rd column, it is padded with 1.

[0057] In the embodiment of the present invention, in S24, the calculation formula of the semantic expression value B of the sentence is:

[0058] ; where μ represents the maximum singular value of the semantic expression matrix, λ represents the regularization parameter, Z represents the semantic expression matrix of the sentence, T represents the matrix transpose, and tanh(·) represents the hyperbolic tangent function.

[0059] In this embodiment of the present invention, S3 includes the following sub-steps:

[0060] S31, removing stop words from the questionable expression content to obtain standard questionable expression content;

[0061] S32, taking the words contained in all sentences in the standard questionable expression content as a questionable expression word set;

[0062] S33. Calculate the similarity between each word in the questionable expression word set and the words in the sensitive word database. If the similarity is greater than or equal to the similarity threshold, the output text contains sensitive words.

[0063] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An artificial intelligence recognition method based on big data, characterized in that: The following steps are involved: S1. Obtain the user's output text, extract the feature parameters of the output text, and construct a decomposition function for each sentence; S2, using the decomposition function of each sentence to construct a decomposition feature value and extract the questionable expression content of the output text; S3, comparing the questionable expression content with the sensitive word database to identify whether the output text contains sensitive words; The S1 comprises the following sub-steps: S11, obtaining the user's output text, and extracting the query vector, key vector, and value vector of the output text; S12, taking the mean between the query vector value and the key vector value as the first linear coefficient, taking the mean between the query vector value and the value vector value as the second linear coefficient, and taking the mean between the key vector value and the value vector value as the third linear coefficient; S13, constructing a decomposition function for each sentence of the output text based on the first linear coefficient, the second linear coefficient and the third linear coefficient; In S13, the expression of the sentence decomposition function Part is: ; In the formula, a1 represents the first linear coefficient, a2 represents the second linear coefficient, a3 represents the third linear coefficient, f1 represents the frequency of the first word in the sentence, c1 represents the word vector of the first word in the sentence, and f m represents the frequency of the mth word in the sentence, c m Represents the word vector of the mth word in the sentence, f M represents the frequency of the Mth word in the sentence, c M represents the word vector of the Mth word in the sentence, log(·) represents the logarithmic function, and max(·) represents the maximum value function; The S2 comprises the following sub-steps: S21, taking the decomposition function values ​​of all sentences as decomposition value sequences; S22, determining a disassembly characteristic value according to the disassembly value sequence; S23, constructing a semantic expression matrix for each sentence; S24, determining the semantic expression value of each sentence according to the maximum singular value of the semantic expression matrix; S24, taking sentences with semantic expression values ​​less than the decomposition feature value as questionable expression content; In S22, the calculation formula of the disassembly characteristic value x is: ; In the formula, max(·) represents the maximum value function, media(·) represents the median function, Part k Represents the decomposition function value of the kth sentence; In S23, the method for constructing the semantic expression matrix is ​​as follows: according to the number of words contained in the sentence, the number of rows and columns of the semantic expression matrix is ​​determined, and the words of the sentence are sorted in descending order of word frequency, and the word vector values ​​are sequentially filled into the semantic expression matrix, and the idle positions are filled with 1; In S24, the calculation formula of the semantic expression value B of the sentence is: ; where μ represents the maximum singular value of the semantic expression matrix, λ represents the regularization parameter, Z represents the semantic expression matrix of the sentence, T represents the matrix transpose, and tanh(·) represents the hyperbolic tangent function.

2. The artificial intelligence recognition method based on big data according to claim 1 is characterized in that: The S3 comprises the following sub-steps: S31, removing stop words from the questionable expression content to obtain standard questionable expression content; S32, taking the words contained in all sentences in the standard questionable expression content as a questionable expression word set; S33. Calculate the similarity between each word in the questionable expression word set and the words in the sensitive word database. If the similarity is greater than or equal to the similarity threshold, the output text contains sensitive words.

Citation Information

Patent Citations

  • Text recognition method and device, computer equipment and computer readable storage medium

    CN115659965A

  • Sensitive word auditing method based on large language model, storage medium and electronic equipment

    CN116720515A

  • Information processing method and apparatus, and storage medium

    US20210182501A1