AI-based deep synthesis detection methods and systems

Abstract

This invention provides an AI-based deep synthesis detection method and system. By identifying the type of different data packets, different feature extraction methods are applied to image, audio, or text types. For image types, dimensionality reduction sampling and sliding window secondary sampling are used; for audio types, reference voiceprint matching and sliding box secondary sampling are used; and for text types, sentence segmentation and recombination clustering are used. While fusing image features, sound features, and sequence features, the system also utilizes the emotional information obtained from feature synthesis to eliminate ambiguity and uncertainty, resulting in accurate classification results. Through the above processing, the problem of existing fusion classification processes not being able to completely eliminate interference is overcome.

Description

Technical Field

[0001] This application relates to the field of cybersecurity technology, and in particular to an AI-based deep synthetic detection method and system. Background Technology

[0002] Existing content detection methods mostly employ convolutional neural networks (CNNs) or recurrent neural networks (RNNs). CNNs extract image features, use pooling layers to reduce the dimensionality of these features, and employ flattening and densely connected layers to perform nonlinear transformations, achieving classification. RNNs are used to extract features from training data sequences, generating specific-dimensional data to represent sequence features by traversing all elements of the data.

[0003] However, directly using convolutional neural networks or recurrent neural networks to extract and fuse features of image or text types cannot completely eliminate interference during the fusion classification process, and is still insufficient to cope with complex network environments.

[0004] Therefore, there is an urgent need for a targeted AI-based deep synthesis detection method and system. Summary of the Invention

[0005] The purpose of this invention is to provide an AI-based deep synthesis detection method and system to solve the problem that interference cannot be completely eliminated in the existing fusion classification process.

[0006] Firstly, this application provides an AI-based deep synthesis detection method, the method comprising:

[0007] Different types of data packets are collected from the Internet network. Based on the metadata carried by the data packets, the type of data packet is identified as image type, audio type, or text type.

[0008] When the data stream of the acquired data packet is identified as an image, it is discretized by sampling according to the temporal continuous domain to obtain a dimension-reduced discrete data stream, which is then converted into a grayscale image. The first image feature of the grayscale image is extracted, vectorized, and input into an N-layer convolutional unit to output the first intermediate result after convolution. Based on the feature value distribution of the first intermediate result, the width of the sliding window is determined, and the sliding window is used to sample the data stream of the acquired data packet again to directly extract the second image feature.

[0009] When the data stream of the acquired data packet is identified as an audio type, the data stream of the acquired data packet is discretized and sampled. The sampled values ​​are arranged into a sequence and matched with the reference voiceprints stored in the database. A sliding box is generated according to the matching reference voiceprints. The data stream of the acquired data packet is sampled again using the sliding box to directly extract the sound features.

[0010] When identified as text, the sentence is extracted from the stream sequence of the collected data packet, and the sentence is input into the syntactic model for preliminary sentence segmentation to obtain the first word component. According to the pre-set mapping relationship between word group type and weight value, all the first word components after preliminary sentence segmentation are analyzed. The first word components with weight values ​​greater than the threshold are clustered to form new sentences, and sequence features are extracted from them.

[0011] The second image features, sound features, and sequence features are combined and judged to obtain emotional information;

[0012] The second image feature, sound feature, and sequence feature are fused and fed into the convolutional layer of the recognition model. Sliding windows of different sizes are used to select local feature components, and the local feature components are concatenated to obtain a first feature matrix. This feature matrix is ​​then fed into the pooling layer of the recognition model. The pooling layer selects effective feature values ​​by choosing a pooling function, and the features are concatenated again to obtain a second feature matrix.

[0013] The second feature matrix is ​​matched with a reference matrix stored in the database, wherein the matching also uses the sentiment information to eliminate ambiguity and uncertainty, and obtains a classification result;

[0014] The collected data packets are managed according to the classification results.

[0015] Secondly, this application provides an AI-based deep synthesis detection system, the system comprising:

[0016] The data acquisition and identification unit is used to acquire different types of data packets from the Internet network, and to identify the type of the data packet as image type, audio type or text type based on the metadata carried by the data packet.

[0017] The image feature extraction unit is used to discretize the data stream of the acquired data packet when it is identified as an image type. It samples continuously in the time domain to obtain a dimension-reduced discrete data stream, which is then converted into a grayscale image. The first image feature of the grayscale image is extracted, vectorized, and input into an N-layer convolutional unit to output the first intermediate result after convolution. Based on the feature value distribution of the first intermediate result, the width of the sliding window is determined, and the sliding window is used to sample the data stream of the acquired data packet again to directly extract the second image feature.

[0018] The sound feature extraction unit is used to discretize the data stream of the acquired data packet when it is identified as an audio type, form a sequence of sampled values, match it with the reference voiceprints stored in the database, generate a sliding box according to the matching reference voiceprints, and use the sliding box to sample the data stream of the acquired data packet again to directly extract sound features.

[0019] The sequence feature extraction unit is used to extract sentences from the stream sequence of the collected data packet when the data is identified as text, input the sentences into the syntactic model, perform preliminary sentence segmentation to obtain the first word component; according to the pre-set mapping relationship between word group type and weight value, analyze all the first word components after preliminary sentence segmentation, cluster the first word components with weight values ​​greater than the threshold to form new sentences, and extract sequence features from them;

[0020] The judgment unit is used to synthesize and judge the second image features, sound features and sequence features to obtain emotional information;

[0021] The fusion unit is used to fuse the second image features, sound features, and sequence features, and feed them into the convolutional layer of the recognition model. It uses sliding windows of different sizes to select local feature components, concatenates the local feature components to obtain a first feature matrix, and feeds the feature matrix into the pooling layer of the recognition model. The pooling layer selects effective feature values ​​by selecting a pooling function, and concatenates them again to obtain a second feature matrix.

[0022] A classification unit is used to match the second feature matrix with a reference matrix stored in the database, wherein the matching also utilizes the sentiment information to eliminate ambiguity and uncertainty, and obtain a classification result;

[0023] The management unit is used to manage the collected data packets based on the classification results.

[0024] Thirdly, this application provides an AI-based deep synthesis detection system, the system comprising a processor and a memory:

[0025] The memory is used to store program code and transmit the program code to the processor;

[0026] The processor is configured to execute any one of the four possible methods of the first aspect according to the instructions in the program code.

[0027] Fourthly, this application provides a computer-readable storage medium for storing program code for performing any one of the four possible methods of the first aspect.

[0028] Beneficial effects

[0029] This invention provides an AI-based deep synthesis detection method and system. By identifying the type of different data packets, different feature extraction methods are applied to image, audio, or text types. For image types, dimensionality reduction sampling and sliding window secondary sampling are used; for audio types, reference voiceprint matching and sliding box secondary sampling are used; and for text types, sentence segmentation and recombination clustering are used. While fusing image features, sound features, and sequence features, the system also utilizes the emotional information obtained from feature synthesis to eliminate ambiguity and uncertainty, resulting in accurate classification results. Through the above processing, the problem of existing fusion classification processes not being able to completely eliminate interference is overcome. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0031] Figure 1 This is a general flowchart of the AI-based deep synthesis detection method of the present invention;

[0032] Figure 2 This is an architecture diagram of the AI-based deep synthesis detection system of the present invention. Detailed Implementation

[0033] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0034] Figure 1 A general flowchart of the AI-based deep synthesis detection method provided in this application, the method comprising:

[0035] Different types of data packets are collected from the Internet network. Based on the metadata carried by the data packets, the type of data packet is identified as image type, audio type, or text type.

[0036] When the data stream of the acquired data packet is identified as an image, it is discretized by sampling according to the temporal continuous domain to obtain a dimension-reduced discrete data stream, which is then converted into a grayscale image. The first image feature of the grayscale image is extracted, vectorized, and input into an N-layer convolutional unit to output the first intermediate result after convolution. Based on the feature value distribution of the first intermediate result, the width of the sliding window is determined, and the sliding window is used to sample the data stream of the acquired data packet again to directly extract the second image feature.

[0037] When the data stream of the acquired data packet is identified as an audio type, the data stream of the acquired data packet is discretized and sampled. The sampled values ​​are arranged into a sequence and matched with the reference voiceprints stored in the database. A sliding box is generated according to the matching reference voiceprints. The data stream of the acquired data packet is sampled again using the sliding box to directly extract the sound features.

[0038] When identified as text, the sentence is extracted from the stream sequence of the collected data packet, and the sentence is input into the syntactic model for preliminary sentence segmentation to obtain the first word component. According to the pre-set mapping relationship between word group type and weight value, all the first word components after preliminary sentence segmentation are analyzed. The first word components with weight values ​​greater than the threshold are clustered to form new sentences, and sequence features are extracted from them.

[0039] The second image features, sound features, and sequence features are combined and judged to obtain emotional information;

[0040] The second image feature, sound feature, and sequence feature are fused and fed into the convolutional layer of the recognition model. Sliding windows of different sizes are used to select local feature components, and the local feature components are concatenated to obtain a first feature matrix. This feature matrix is ​​then fed into the pooling layer of the recognition model. The pooling layer selects effective feature values ​​by choosing a pooling function, and the features are concatenated again to obtain a second feature matrix.

[0041] The second feature matrix is ​​matched with a reference matrix stored in the database, wherein the matching also uses the sentiment information to eliminate ambiguity and uncertainty, and obtains a classification result;

[0042] The collected data packets are managed according to the classification results.

[0043] In some preferred embodiments, during training, the recognition model minimizes the entropy loss function through backpropagation to avoid oversaturation. When the accuracy of the recognition model meets the threshold requirement, it indicates that the training of the recognition model is complete. It can then be used for data validation.

[0044] In some preferred embodiments, the fusion of the second image features, sound features, and sequence features includes: extracting the second image features one by one by row or column and writing them into a single-dimensional matrix; normalizing the sound features one by one and writing them into a single-dimensional matrix; extracting the sequence features one by one and writing them into a single-dimensional matrix; and weighting or summing the values ​​of the corresponding bits of the three single-dimensional matrices after pairwise conjugation to obtain the fused feature matrix.

[0045] In some preferred embodiments, the use of the emotional information to exclude ambiguous and uncertain information specifically means that the types of emotional information include joy, anger, sorrow, and happiness. If the matching result of the second feature matrix and the reference matrix does not match the type of emotional information, it will be excluded.

[0046] In some preferred embodiments, a voting mechanism may be introduced during the classification process, wherein the voting method includes weighted summation of the output results of each decision tree.

[0047] Matrix matching forms a tree structure, which includes multiple decision trees.

[0048] Each decision tree has a specific classification capability. The specified feature values ​​are derived based on different classifications. By classifying the same feature vector matrix through decision trees from different perspectives, the system integrates different classification capabilities. Its classification performance is higher than that of a single classifier.

[0049] Figure 2 This application provides an architecture diagram of an AI-based deep synthesis detection system, which includes:

[0050] The data acquisition and identification unit is used to acquire different types of data packets from the Internet network, and to identify the type of the data packet as image type, audio type or text type based on the metadata carried by the data packet.

[0051] The image feature extraction unit is used to discretize the data stream of the acquired data packet when it is identified as an image type. It samples continuously in the time domain to obtain a dimension-reduced discrete data stream, which is then converted into a grayscale image. The first image feature of the grayscale image is extracted, vectorized, and input into an N-layer convolutional unit to output the first intermediate result after convolution. Based on the feature value distribution of the first intermediate result, the width of the sliding window is determined, and the sliding window is used to sample the data stream of the acquired data packet again to directly extract the second image feature.

[0052] The sound feature extraction unit is used to discretize the data stream of the acquired data packet when it is identified as an audio type, form a sequence of sampled values, match it with the reference voiceprints stored in the database, generate a sliding box according to the matching reference voiceprints, and use the sliding box to sample the data stream of the acquired data packet again to directly extract sound features.

[0053] The sequence feature extraction unit is used to extract sentences from the stream sequence of the collected data packet when the data is identified as text, input the sentences into the syntactic model, perform preliminary sentence segmentation to obtain the first word component; according to the pre-set mapping relationship between word group type and weight value, analyze all the first word components after preliminary sentence segmentation, cluster the first word components with weight values ​​greater than the threshold to form new sentences, and extract sequence features from them;

[0054] The judgment unit is used to synthesize and judge the second image features, sound features and sequence features to obtain emotional information;

[0055] The fusion unit is used to fuse the second image features, sound features, and sequence features, and feed them into the convolutional layer of the recognition model. It uses sliding windows of different sizes to select local feature components, concatenates the local feature components to obtain a first feature matrix, and feeds the feature matrix into the pooling layer of the recognition model. The pooling layer selects effective feature values ​​by selecting a pooling function, and concatenates them again to obtain a second feature matrix.

[0056] A classification unit is used to match the second feature matrix with a reference matrix stored in the database, wherein the matching also utilizes the sentiment information to eliminate ambiguity and uncertainty, and obtain a classification result;

[0057] The management unit is used to manage the collected data packets based on the classification results.

[0058] This application provides an AI-based deep synthesis detection system, the system comprising: a processor and a memory.

[0059] The memory is used to store program code and transmit the program code to the processor;

[0060] The processor is configured to execute the method described in any one of the embodiments of the first aspect according to the instructions in the program code.

[0061] This application provides a computer-readable storage medium for storing program code for performing the method described in any one of the embodiments of the first aspect.

[0062] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program, when executed, may include some or all of the steps in the various embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0063] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0064] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0065] The embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention.

Claims

1. An AI-based deep synthesis detection method, characterized in that, The method comprises: Collecting different types of collection data packets in the Internet network, and identifying the types of the data packets as image type, audio type or text type according to metadata carried by the collection data packets; When the type is identified as the image type, the data stream of the collection data packet is discretized and sampled according to time domain continuity to obtain a discrete data stream after dimension reduction, and a grayscale image is converted; first image features of the grayscale image are extracted, vectorized and input into an N-layer convolution unit to output a first intermediate result after convolution; the width of a sliding window is determined according to the feature value distribution of the first intermediate result, and the data stream of the collection data packet is sampled again using the sliding window to directly extract second image features therefrom; When the type is identified as the audio type, the data stream of the collection data packet is discretized and sampled, and the sampling values are combined into a sequence and matched with reference voiceprints stored in a database to generate a sliding box according to the reference voiceprints that match, and the data stream of the collection data packet is sampled again using the sliding box to directly extract sound features therefrom; When the type is identified as the text type, a sentence is extracted from the stream sequence of the collection data packet, and the sentence is input into a syntax model to perform preliminary sentence segmentation to obtain first word components; all the first word components after preliminary sentence segmentation are analyzed according to a pre-set mapping relationship between word group types and weight values, the first word components with weight values greater than a threshold value are clustered to form a new sentence, and sequence features are extracted therefrom; The second image features, sound features and sequence features are combined to obtain emotional information; The second image features, sound features and sequence features are fused and input into a convolution layer of a recognition model, local feature components are selected using sliding windows of different sizes, the local feature components are spliced to obtain a first feature matrix, and the feature matrix is input into a pooling layer of the recognition model; the pooling layer selects effective feature values by selecting a pooling function to obtain a second feature matrix again; The fusing of the second image features, sound features and sequence features comprises: extracting the second image features row by row or column by column and writing them into a matrix of a single dimension one by one; writing the sound features into the matrix of the single dimension one by one after normalization; extracting the sequence features and writing them into the matrix of the single dimension one by one; and weighting or accumulating the values of the three matrices of the single dimension corresponding to positions after conjugation to obtain a fused feature matrix; The second feature matrix is matched with reference matrices stored in a database, wherein the matching further excludes ambiguous and uncertain information using the emotional information to obtain a classification result; The collection data packet is managed according to the classification result.

2. The method of claim 1, wherein: During training of the recognition model, an entropy loss function is minimized by a backward propagation mode to avoid over-saturation, and when the accuracy of the recognition model meets the threshold requirement, it is indicated that the training of the recognition model is completed.

3. The method of claim 2, wherein: The emotional information includes joy, anger, sadness and happiness, and if the matching result of the second feature matrix and the reference matrix does not match the type of the emotional information, the matching result is excluded.

4. An AI-based deep synthesis detection system, characterized by, The system comprises: The acquisition identification unit is used for acquiring different types of acquisition data packets in the Internet network, and identifying the type of the data packet as an image type, an audio type or a text type according to metadata carried by the acquisition data packet; The image feature extraction unit is used for, when the type is identified as the image type, performing discretization processing on a data stream of the acquisition data packet, sampling according to time domain continuity to obtain a discrete data stream after dimension reduction, and converting to obtain a gray image; extracting a first image feature of the gray image, inputting the first image feature into an N-layer convolution unit after vectorization, and outputting a first intermediate result after convolution; determining the width of a sliding window according to the feature value distribution of the first intermediate result, and sampling the data stream of the acquisition data packet again using the sliding window to directly extract a second image feature therefrom; The sound feature extraction unit is used for, when the type is identified as the audio type, performing discretization sampling on the data stream of the acquisition data packet, grouping sample values into a sequence, matching the sequence with a reference voiceprint stored in a database, generating a sliding box according to the reference voiceprint that matches, sampling the data stream of the acquisition data packet again using the sliding box to directly extract a sound feature therefrom; The sequence feature extraction unit is used for, when the type is identified as the text type, extracting a sentence from the stream sequence of the acquisition data packet, inputting the sentence into a syntax model to perform preliminary sentence breaking to obtain a first word component; analyzing all the first word components after preliminary sentence breaking according to a mapping relationship between a preset word group type and a weight value, clustering the first word components with weight values greater than a threshold value to form a new sentence, and extracting a sequence feature therefrom; The judgment unit is used for synthesizing the second image feature, the sound feature and the sequence feature to obtain emotional information; The fusion unit is used for fusing the second image feature, the sound feature and the sequence feature, inputting the second image feature, the sound feature and the sequence feature into a convolution layer of an identification model, selecting local feature components using sliding windows of different sizes, splicing the local feature components to obtain a first feature matrix, and inputting the first feature matrix into a pooling layer of the identification model; the pooling layer selects effective feature values by selecting a pooling function to obtain a second feature matrix again; The fusing of the second image feature, the sound feature and the sequence feature comprises: extracting the second image feature row by row or column by column as a unit, and writing the second image feature into a matrix of a single dimension one by one; normalizing the sound feature one by one and writing the sound feature into the matrix of the single dimension; extracting the sequence feature one by one and writing the sequence feature into the matrix of the single dimension; and weighting or accumulating the values of the three matrices of the single dimension corresponding to positions after being conjugated to obtain a fused feature matrix; The classification unit is used for matching the second feature matrix with a reference matrix stored in a database, wherein the matching further utilizes the emotional information to exclude ambiguous and uncertain information to obtain a classification result; The management unit is used for managing the acquisition data packet according to the classification result.

5. An AI-based deep synthesis detection system, characterized by, The system comprises a processor and a memory: The memory is used for storing program code and transmitting the program code to the processor; The processor is configured to execute instructions in the program code to implement the method of any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store program code for implementing the method of any one of claims 1-3.

Images