Data processing method, apparatus and device

Abstract

Embodiments of the present specification provide a data processing method, device and equipment, wherein the method comprises: splitting target video data to be detected into a plurality of sub-video data, performing image feature extraction processing on the sub-video data based on a feature extraction layer in a pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and performing time sequence feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data, inputting the first feature data and the second feature data corresponding to the sub-video data into a network layer after the feature extraction layer in the pre-trained anomaly detection model to obtain a type label of the target video data, and judging whether image data of a user contained in the target video data is synthetic data based on the type label of the target video data to obtain an anomaly detection result for the target video data.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a data processing method, apparatus and equipment. Background Technology

[0002] With the development and maturation of image synthesis technology, synthesized images are becoming increasingly realistic, posing significant security risks to security scenarios such as identity recognition. Because synthesized images utilize new technologies such as artificial intelligence, machine learning, and big data mining in malicious theft scenarios, they possess a high level of technological sophistication, making them more deceptive and misleading. Therefore, a solution is needed to improve the accuracy of detecting whether an image is synthesized. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to provide a data processing method, apparatus, and device to provide a solution that can improve the detection accuracy of whether a face image is a synthetic image.

[0004] To achieve the above technical solution, the embodiments in this specification are implemented as follows:

[0005] In a first aspect, embodiments of this specification provide a data processing method, comprising: acquiring target video data to be detected, and splitting the target video data into multiple sub-video data, wherein the target video data includes user image data; performing image feature extraction processing on the sub-video data based on a feature extraction layer in a pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and performing temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data; inputting the first feature data and the second feature data corresponding to the sub-video data into a network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data to obtain a type label for the target video data; and determining whether the user image data included in the target video data is synthetic data based on the type label of the target video data to obtain an anomaly detection result for the target video data.

[0006] Secondly, embodiments of this specification provide a data processing apparatus, the apparatus comprising: a first acquisition module, configured to acquire target video data to be detected and split the target video data into multiple sub-video data, the target video data including user image data; a first extraction module, configured to perform image feature extraction processing on the sub-video data based on a feature extraction layer in a pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and perform temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data; a first detection module, configured to input the first feature data and the second feature data corresponding to the sub-video data into a network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data to obtain a type label of the target video data; and a data judgment module, configured to judge whether the user image data included in the target video data is synthetic data based on the type label of the target video data to obtain anomaly detection results for the target video data.

[0007] Thirdly, embodiments of this specification provide a data processing device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to: acquire target video data to be detected, and split the target video data into multiple sub-video data, the target video data including user image data; perform image feature extraction processing on the sub-video data based on a feature extraction layer in a pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and perform temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data; input the first feature data and the second feature data corresponding to the sub-video data into a network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data to obtain a type label for the target video data; and, based on the type label of the target video data, determine whether the user image data included in the target video data is synthetic data to obtain an anomaly detection result for the target video data.

[0008] Fourthly, embodiments of this specification provide a storage medium for storing computer-executable instructions. When executed, these instructions implement the following process: acquiring target video data to be detected and splitting the target video data into multiple sub-video data, the target video data including user image data; performing image feature extraction processing on the sub-video data based on a feature extraction layer in a pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and performing temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data; inputting the first and second feature data corresponding to the sub-video data into a network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data; and determining whether the user image data included in the target video data is synthetic data based on the type label of the target video data, obtaining an anomaly detection result for the target video data. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of a data processing system described in this specification;

[0011] Figure 2A This is a flowchart illustrating an embodiment of a data processing method described in this specification;

[0012] Figure 2B This is a schematic diagram of the processing procedure of one data processing method described in this specification;

[0013] Figure 3 This is a schematic diagram illustrating the processing procedure of one data processing method described in this specification;

[0014] Figure 4 This is a schematic diagram illustrating the processing procedure of another data processing method described in this specification;

[0015] Figure 5 This is a schematic diagram illustrating the processing procedure of another data processing method described in this specification.

[0016] Figure 6This is a schematic diagram illustrating the processing procedure of another data processing method described in this specification.

[0017] Figure 7 This is a schematic diagram of the structure of an embodiment of a data processing device according to this specification;

[0018] Figure 8 This is a schematic diagram of the structure of a data processing device described in this specification. Detailed Implementation

[0019] This specification provides a data processing method, apparatus, and device through its embodiments.

[0020] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0021] This specification provides a data processing method, apparatus, and device through its embodiments.

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0023] The technical solutions in this specification can be applied to data processing systems, such as... Figure 1 As shown, the data processing system can have terminal devices and servers. The server can be a standalone server or a server cluster composed of multiple servers. The terminal device can be a personal computer or a mobile terminal device such as a mobile phone or tablet computer. Alternatively, the terminal device can be an Internet of Things device equipped with a camera component.

[0024] The data processing system may include n terminal devices and m servers, where n and m are positive integers greater than or equal to 1. The terminal devices can be used to collect video data samples. For example, the terminal devices can acquire corresponding video data samples for different synthetic image detection scenarios. For example, for synthetic image detection scenarios for identity authentication, the terminal devices can acquire video data containing user image data through configured camera components. Alternatively, the terminal devices can also receive video data containing user image data, and the terminal devices can use the acquired video data as video data samples, etc.

[0025] Terminal devices can send collected video data samples to any server in the data processing system. The server can then perform synthetic image detection based on the collected video data samples. Alternatively, the server can store the collected video data samples so that, when the model training cycle is reached, the stored video data samples can be used as historical video data to train the detection model.

[0026] When detecting whether user image data is synthetic, on the one hand, the server can use video data as input instead of the original single-frame image input. This allows the anomaly detection model to extract richer spatiotemporal information, assisting in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data through the anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user image data contained in the target video data is synthetic data, thus improving the detection accuracy of whether the image to be detected is synthetic, and consequently improving the accuracy of face recognition.

[0027] Based on the above data processing system architecture, the data processing methods in the following embodiments can be implemented.

[0028] Example 1

[0029] like Figure 2A and Figure 2B As shown in the embodiments of this specification, a data processing method is provided. The execution subject of this method can be a server, which can be a standalone server or a server cluster composed of multiple servers. The method specifically includes the following steps:

[0030] In S202, the target video data to be detected is acquired, and the target video data is split into multiple sub-video data.

[0031] The target video data may include the user's image data, which may contain the user's biometric data, such as data containing the user's facial image.

[0032] In practice, with the development and maturation of image synthesis technology, synthesized images are becoming increasingly realistic, posing significant security risks to security scenarios such as identity recognition. Because synthesized images utilize new technologies such as artificial intelligence, machine learning, and big data mining in scenarios involving malicious theft, they possess a high level of technological sophistication, making them more deceptive and misleading. Therefore, a solution is needed to improve the accuracy of detecting whether an image is a synthesized image. To this end, embodiments of this specification provide a technical solution that can address the above-mentioned problems, as detailed below.

[0033] Taking the synthetic image detection scenario for identity authentication as an example, when a terminal device receives an identity recognition command triggered by a user, it can collect video data containing the user's image data through the terminal device's camera component and send the collected video data to the server.

[0034] The server can identify the video data as the target video data and split the target video data into multiple sub-video data. The server can split the target video data into multiple sub-video data of equal length based on the timing information of the target video data. In addition, there are many other methods to split the target video data, and the embodiments in this specification do not specifically limit them.

[0035] In S204, based on the feature extraction layer in the pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain the first feature data corresponding to the sub-video data. Then, based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain the second feature data corresponding to the sub-video data.

[0036] Among them, the anomaly detection model can be a model built based on a preset machine learning algorithm to detect and process video data in order to obtain the type label of the video data.

[0037] In implementation, such as Figure 3As shown, the anomaly detection model can include a feature extraction layer 1 for image feature extraction processing of video data and a feature extraction layer 2 for temporal feature extraction processing of video data. The server can input the n sub-video data obtained from the splitting into feature extraction layer 1 to perform image feature extraction processing on each sub-video data, obtaining first feature data corresponding to each sub-video data, where n is a positive integer greater than or equal to 2. Simultaneously, the server can also input each sub-video data into feature extraction layer 2 to perform temporal feature extraction processing on each sub-video data, obtaining second feature data corresponding to each sub-video data.

[0038] In S206, the first feature data and the second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data, and obtain the type label of the target video data.

[0039] Among them, the category label of the target video data can be used to characterize whether the user image data contained in the target video data is synthetic data.

[0040] In implementation, such as Figure 3 As shown, the server can perform feature fusion processing on the first feature data corresponding to n sub-video data and feature fusion processing on the second feature data corresponding to n sub-video data. The fused feature data is then input into the network layer (such as a fully connected layer) after the feature extraction layer in the pre-trained anomaly detection model. The fully connected layer (FC layer) can perform detection processing on the target video data based on the fused feature data to obtain the type label of the target video data.

[0041] In S208, based on the type label of the target video data, it is determined whether the user image data contained in the target video data is synthetic data, and an anomaly detection result for the target video data is obtained.

[0042] In practice, if the server determines that the user's image data contained in the target video data is synthetic data based on the type label of the target video data, then the anomaly detection result for the target video data is synthetic data.

[0043] Since the feature data obtained by the server through the temporal segmentation network for feature extraction of the target video data is more robust, the obtained feature data can improve the classification accuracy of the target video data (that is, improve the accuracy of determining the type label of the target video data).

[0044] In this context, the temporal segmentation network refers to the server first splitting the target video data into multiple sub-video data using temporal information, then extracting first feature data to represent spatial information and second feature data to represent temporal information from the sub-video data, and finally performing feature fusion processing on the first and second feature data to obtain the type label of the target video data.

[0045] Therefore, the server can perform risk detection and risk control based on the anomaly detection results of the target video data. For example, in the case of synthetic image detection for identity recognition, if the anomaly detection result of the target video data is synthetic data, the server can determine that the identity authentication has failed and send the preset alarm information to the terminal device.

[0046] This specification provides a data processing method that acquires target video data to be detected and splits it into multiple sub-video data, including user image data. Based on the feature extraction layer of a pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Then, based on the feature extraction layer of the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first and second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer of the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data. Based on the type label of the target video data, it is determined whether the user image data included in the target video data is synthetic data, thus obtaining an anomaly detection result for the target video data. In this way, on the one hand, the server can use video data as input instead of the original single-frame image input, thereby enabling the anomaly detection model to extract richer spatiotemporal information to assist in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data using an anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user's image data contained in the target video data is synthetic data, which can improve the detection accuracy of whether the image to be detected is a synthetic image, and thus improve the accuracy of face recognition.

[0047] Example 2

[0048] like Figure 4As shown in the embodiments of this specification, a data processing method is provided. The execution subject of this method can be a server, which can be a standalone server or a server cluster composed of multiple servers. The method specifically includes the following steps:

[0049] In S402, historical video data used to train the anomaly detection model and the corresponding type labels for the historical video data are obtained, and the historical video data is split into multiple sub-data.

[0050] Among them, the type label corresponding to the historical video data can be used to characterize whether the image data of the user contained in the historical video data is synthetic data.

[0051] In practice, historical video data can be video data collected by the terminal device during the model training cycle that corresponds to the anomaly detection model. For example, assuming that the anomaly detection model is a model that detects whether the user image data contained in video instructions is synthetic data, the terminal device can send the collected user input video instruction data to the server during the model training cycle (such as the last month, the last three months, etc.). The server can store the video data containing user image data in the received video instruction data into the database corresponding to the anomaly detection model, and when the model training cycle is reached, select a preset number of video data from the database to train the detection model. The selected video data is the historical video data.

[0052] In addition, to improve the model training effect, a preset number of video data containing synthetic data can be obtained for training the anomaly detection model. That is, the historical video data used to train the anomaly detection model can contain a preset number of negative sample data.

[0053] In addition, the feature extraction layer can include multiple feature extraction modules. The network structure and initial parameters of the multiple feature extraction modules are the same. During the training of the anomaly detection model, the consistency of parameters of the multiple feature extraction modules can be ensured through weight sharing.

[0054] The server can split historical video data into multiple sub-data sets based on the number of feature extraction modules, for example, such as... Figure 5 As shown, assuming the feature extraction layer includes three feature extraction modules, the server can split the historical video data into three sub-data of equal duration based on the temporal information of the historical video data.

[0055] In S404, the feature extraction layer based on the anomaly detection model performs image feature extraction processing on the sub-data to obtain the third feature data corresponding to the sub-data, and performs temporal feature extraction processing on the sub-data based on the feature extraction layer of the anomaly detection model to obtain the fourth feature data corresponding to the sub-data.

[0056] In implementation, such as Figure 5 As shown, the server can perform image feature extraction and temporal feature extraction processing on the sub-data corresponding to the feature extraction module based on the feature extraction module, so as to obtain the third feature data and the fourth feature data corresponding to the sub-data.

[0057] Furthermore, in practical applications, the processing method of S404 can vary. The following is one optional implementation method, which can be found in steps one to two below:

[0058] Step 1: Based on the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the third feature data corresponding to the sub-data.

[0059] In implementation, the server can obtain the third feature data corresponding to the sub-data through the following steps A1 to A3:

[0060] In step A1, based on the first convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the first sub-feature data.

[0061] The first extraction network of the feature extraction module may include a first preset number of first convolutional layers and a second preset number of second convolutional layers. The dimension of the second convolutional layer is greater than the dimension of the first convolutional layer. For example, the first convolutional layer may be a two-dimensional convolutional layer and the second convolutional layer may be a three-dimensional convolutional layer. The ratio of the number of the first convolutional layer to the number of the second convolutional layer may vary depending on the actual application scenario. This specification does not specifically limit the first preset number, the second preset number, or the ratio of the first preset number to the second preset number in the embodiments.

[0062] In implementation, the server can fuse the feature data obtained by image feature extraction processing of the sub-data corresponding to the feature extraction module based on the first convolutional layer of the first extraction network and the feature data obtained by attention feature extraction processing of the sub-data corresponding to the feature extraction module based on the attention layer of the first extraction network to obtain the first sub-feature data.

[0063] In step A2, based on the second convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the second sub-feature data.

[0064] In step A3, based on the first sub-feature data and the second sub-feature data, the third feature data corresponding to the sub-data is determined.

[0065] In implementation, taking the feature extraction module 1 performing feature extraction processing on sub-data 1 as an example, such as... Figure 6 As shown, the first extraction network may include two first convolutional layers and two second convolutional layers. The server can input sub-data 1 into the first convolutional layer 1 of the feature extraction module, and then input the output of the first convolutional layer 1 into the first convolutional layer 2 to obtain the first sub-feature data corresponding to sub-data 1. At the same time, the server can input sub-data 1 into the second convolutional layer 1 of the feature extraction module, and then input the output of the second convolutional layer 1 into the second convolutional layer 2 to obtain the second sub-feature data corresponding to sub-data 1. Finally, by performing feature fusion processing on the first sub-feature data and the second sub-feature data, the third feature data corresponding to sub-data 1 can be obtained.

[0066] Since processing data using only high-dimensional convolutional layers (such as 3D convolutional layers) results in high memory consumption, while processing data using only low-dimensional convolutional layers (such as 2D convolutional layers) results in relatively weak feature extraction robustness, the fusion of the first and second convolutional layers can balance these issues. That is, the fusion convolutional structure can extract more robust spatiotemporal features, thereby improving the accuracy of subsequent data processing.

[0067] In addition, an attention layer (such as an attention module) can be added between every two first convolutional layers to enhance the feature aggregation effect of sensitive areas for synthetic image forgery attacks while extracting features.

[0068] Step 2: Based on the second extraction network of the feature extraction module, perform temporal feature extraction processing on the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data.

[0069] In implementation, the server can use the fourth feature data corresponding to the sub-data in steps B1 to B2 as follows:

[0070] In step B1, the optical flow information corresponding to the sub-data is obtained.

[0071] In practice, the server can calculate the optical flow information corresponding to the sub-data, and the optical flow information can be used to characterize the temporal information of the sub-data.

[0072] In step B2, based on the second extraction network of the feature extraction module, the optical flow information of the sub-data corresponding to the feature extraction module is subjected to temporal feature extraction processing to obtain the fourth feature data corresponding to the sub-data.

[0073] In implementation, the server can extract the RGB information of the sub-data and input it into the first extraction network of the feature extraction module. The first extraction network performs image feature extraction processing on the RGB information of the sub-data to obtain third feature data that can characterize the spatial information of the sub-data. The server then inputs the optical flow information corresponding to the sub-data into the second extraction network of the feature extraction module. The second extraction network performs temporal feature extraction processing on the optical flow information of the sub-data to obtain fourth feature data that can characterize the temporal information of the sub-data.

[0074] Furthermore, when the duration of historical video data exceeds a preset duration threshold, and / or the frequency of temporal information changes in historical video data determined based on optical flow information is higher than a preset frequency threshold, the network structure of the second extraction network can be the same as that of the first extraction network. That is, the second extraction network can also include a first preset number of first convolutional layers and a second preset number of second convolutional layers. Additionally, an attention layer can be added to the second extraction network.

[0075] In S406, the third and fourth feature data corresponding to the sub-data are input into the network layer after the feature extraction layer in the anomaly detection model to perform detection processing on the third and fourth feature data corresponding to the sub-data, and obtain the predicted label of the historical video data.

[0076] In practical applications, the processing method of S406 described above can vary. The following is one optional implementation method, which can be found in steps one to two below:

[0077] Step 1: Perform feature fusion processing on the third feature data corresponding to multiple sub-data to obtain the fifth feature data, and perform feature fusion processing on the fourth feature data corresponding to multiple sub-data to obtain the sixth feature data.

[0078] Step two: Input the fifth and sixth feature data corresponding to the sub-data into the network layer after the feature extraction layer in the anomaly detection model to perform detection processing on the fifth and sixth feature data corresponding to the sub-data and obtain the predicted labels of the historical video data.

[0079] In S408, the anomaly detection model is iteratively trained based on the type labels and predicted labels of historical video data until the anomaly detection model converges, resulting in the trained anomaly detection model.

[0080] In implementation, the processing method of S408 can vary. The following is one optional implementation method, which can be found in steps one to two below:

[0081] Step 1: Based on the type labels and predicted labels of historical video data, if it is determined that the anomaly detection model has not converged, the parameters of the first extraction network of the feature extraction module are updated to obtain the updated anomaly detection model.

[0082] Specifically, when the duration of historical video data exceeds a preset duration threshold, and / or the frequency of temporal information changes in historical video data is determined to be higher than a preset frequency threshold based on the optical flow information of historical video data, the parameters of the first extraction network and the second extraction network of the feature extraction module are updated to obtain the updated anomaly detection model.

[0083] In practice, since the temporal information in video data has little impact on determining whether the user images contained in the video data are synthetic data, in order to improve the model training efficiency, the server can adjust the network parameters of the first extraction network of each feature extraction module only during reverse tuning when training the anomaly detection model.

[0084] When the duration of historical video data exceeds a preset duration threshold, and / or the frequency of temporal information changes in historical video data determined based on optical flow information is higher than a preset frequency threshold, it can be considered that the temporal information in the video data has a significant impact on determining whether the user images contained in the video data are synthetic data. Therefore, in order to improve the accuracy of model training, the network parameters of the first and second extraction networks of each feature extraction module can be adjusted during reverse tuning.

[0085] Step 2: Continue iteratively training the updated anomaly detection model based on historical video data until the anomaly detection model converges, thus obtaining the trained anomaly detection model.

[0086] In S202, the target video data to be detected is acquired, and the target video data is split into multiple sub-video data.

[0087] The target video data may include the user's image data.

[0088] In practice, the server can split the target video data based on the number of feature extraction modules to obtain multiple sub-video data.

[0089] In S204, based on the feature extraction layer in the pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain the first feature data corresponding to the sub-video data. Then, based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain the second feature data corresponding to the sub-video data.

[0090] In S206, the first feature data and the second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data, and obtain the type label of the target video data.

[0091] In S208, based on the type label of the target video data, it is determined whether the user image data contained in the target video data is synthetic data, and an anomaly detection result for the target video data is obtained.

[0092] In implementation, firstly, the server, based on the main framework of a temporal segmentation network, temporally splits the input target video data into multiple equal-length sub-video data. Optical flow information is calculated for each sub-video data. Thus, spatial information of the sub-video data can be extracted using its RGB information, and temporal information can be extracted using its corresponding optical flow information. Subsequently, multiple weight-sharing feature extraction modules can be used for feature extraction. Then, the server can perform spatiotemporal feature fusion (i.e., feature fusion of the features output by the first extraction network) and temporal feature fusion (i.e., feature fusion of the features output by the second extraction network) on the features extracted by the multiple feature extraction modules. Finally, the server can use the fused features to connect to a fully connected (FC) layer for final anomaly detection processing (i.e., synthetic data detection, classification, and prediction). The anomaly detection model's structure enables end-to-end training and prediction.

[0093] In addition, during the training of the anomaly detection model, negative sample datasets (i.e., video data containing synthetic data) can be introduced for optimization, and multi-task loss function constraints can be used during the training process.

[0094] The feature extraction module can be a fusion convolutional module containing convolutional layers of different dimensions. For example, the input sub-video data can be subjected to two 2D convolutions and two consecutive 3D convolutions, and finally classified and predicted after passing through the FC layer.

[0095] In this way, the anomaly detection model can fully extract the spatiotemporal information of user image data in video data, thereby obtaining more robust features, making an important contribution to improving the accuracy of the final synthetic data detection and recognition, and enabling generalized deployment in open scenarios.

[0096] This specification provides a data processing method that acquires target video data to be detected and splits it into multiple sub-video data, including user image data. Based on the feature extraction layer of a pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Then, based on the feature extraction layer of the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first and second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer of the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data. Based on the type label of the target video data, it is determined whether the user image data included in the target video data is synthetic data, thus obtaining an anomaly detection result for the target video data. In this way, on the one hand, the server can use video data as input instead of the original single-frame image input, thereby enabling the anomaly detection model to extract richer spatiotemporal information to assist in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data using an anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user's image data contained in the target video data is synthetic data, which can improve the detection accuracy of whether the image to be detected is a synthetic image, and thus improve the accuracy of face recognition.

[0097] Example 3

[0098] The above describes the data processing method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a data processing device, such as... Figure 7 As shown.

[0099] The data processing device includes: a first acquisition module 701, a first extraction module 702, a first detection module 703, and a data judgment module 704, wherein:

[0100] The first acquisition module 701 is used to acquire target video data to be detected and split the target video data into multiple sub-video data, wherein the target video data includes the user's image data;

[0101] The first extraction module 702 is used to perform image feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and to perform temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data.

[0102] The first detection module 703 is used to input the first feature data and the second feature data corresponding to the sub-video data into the network layer after the feature extraction layer in the pre-trained anomaly detection model, so as to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data to obtain the type label of the target video data.

[0103] A data judgment module 704 is used to determine whether the user image data contained in the target video data is synthetic data based on the type label of the target video data, and to obtain an anomaly detection result for the target video data. In this embodiment of the specification, the apparatus further includes:

[0104] The second acquisition module is used to acquire historical video data used to train the anomaly detection model, as well as type labels corresponding to the historical video data, and to split the historical video data into multiple sub-data. The type labels corresponding to the historical video data are used to characterize whether the image data of the user contained in the historical video data is synthetic data.

[0105] The second extraction module is used to perform image feature extraction processing on the sub-data based on the feature extraction layer of the anomaly detection model to obtain third feature data corresponding to the sub-data, and to perform temporal feature extraction processing on the sub-data based on the feature extraction layer of the anomaly detection model to obtain fourth feature data corresponding to the sub-data.

[0106] The second detection module is used to input the third and fourth feature data corresponding to the sub-data into the network layer after the feature extraction layer in the anomaly detection model, so as to perform detection processing on the third and fourth feature data corresponding to the sub-data and obtain the predicted label of the historical video data.

[0107] The model training module is used to iteratively train the anomaly detection model based on the type labels and prediction labels of the historical video data until the anomaly detection model converges, thus obtaining the trained anomaly detection model.

[0108] In the embodiments of this specification, the feature extraction layer includes multiple feature extraction modules, and the network structure and initial parameters of the multiple feature extraction modules are the same. The second acquisition module is used for:

[0109] Based on the number of feature extraction modules, the historical video data is split to obtain the multiple sub-data.

[0110] The feature extraction layer based on the anomaly detection model performs image feature extraction processing on the sub-data to obtain third feature data corresponding to the sub-data, and performs temporal feature extraction processing on the sub-data based on the feature extraction layer of the anomaly detection model to obtain fourth feature data corresponding to the sub-data, including:

[0111] Based on the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the third feature data corresponding to the sub-data. Based on the second extraction network of the feature extraction module, temporal feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data.

[0112] In the embodiments of this specification, the second acquisition module is used for:

[0113] Obtain the optical flow information corresponding to the sub-data;

[0114] Based on the second extraction network of the feature extraction module, temporal feature extraction processing is performed on the optical flow information of the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data.

[0115] In the embodiments of this specification, the second detection module is used for:

[0116] The third feature data corresponding to the multiple sub-data are subjected to feature fusion processing to obtain the fifth feature data, and the fourth feature data corresponding to the multiple sub-data are subjected to feature fusion processing to obtain the sixth feature data;

[0117] The fifth and sixth feature data corresponding to the sub-data are input into the network layer after the feature extraction layer in the anomaly detection model to perform detection processing on the fifth and sixth feature data corresponding to the sub-data, thereby obtaining the predicted label of the historical video data.

[0118] In the embodiments of this specification, the model training module is used for:

[0119] If, based on the type label and predicted label of the historical video data, it is determined that the anomaly detection model has not converged, the parameters of the first extraction network of the feature extraction module are updated to obtain the updated anomaly detection model.

[0120] Based on the historical video data, the updated anomaly detection model is iteratively trained until the anomaly detection model converges, resulting in the trained anomaly detection model.

[0121] In the embodiments of this specification, the model training module is used for:

[0122] When the duration of the historical video data exceeds a preset duration threshold, and / or the frequency of temporal information change of the historical video data is determined to be higher than a preset frequency threshold based on the optical flow information of the historical video data, the parameters of the first extraction network and the second extraction network of the feature extraction module are updated to obtain the updated anomaly detection model.

[0123] In this embodiment of the specification, the first extraction network of the feature extraction module includes a first preset number of first convolutional layers and a second preset number of second convolutional layers, wherein the dimension of the second convolutional layer is greater than the dimension of the first convolutional layer. The second extraction module is used for:

[0124] Based on the first convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the first sub-feature data;

[0125] Based on the second convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the second sub-feature data.

[0126] Based on the first sub-feature data and the second sub-feature data, the third feature data corresponding to the sub-data is determined.

[0127] In the embodiments of this specification, the first extraction network further includes an attention layer, and the second extraction module is used for:

[0128] The first sub-feature data is obtained by fusing the feature data obtained by performing image feature extraction processing on the sub-data corresponding to the feature extraction module based on the first convolutional layer of the first extraction network and the feature data obtained by performing attention feature extraction processing on the sub-data corresponding to the feature extraction module based on the attention layer of the first extraction network.

[0129] This specification provides a data processing apparatus that acquires target video data to be detected and splits it into multiple sub-video data, including user image data. Based on a feature extraction layer in a pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Then, based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first and second feature data corresponding to the sub-video data are input into a network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data. Based on the type label, it is determined whether the user image data included in the target video data is synthetic data, thus obtaining an anomaly detection result for the target video data. In this way, on the one hand, the server can use video data as input instead of the original single-frame image input, thereby enabling the anomaly detection model to extract richer spatiotemporal information to assist in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data using an anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user's image data contained in the target video data is synthetic data, which can improve the detection accuracy of whether the image to be detected is a synthetic image, and thus improve the accuracy of face recognition.

[0130] Example 4

[0131] Following the same line of thought, embodiments of this specification also provide a data processing device, such as... Figure 8 As shown.

[0132] Data processing devices can vary considerably due to differences in configuration or performance. They may include one or more processors 801 and memory 802, with memory 802 storing one or more application programs or data. Memory 802 can be temporary or persistent storage. The application programs stored in memory 802 may include one or more modules (not shown), each module including a series of computer-executable instructions for the data processing device. Furthermore, processor 801 may be configured to communicate with memory 802 and execute the series of computer-executable instructions stored in memory 802 on the data processing device. The data processing device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, and one or more keyboards 806.

[0133] Specifically, in this embodiment, the data processing device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the data processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0134] Acquire target video data to be detected, and split the target video data into multiple sub-video data, wherein the target video data includes the user's image data;

[0135] Based on the feature extraction layer in the pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data.

[0136] The first feature data and the second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data, and obtain the type label of the target video data.

[0137] Based on the type label of the target video data, it is determined whether the user image data contained in the target video data is synthetic data, and an anomaly detection result for the target video data is obtained.

[0138] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the data processing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0139] This specification provides a data processing device that acquires target video data to be detected and splits it into multiple sub-video data, including user image data. Based on the feature extraction layer of a pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Then, based on the feature extraction layer of the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first and second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer of the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data. Based on the type label of the target video data, it is determined whether the user image data included in the target video data is synthetic data, thus obtaining an anomaly detection result for the target video data. In this way, on the one hand, the server can use video data as input instead of the original single-frame image input, thereby enabling the anomaly detection model to extract richer spatiotemporal information to assist in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data using an anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user's image data contained in the target video data is synthetic data, which can improve the detection accuracy of whether the image to be detected is a synthetic image, and thus improve the accuracy of face recognition.

[0140] Example 5

[0141] This specification also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described data processing method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may include, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0142] This specification provides a computer-readable storage medium that acquires target video data to be detected and splits the target video data into multiple sub-video data, including user image data. Based on the feature extraction layer of a pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Then, based on the feature extraction layer of the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first and second feature data corresponding to the sub-video data are input into a network layer after the feature extraction layer of the pre-trained anomaly detection model to perform detection processing on the first and second feature data corresponding to the sub-video data, obtaining a type label for the target video data. Based on the type label of the target video data, it is determined whether the user image data included in the target video data is synthetic data, thus obtaining an anomaly detection result for the target video data. In this way, on the one hand, the server can use video data as input instead of the original single-frame image input, thereby enabling the anomaly detection model to extract richer spatiotemporal information to assist in the final synthetic image detection processing. On the other hand, by performing image feature extraction and temporal feature extraction on each sub-video data in the target video data using an anomaly detection model, more robust spatiotemporal features can be extracted, thereby improving the classification accuracy of the target video data. Therefore, based on the type label of the target video data, it is possible to accurately determine whether the user's image data contained in the target video data is synthetic data, which can improve the detection accuracy of whether the image to be detected is a synthetic image, and thus improve the accuracy of face recognition.

[0143] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0144] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0145] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0146] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0147] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0148] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] The embodiments described herein are illustrated with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0150] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0151] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0152] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0153] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0154] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0156] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0158] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0159] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A data processing method, comprising: Acquire target video data to be detected, and split the target video data into multiple sub-video data, wherein the target video data includes the user's image data; Based on the feature extraction layer in the pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first feature data and the second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data, and obtain the type label of the target video data. Based on the type label of the target video data, it is determined whether the user image data contained in the target video data is synthetic data, and an anomaly detection result for the target video data is obtained. The feature extraction layer includes a first extraction network for image feature extraction processing of video data and a second extraction network for temporal feature extraction processing of video data. The network layer is used to determine the type label of the target video data based on the fused feature data. The fused feature data is obtained by performing feature fusion processing on the feature data extracted by the first extraction network and the feature data extracted by the second extraction network.

2. The method according to claim 1, before performing image feature extraction processing on the sub-video data in the feature extraction layer of the pre-trained anomaly detection model to obtain the first feature data corresponding to the sub-video data, further comprising: Obtain historical video data used to train the anomaly detection model, as well as type labels corresponding to the historical video data, and split the historical video data into multiple sub-data. The type labels corresponding to the historical video data are used to characterize whether the image data of the user contained in the historical video data is synthetic data. Based on the feature extraction layer of the anomaly detection model, image feature extraction processing is performed on the sub-data to obtain third feature data corresponding to the sub-data, and temporal feature extraction processing is performed on the sub-data based on the feature extraction layer of the anomaly detection model to obtain fourth feature data corresponding to the sub-data. The third and fourth feature data corresponding to the sub-data are input into the network layer after the feature extraction layer in the anomaly detection model to detect and process the third and fourth feature data corresponding to the sub-data, thereby obtaining the predicted label of the historical video data. Based on the type labels and predicted labels of the historical video data, the anomaly detection model is iteratively trained until the anomaly detection model converges, resulting in the trained anomaly detection model.

3. The method according to claim 2, wherein the feature extraction layer comprises multiple feature extraction modules, the network structure and initial parameters of the multiple feature extraction modules are identical, and the step of splitting the historical video data into multiple sub-data includes: Based on the number of feature extraction modules, the historical video data is split to obtain the multiple sub-data; The feature extraction layer based on the anomaly detection model performs image feature extraction processing on the sub-data to obtain third feature data corresponding to the sub-data, and performs temporal feature extraction processing on the sub-data based on the feature extraction layer of the anomaly detection model to obtain fourth feature data corresponding to the sub-data, including: Based on the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the third feature data corresponding to the sub-data. Based on the second extraction network of the feature extraction module, temporal feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data.

4. The method according to claim 3, wherein the second extraction network based on the feature extraction module performs temporal feature extraction processing on the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data, comprising: Obtain the optical flow information corresponding to the sub-data; Based on the second extraction network of the feature extraction module, temporal feature extraction processing is performed on the optical flow information of the sub-data corresponding to the feature extraction module to obtain the fourth feature data corresponding to the sub-data.

5. The method according to claim 4, wherein inputting the third feature data and fourth feature data corresponding to the sub-data into the network layer after the feature extraction layer in the anomaly detection model to perform detection processing on the third feature data and fourth feature data corresponding to the sub-data to obtain the predicted label of the historical video data includes: The third feature data corresponding to the multiple sub-data are subjected to feature fusion processing to obtain the fifth feature data, and the fourth feature data corresponding to the multiple sub-data are subjected to feature fusion processing to obtain the sixth feature data; The fifth and sixth feature data corresponding to the sub-data are input into the network layer after the feature extraction layer in the anomaly detection model to perform detection processing on the fifth and sixth feature data corresponding to the sub-data, thereby obtaining the predicted label of the historical video data.

6. The method according to claim 5, wherein iteratively training the anomaly detection model based on the type labels and predicted labels of the historical video data until the anomaly detection model converges to obtain the trained anomaly detection model, comprises: If, based on the type label and predicted label of the historical video data, it is determined that the anomaly detection model has not converged, the parameters of the first extraction network of the feature extraction module are updated to obtain the updated anomaly detection model. Based on the historical video data, the updated anomaly detection model is iteratively trained until the anomaly detection model converges, resulting in the trained anomaly detection model.

7. The method according to claim 6, wherein updating the parameters of the first extraction network of the feature extraction module to obtain the updated anomaly detection model includes: When the duration of the historical video data exceeds a preset duration threshold, and / or the frequency of temporal information change of the historical video data is determined to be higher than a preset frequency threshold based on the optical flow information of the historical video data, the parameters of the first extraction network and the second extraction network of the feature extraction module are updated to obtain the updated anomaly detection model.

8. The method according to claim 7, wherein the first extraction network of the feature extraction module includes a first preset number of first convolutional layers and a second preset number of second convolutional layers, the dimension of the second convolutional layers being greater than the dimension of the first convolutional layers, and the step of performing image feature extraction processing on the sub-data corresponding to the feature extraction module based on the first extraction network of the feature extraction module to obtain the third feature data corresponding to the sub-data includes: Based on the first convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the first sub-feature data; Based on the second convolutional layer of the first extraction network of the feature extraction module, image feature extraction processing is performed on the sub-data corresponding to the feature extraction module to obtain the second sub-feature data. Based on the first sub-feature data and the second sub-feature data, the third feature data corresponding to the sub-data is determined.

9. The method according to claim 8, wherein the first extraction network further comprises an attention layer, and the first convolutional layer of the first extraction network based on the feature extraction module performs image feature extraction processing on the sub-data corresponding to the feature extraction module to obtain first sub-feature data, including: The first sub-feature data is obtained by fusing the feature data obtained by performing image feature extraction processing on the sub-data corresponding to the feature extraction module based on the first convolutional layer of the first extraction network and the feature data obtained by performing attention feature extraction processing on the sub-data corresponding to the feature extraction module based on the attention layer of the first extraction network.

10. A data processing apparatus, comprising: The first acquisition module is used to acquire target video data to be detected and to split the target video data into multiple sub-video data, wherein the target video data includes the user's image data; The first extraction module is used to perform image feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain first feature data corresponding to the sub-video data, and to perform temporal feature extraction processing on the sub-video data based on the feature extraction layer in the pre-trained anomaly detection model to obtain second feature data corresponding to the sub-video data. The first detection module is used to input the first feature data and the second feature data corresponding to the sub-video data into the network layer after the feature extraction layer in the pre-trained anomaly detection model, so as to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data and obtain the type label of the target video data. The data judgment module is used to determine whether the user image data contained in the target video data is synthetic data based on the type label of the target video data, and to obtain the anomaly detection result for the target video data; The feature extraction layer includes a first extraction network for image feature extraction processing of video data and a second extraction network for temporal feature extraction processing of video data. The network layer is used to determine the type label of the target video data based on the fused feature data. The fused feature data is obtained by performing feature fusion processing on the feature data extracted by the first extraction network and the feature data extracted by the second extraction network.

11. A data processing apparatus, the data processing apparatus comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Acquire target video data to be detected, and split the target video data into multiple sub-video data, wherein the target video data includes the user's image data; Based on the feature extraction layer in the pre-trained anomaly detection model, image feature extraction processing is performed on the sub-video data to obtain first feature data corresponding to the sub-video data. Based on the feature extraction layer in the pre-trained anomaly detection model, temporal feature extraction processing is performed on the sub-video data to obtain second feature data corresponding to the sub-video data. The first feature data and the second feature data corresponding to the sub-video data are input into the network layer after the feature extraction layer in the pre-trained anomaly detection model to perform detection processing on the first feature data and the second feature data corresponding to the sub-video data, and obtain the type label of the target video data. Based on the type label of the target video data, it is determined whether the user image data contained in the target video data is synthetic data, and an anomaly detection result for the target video data is obtained. The feature extraction layer includes a first extraction network for image feature extraction processing of video data and a second extraction network for temporal feature extraction processing of video data. The network layer is used to determine the type label of the target video data based on the fused feature data. The fused feature data is obtained by performing feature fusion processing on the feature data extracted by the first extraction network and the feature data extracted by the second extraction network.

Images