Video account identification method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses a video account identification method and device, equipment, a medium and a product. The method comprises the following steps: acquiring at least one target video uploaded by a target video account; key features corresponding to the key information of the target video are extracted, and feature matching processing is carried out on the key features to obtain first features used for detecting the content authenticity of the target video; performing multi-modal feature extraction processing on the target video to obtain a second feature used for describing the content feature of the target video; and performing feature fusion processing on the first feature and the second feature to obtain a target fusion feature, and identifying the target video account according to the target fusion feature to obtain an identification result used for representing whether the target video account is a carrying account or not. According to the technical scheme of the embodiment of the invention, whether the video account is the carrying account or not can be accurately identified, and the accuracy and efficiency of carrying account identification are improved.

Description

Technical Field

[0001] This application relates to the field of video account technology, and more specifically, to a video account identification method, a video account identification device, an electronic device, a computer-readable storage medium, and a computer product. Background Technology

[0002] With the rapid development of the internet, more and more people are uploading their own videos to video platforms. Driven by the platform's incentive mechanisms, in order to quickly gain traffic and revenue, some people steal video content from other original accounts by means of copying, filming, or hacking, resulting in a large number of plagiarism accounts. These accounts not only infringe on the rights of original authors, but also threaten the content ecosystem of video platforms. In order to maintain the healthy development of the platform, especially to prevent the spread of plagiarism accounts and their variants, effective identification methods are urgently needed. However, the relevant technologies mainly rely on human experience, and it is very labor-intensive to judge whether an account is a plagiarism account through manual review and analysis, resulting in low identification efficiency. Summary of the Invention

[0003] The embodiments of this application provide a video account identification method, a video account identification device, an electronic device, a computer-readable storage medium, and a computer program product, which can accurately identify whether a video account is a reposted account, thereby improving the accuracy and efficiency of reposted account identification.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a video account identification method is provided, including:

[0006] The process involves: acquiring at least one target video uploaded by a target video account; extracting key features corresponding to key information of the target video and performing feature matching processing on the key features to obtain a first feature for detecting the authenticity of the target video content; performing multimodal feature extraction processing on the target video to obtain a second feature for describing the content characteristics of the target video; performing feature fusion processing on the first feature and the second feature to obtain a target fusion feature; and identifying the target video account based on the target fusion feature to obtain an identification result characterizing whether the target video account is a reposting account.

[0007] According to one aspect of the embodiments of this application, a video account recognition device is provided, comprising:

[0008] The acquisition module is used to acquire at least one target video uploaded by the target video account; the extraction module is used to extract key features corresponding to key information of the target video, and perform feature matching processing on the key features to obtain a first feature for detecting the authenticity of the content of the target video; the extraction module is also used to perform multimodal feature extraction processing on the target video to obtain a second feature for describing the content characteristics of the target video; the recognition module is used to perform feature fusion processing on the first feature and the second feature to obtain a target fusion feature, and recognize the target video account based on the target fusion feature to obtain a recognition result for characterizing whether the target video account is a plagiarism account.

[0009] In one embodiment of this application, the first feature includes a content detection feature for detecting whether the content of the target video is plagiarized content; the extraction module is further configured to extract key text features corresponding to key text in the target video, and key image features corresponding to key image frames in the target video; perform cross-modal alignment processing on the key text features and the key image features to obtain target text features and target key image features; search existing video feature libraries according to the target text features and target key image features to obtain feature similarity between the features of the target video and the features of the existing video feature library; and generate the content detection feature based on the feature similarity.

[0010] In one embodiment of this application, the extraction module is further configured to generate the content detection feature based on the feature similarity if the feature similarity is greater than a first preset similarity threshold; obtain a preset exemption whitelist if the feature similarity is greater than a second preset similarity threshold and less than the first preset similarity threshold; match the video attributes of the target video with the video attributes in the exemption whitelist, and obtain the content detection feature based on the matching result; wherein, the video attributes include at least one of video account and video segment.

[0011] In one embodiment of this application, the target video includes a face, and the first feature includes a target face feature for detecting whether the target video is a real person appearing in the video; the extraction module is further configured to perform face detection on each image frame of the target video to obtain a face image region of each image frame; perform key point detection on the face image region of each image frame, and perform face alignment processing based on the detected key point coordinates to obtain a target face image of each image frame; perform face recognition on the target face image of each image frame to obtain face features of each face; and perform clustering processing on the face features of each face to obtain the target face features.

[0012] In one embodiment of this application, the extraction module is further configured to calculate the facial similarity between each pair of faces based on the facial features of each face; aggregate similar facial features into the same cluster based on a set similarity threshold and the facial similarity; perform feature fusion on the facial features in each cluster to obtain a representative feature vector of each cluster, and generate the target facial features based on the representative feature vector of each cluster.

[0013] In one embodiment of this application, the second feature includes classification features for describing the content classification of the target video; the extraction module is used to perform multimodal feature extraction on the target video through the feature layer of a pre-trained classification model to obtain the classification features generated by the feature layer; wherein, the multimodality includes at least two of image modality, text modality, and audio modality.

[0014] In one embodiment of this application, the second feature includes the content features of the target video; the extraction module is further configured to acquire video modal information and text modal information of each image frame of the target video; acquire a pre-trained video encoder and text encoder, wherein the video encoder and the text encoder are obtained by training an intermediate video encoder and an intermediate text encoder using sample videos and sample text, and then fine-tuning the parameters of the intermediate video encoder and the intermediate text encoder using positive sample pairs and negative sample pairs; the positive sample pairs include matching sample videos and sample text, and the negative sample pairs include unmatched sample videos and sample text; the video encoder extracts features from the video modal information of each image frame to obtain video features, and the text encoder extracts features from the text information of each image frame to obtain text features; and the content features are obtained by feature fusion based on the video features and the text features.

[0015] In one embodiment of this application, the apparatus further includes a training module, configured to extract features from the sample video using an initial video encoder to obtain sample video features, and extract features from the sample text using an initial text encoder to obtain sample text features; fuse the sample video features and sample text features to obtain a first sample fusion feature, and calculate a loss function based on the first sample fusion feature; and adjust the parameters of the initial video encoder and the initial text encoder according to the loss function to obtain the intermediate video encoder and the intermediate text encoder.

[0016] In one embodiment of this application, the training module is further configured to extract video and text features for each positive sample pair and each negative sample pair using the intermediate video encoder and the intermediate text encoder, and generate a second sample fusion feature corresponding to the positive sample pair and a third sample fusion feature corresponding to the negative sample pair through feature fusion processing; keep the original parameters of the intermediate video encoder and the intermediate text encoder frozen, and introduce new parameters into the intermediate video encoder and the intermediate text encoder; construct a contrastive loss function based on the second sample fusion feature and the third sample fusion feature; and adjust the new parameters corresponding to the intermediate video encoder and the intermediate text encoder respectively according to the contrastive loss function to obtain the video encoder and the text encoder.

[0017] In one embodiment of this application, the recognition module is further configured to input the first feature and the second feature into a pre-trained account recognition model, so as to map the first feature and the second feature to a target feature space through the fusion layer of the account recognition model, and to perform clustering processing on the features of the target feature space to obtain target fusion features; and to perform classification prediction on the target fusion features through the classification layer of the account recognition model to obtain the recognition result.

[0018] In one embodiment of this application, the apparatus further includes a training module, which is used to acquire a labeled dataset, perform data cleaning on the labeled dataset through a data processing model trained in multiple rounds to obtain sample data, the sample data including each sample video under a sample account, and the account tag of the sample account; extract a first sample feature and a second sample feature for each sample video corresponding to the sample account; map the first sample feature and the second sample feature to a target feature space through the fusion layer of the model to be trained, and perform clustering processing on the features of the target feature space to obtain sample fusion features, and perform classification prediction on the sample fusion features through the classification layer of the model to be trained to obtain the sample account recognition result of the sample account; construct a loss function based on the sample account recognition result and the account tag of the sample account, and adjust the model parameters of the model to be trained according to the loss function to obtain the account recognition model.

[0019] In one embodiment of this application, the training module is further configured to divide the labeled dataset into training data and prediction data, perform feature extraction on the training data and prediction data respectively to obtain training sets and prediction sets, wherein both the training sets and prediction sets include account attribute information and video attribute information of the training accounts; train a set model based on the training set to obtain the data processing model, identify abnormal data in the prediction set through the data processing model, and perform data cleaning on the abnormal data to obtain a target prediction set; repeat the steps of dividing the labeled dataset, training the data processing model, and data cleaning until the number of training iterations of the data processing model reaches a preset number, and generate the sample data based on the target prediction set obtained from each data cleaning process.

[0020] According to one aspect of the embodiments of this application, an electronic device is provided, including one or more processors; and a storage device for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the video account recognition method as described above.

[0021] According to one aspect of the embodiments of this application, the embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform the video account identification method as described above.

[0022] According to one aspect of the embodiments of this application, this application provides a computer program product, including a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the video account recognition method described above.

[0023] In the technical solution provided by the embodiments of this application, at least one target video uploaded by the target video account is obtained; key features corresponding to the key information of the target video are extracted. The key features can accurately capture the most important elements in the video and are used quickly and accurately to detect the authenticity of the video content in feature matching processing, forming a first feature; multimodal feature extraction processing is performed on the target video to obtain a second feature used to describe the content characteristics of the target video. That is, the second feature can more comprehensively describe the video content through multimodal feature analysis, increasing the accuracy and robustness of detection; the first feature and the second feature are fused together to obtain a target fused feature that can not only reflect the authenticity of the video content but also describe the characteristics of the video content. Then, the target video account is identified based on the target fused feature, which can effectively identify the account-level behavior patterns and content quality, thereby obtaining an identification result used to characterize whether the target video account is a plagiarism account, greatly reducing the possibility of misjudgment and improving the accuracy and efficiency of plagiarism account identification.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0026] Figure 1 This is a schematic diagram of one implementation environment involved in this application;

[0027] Figure 2 This is a flowchart illustrating a video account recognition method in an exemplary embodiment of this application;

[0028] Figure 3 This is a schematic diagram illustrating another video account identification method as an exemplary embodiment of this application;

[0029] Figure 4 This is a schematic diagram illustrating another video account identification method as an exemplary embodiment of this application;

[0030] Figure 5 This is a flowchart illustrating another video account identification method as shown in an exemplary embodiment of this application;

[0031] Figure 6 This is a flowchart illustrating another video account identification method as shown in an exemplary embodiment of this application;

[0032] Figure 7 This is a schematic diagram illustrating the training of another encoder, as shown in an exemplary embodiment of this application;

[0033] Figure 8 This is a flowchart illustrating another video account identification method as shown in an exemplary embodiment of this application;

[0034] Figure 9 This is a flowchart illustrating another video account identification method as shown in an exemplary embodiment of this application;

[0035] Figure 10 This is a schematic diagram illustrating the entry point of the video number in an exemplary embodiment of this application;

[0036] Figure 11 This is an exemplary embodiment of the present application illustrating a video transfer account;

[0037] Figure 12 This is a schematic diagram illustrating model training in an exemplary embodiment of this application;

[0038] Figure 13 This is a schematic diagram illustrating the cleaning of dirty data, as shown in an exemplary embodiment of this application;

[0039] Figure 14 This is a schematic diagram illustrating feature extraction in an exemplary embodiment of this application;

[0040] Figure 15 This is a schematic diagram illustrating the extraction of brute-force attack results, as shown in an exemplary embodiment of this application;

[0041] Figure 16 This is a schematic diagram illustrating the extraction of facial features in an exemplary embodiment of this application;

[0042] Figure 17 This is a schematic diagram illustrating the extraction of classification features in an exemplary embodiment of this application;

[0043] Figure 18 This is a schematic diagram illustrating an account recognition model as an exemplary embodiment of this application;

[0044] Figure 19 This is a structural block diagram of a video account recognition device illustrated in an exemplary embodiment of this application;

[0045] Figure 20 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0048] The flowcharts shown in the accompanying diagrams are merely illustrative and do not necessarily include all content and operations, nor do they necessarily have to be executed in the described order. For example, some operations may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0049] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0050] The technical solutions of the embodiments of this application are described in detail below:

[0051] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment related to this application. The implementation environment includes a terminal 10 and a server 20.

[0052] Terminal 10 is used to record at least one target video uploaded by the target video account and send the uploaded target video to server 20.

[0053] Server 20 is used to identify whether an account is a plagiarism account by performing multi-dimensional feature analysis on the video content uploaded by the target video account. Specifically, it extracts key features corresponding to key information of the target video and performs feature matching processing on these key features to obtain a first feature used to detect the authenticity of the target video content; it then performs multi-modal feature extraction processing on the target video to obtain a second feature describing the content characteristics of the target video; finally, it performs feature fusion processing on the first and second features to obtain a target fused feature, and uses this target fused feature to identify the target video account to obtain an identification result characterizing whether the target video account is a plagiarism account.

[0054] The server 20 can also send the identification results to the terminal, so that the terminal can determine whether to ban or limit the target video account based on the identification results.

[0055] In some embodiments, the server 20 may also obtain the target video uploaded by the target video account, then obtain the first feature through feature matching, obtain the second feature through feature extraction, and obtain the target fusion feature through feature fusion, and identify the target video account based on the target fusion feature.

[0056] In some embodiments, the terminal 10 can also implement the video account recognition process independently, that is, the terminal 10 acquires the target video, performs feature matching, feature extraction, and feature fusion to obtain the target fused features, and then identifies the target video account.

[0057] The aforementioned terminal 10 can be any electronic device capable of displaying video, such as a smartphone, tablet, laptop, computer, smart voice interaction device, smart home appliance, vehicle terminal, or aircraft. The server 20 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This document does not impose any restrictions on this.

[0058] Terminal 10 and server 20 establish a communication connection via a network beforehand, enabling them to communicate with each other. The network can be a wired network or a wireless network, and this is not a limitation.

[0059] It should be noted that in the specific implementation of this application, the video involves objects. When the embodiments of this application are applied to specific products or technologies, permission or consent from the objects is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0060] The following details the various implementation details of the technical solutions in the embodiments of this application:

[0061] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a video account recognition method according to an embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. The video account identification method can include steps S210 to S240, which are detailed below:

[0062] S210. Obtain at least one target video uploaded by the target video account.

[0063] In this embodiment of the application, a target video account refers to an account that allows an object to publish and upload videos, and the object can also see videos published and uploaded by other objects through the target video account; wherein, there can be one or more target video accounts, and if there are multiple target video accounts, these multiple target video accounts are video accounts of the same video platform.

[0064] The target video account may upload one or more of the most recently uploaded videos, or videos that are not recently uploaded; it may also consist of some recently uploaded videos and some videos that are not recently uploaded, without any restrictions.

[0065] S220. Extract the key features corresponding to the key information of the target video, and perform feature matching processing on the key features to obtain the first feature used to detect the authenticity of the content of the target video.

[0066] In one example, the key information of the target video can be determined based on the video type of the target video. For example, if the target video is a music video, the key information includes the video content and the video sound; if the target video is a drama video (such as a TV / audio clip), the key information includes the video content, text, and sound, and the video content includes the facial information of the characters.

[0067] In another example, if the video duration of the target video is less than a preset time threshold, it indicates that the target video is a short video. The first n frames and the images in the middle time period of the target video can be used as key images to obtain the key information corresponding to the key images.

[0068] Depending on the key information, key features corresponding to the key information can be extracted through different feature extraction methods. Then, feature matching processing is performed on the key features. This can be feature matching between key features or feature matching between key features and other features to obtain the first feature.

[0069] In this embodiment of the application, the first feature is used to detect the authenticity of the target video content. The authenticity of the content includes the authenticity of the content (i.e., the information expressed by the content is true and reliable) and the originality of the content (i.e., it is not plagiarized or copied, but is independently created by the author).

[0070] S230. Perform multimodal feature extraction processing on the target video to obtain a second feature used to describe the content characteristics of the target video.

[0071] The target video includes multimodal information, where modality can be understood as different representations or acquisition methods of data. For example, images and text belong to different modalities. Multimodality includes at least two modalities. Multimodal feature extraction of the target video means extracting features from each of the multiple modal information separately. The resulting second feature can be the modal features corresponding to each of the multiple modal information. For example, it can be feature extraction from the video content of the target video, feature extraction from the text information of the target video, or feature extraction from the audio information of the target video. The first multimodal feature includes video features corresponding to the video content, text features corresponding to the text information, and audio features corresponding to the audio information. The second feature can also be a feature obtained by fusing multiple modal features, such as fusing video features, text features, and audio features to obtain the second feature.

[0072] In this application embodiment, the second feature is used to describe the content characteristics of the target video. The content characteristics refer to the unique properties or features exhibited by the content. In one example, the second feature includes content classification features and content inherent property features.

[0073] It should be noted that feature extraction processing is involved in both extracting key features and extracting multimodal features, but the feature extraction processing methods for key features and multimodal features are different.

[0074] S240. The first feature and the second feature are fused to obtain the target fused feature, and the target video account is identified based on the target fused feature to obtain the identification result used to characterize whether the target video account is a reposting account.

[0075] In this embodiment of the application, both the first feature and the second feature are extracted from the target video published by the target account, thus reflecting certain characteristics and behaviors of the account. The first feature helps to determine the authenticity of the account and the originality of the content, while the second feature helps to understand the content direction of the account. Therefore, the first feature and the second feature are fused to obtain the target fused feature. By analyzing the target fused feature, the behavior and content quality of the account can be better understood and evaluated. Then, the target video account is identified based on the target fused feature to obtain the identification result used to characterize whether the target video account is a plagiarized account. The plagiarized account refers to an account that steals videos from the original account by using screen recording, filming, or hacking methods and publishes them on its own account to gain traffic.

[0076] In one example, the first feature and the second feature can be fused using one feature fusion method to obtain the target fused feature, or multiple feature fusion methods can be used to fuse the first feature and the second feature to obtain the target fused feature; for example, fused feature A can be obtained by feature fusion method A, fused feature B can be obtained by feature fusion method B, and then fused feature A and fused feature B can be weighted and summed to obtain the final target fused feature.

[0077] In one example, the first and second features are input into a pre-trained account recognition model, and the recognition result is obtained by feature fusion and account recognition through the account recognition model.

[0078] In this embodiment, at least one target video uploaded by the target video account is obtained; key features corresponding to the key information of the target video are extracted. These key features can accurately capture the most important elements in the video and are used quickly and accurately in feature matching processing to detect the authenticity of the video content, forming a first feature; multimodal feature extraction processing is performed on the target video to obtain a second feature that describes the content characteristics of the target video. That is, through multimodal feature analysis, the second feature can more comprehensively describe the video content, increasing the accuracy and robustness of detection; the first feature and the second feature are fused together to obtain a target fused feature that not only reflects the authenticity of the video content but also describes the characteristics of the video content. Based on the target fused feature, the target video account is identified, which can effectively identify the account-level behavioral patterns and content quality, thereby obtaining an identification result to characterize whether the target video account is a plagiarism account. This significantly reduces the possibility of misjudgment and improves the accuracy and efficiency of plagiarism account identification.

[0079] In one embodiment of this application, another video account recognition method is provided, which can be applied to... Figure 1The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 3 As shown, this video account identification method is in Figure 2 Based on S210 to S240 shown, S220 is extended to S310 to S340, wherein the first feature includes a content detection feature for detecting whether the content of the target video is plagiarized content; S310 to S340 are described in detail below:

[0080] S310. Extract the key text features corresponding to the key text in the target video, and the key image features corresponding to the key image frames in the target video.

[0081] In this embodiment, key information includes key text and key image frames. Key text includes the video title and text content detected and identified from the video, such as subtitles or text converted from human speech. Key information also includes key image frames, which are key frames that can represent the video content. For example, key image frames are frames that contain obvious objects, scenes, or help express the theme of the video. Key image frames can be extracted from the target video through scene switching detection, motion detection, etc. For another example, key image frames include the first n image frames of the target video and image frames extracted from images in the middle time period.

[0082] In one example, a text encoder transforms key text into a high-dimensional key text feature, which includes the semantic information of the text; an image encoder transforms key image frames into image feature vectors; these vectors describe the visual content of the image.

[0083] In another example, key text is preprocessed, such as removing irrelevant content and standardizing text format; key image frames are preprocessed, including adjusting image size, normalizing, and converting to tensors to meet the input requirements of the CLIP model; the preprocessed key text and key image frames are then input.

[0084] S320. Perform cross-modal alignment processing on key text features and key image features to obtain target text features and target key image features.

[0085] Key text features and key image features come from different modalities (text and vision), but they describe the same video content; in order to ensure that these two features can jointly reflect the main content of the video, cross-modal alignment processing is required.

[0086] In one example, key text features and key image features can be input into a pre-trained cross-modal model, such as CLIP (Contrastive Language-Image Pre-training). Through this cross-modal model, text features and image features are mapped to the same feature space to obtain target text features and target key image features. In this space, similar text and image features will be closer together, ensuring alignment between different modalities.

[0087] In one example, the video encoder and text encoder in step S310 are encoders in the CLIP model, that is, the key text and key features are directly input into the CLIP model to obtain the target text features and target key image features.

[0088] S330. Based on the target text features and target key image features, retrieve the existing video feature database to obtain the feature similarity between the features of the target video and the features of the existing video feature database.

[0089] The aligned target text features and image features need to be matched with existing video feature libraries to determine whether the target video is similar to existing videos.

[0090] In one example, the target text features and the target key image features are in the same space. The target text features and the target key image features can be fused to obtain the target video features. The target video features can better reflect the main content of the video. The target video features can be compared with the video features in the existing video feature library, that is, the feature similarity between the target video features and the video features in the existing video feature library can be calculated. The existing video feature library is pre-established, and the method of obtaining the video features in the existing video feature library can be found in S310 to S330.

[0091] In another example, the target text features can be compared with the text features and image features in the existing video feature library. That is, the first similarity between the target text features and the text features in the existing video feature library can be calculated, and the second similarity between the target text features and the image features in the existing video feature library can be calculated. Then, the target key image features can be compared with the text features and image features in the existing video feature library to obtain the third and fourth similarities. Furthermore, the first to fourth similarities can be combined for a comprehensive evaluation, such as averaging or weighting the first to fourth similarities to obtain the final feature similarity, so as to more accurately determine whether the target video is plagiarized content.

[0092] S340. Generate content detection features based on feature similarity.

[0093] In the embodiments of this application, content detection features can be generated based on feature similarity, wherein the content detection features are represented as a single score or a vector, reflecting the overall similarity between the target video and the retrieved existing videos.

[0094] In one example, the generated content detection features may also include a flag indicating whether the target video meets the criteria for plagiarism (e.g., a flag of 1 indicates plagiarism, and a flag of 0 indicates non-plagiarism).

[0095] The identifier value can be determined based on feature similarity. If the feature similarity is greater than the first preset similarity threshold, it can be preliminarily determined that the content of the target video is not original and that the video is copied content, and the identifier value is determined to be 1.

[0096] In one example, if the feature similarity is less than or equal to the first preset similarity threshold, the target video can be directly determined to be an original video, not copied content, and the identifier value is set to 0.

[0097] In another example, if the feature similarity is greater than the second preset similarity threshold and less than the first preset similarity threshold, it indicates that the target video and the existing video are partially similar, such as both the target video and the existing video including common video segments. In this case, a whitelist mechanism can be used to exempt certain videos that are allowed to reuse some content, thus avoiding misjudgment of copied content. The first preset similarity threshold and the second preset similarity threshold can be flexibly adjusted according to the actual situation, such as the first preset similarity threshold being 70% and the second preset similarity threshold being 50%.

[0098] The process involves obtaining a preset exemption whitelist, matching the video attributes of the target video with the video attributes in the exemption whitelist, and obtaining content detection features based on the matching results. The video attributes include at least one of video account and video segment.

[0099] The matching result includes a successful match and a match recognition. If the video attributes of the target video successfully match the video attributes in the exemption whitelist, then the video content of the target video is not copied content, and the identifier value can be determined to be 0; otherwise, the match fails, and the identifier value can be determined to be 1.

[0100] For example, the exemption whitelist includes exempted video clips (such as public video clips) and exempted video accounts. If the video clip of the target video is an exempted clip in the whitelist, it means that the target video has video clips that are partially similar to existing videos and are allowed to be reused, so the identifier value can be determined to be 0. If the target video account of the target video is an exempted account in the whitelist, it means that the target video account is allowed to reuse some similar content from other videos, so the identifier value can be determined to be 0. If the video clip of the target video or the target video account is not in the whitelist, the identifier value is determined to be 1.

[0101] In other embodiments of this application, the exemption whitelist includes exempted videos; if the feature similarity is greater than a first preset similarity threshold, it is further determined that the target video is in the exemption whitelist, and the video can also be marked as exempted, with the identifier value determined to be 0; if the target video is not in the exemption whitelist, the identifier value is determined to be 1.

[0102] It should be noted that, Figure 3 For further details on S210, S230 to S240 shown, please refer to [link / reference]. Figure 2 S210, S230 to S240 shown will not be described again here.

[0103] In this embodiment, key text features and key image features are extracted. Cross-modal alignment processing is performed on the key text features and key image features to ensure alignment between different modalities. Then, an existing video feature library is searched to determine whether the target video is similar to an existing video. Under the content similarity detection and whitelist rules, the originality of the video can be effectively determined to obtain the first feature, thereby ensuring the accuracy and fairness of the first feature generation.

[0104] This application provides another method for video account recognition, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 4 As shown, this video account identification method is in Figure 2 Based on what is shown, Figure 2 The S220 shown is extended to S410-S440. The target video includes a face, and the first feature includes a target face feature used to detect whether the target video features a real person. S410-S440 are described in detail below:

[0105] S410. Perform face detection on each image frame of the target video to obtain the face image region of each image frame.

[0106] S420. Perform key point detection on the face image region of each image frame, and perform face alignment processing based on the detected key point coordinates to obtain the target face image of each image frame.

[0107] S430. Perform face recognition on the target face image of each image frame to obtain the face features of each face.

[0108] S440. Cluster the facial features of each face to obtain the target facial features.

[0109] In this embodiment of the application, the target video includes the face of a person. Face detection can be performed using a face detection model, including a single-shot multibox detector such as SSD (Single Shot MultiBox Detector), a detection model based on convolutional neural networks (CNN), a multi-task convolutional neural network (MTCNN), etc., to obtain rectangular bounding boxes of all faces in each image frame, identify the position and size of the face in the image, and use the rectangular bounding box as the face image region.

[0110] Then, a keypoint detection model, such as DLib, is used to detect multiple keypoints (e.g., 68 or 5 keypoints) on the face image of each image frame. Keypoints include the positions of the eyes, nose tip, mouth corners, chin, etc., to obtain the keypoint coordinates of each face image. Then, face alignment processing is performed based on the detected keypoint coordinates to obtain the target face image of each image frame. Specifically, based on the detected keypoints, the faces are aligned using affine transformation or perspective transformation to obtain the target face image. The goal of alignment is to adjust all faces to a uniform pose, such as aligning the eyes on the horizontal line and the mouth, to reduce errors caused by different face poses.

[0111] In this embodiment, a pre-trained face recognition model (such as ResNet, FaceNet, ArcFace, etc.) is used. The target face image is input into the model, and the corresponding face feature vector is output. The face feature vector is high-dimensional, usually a 128-dimensional or 512-dimensional floating-point vector, which can represent the uniqueness of the face. Then, the face feature vectors extracted from each frame are clustered to aggregate similar face features to obtain the target face features. It is then used to determine whether the same person appears in the video, thereby determining the authenticity of the video content.

[0112] In one example, the process of obtaining the target facial features includes:

[0113] Based on the facial features of each face, calculate the facial similarity between each pair of faces; based on the set similarity threshold and facial similarity, aggregate similar facial features into the same cluster; perform feature fusion on the facial features in each cluster to obtain the representative feature vector of each cluster, and generate the target facial features based on the representative feature vector of each cluster.

[0114] Clustering algorithms such as k-means clustering, hierarchical clustering (H-cluster), and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) can be used. These algorithms aggregate feature vectors with facial similarity higher than a preset similarity threshold to form different clusters, each representing a different face. Duplicate faces are grouped into the same cluster. Euclidean distance or cosine similarity is used to calculate the similarity between facial features during the clustering process.

[0115] For each cluster, feature fusion is performed on the facial features. For example, the mean of all facial features in the cluster can be taken as the representative feature vector of the cluster. This representative feature vector is the embedding of each deduplicated face.

[0116] In one example, clustering can yield one or more clusters. If the number of clusters is too small, such as less than a preset number, it means that almost all the faces in the video are from the same person. The target video can be considered to be a real video of the same person. The target video is a real video. The representative feature vectors of each cluster can be further fused, such as by taking the average value to obtain the target face features. The generated target face features also contain a flag, where a flag of 0 indicates that a real person is in the video and a flag of 1 indicates that a non-real person is in the video.

[0117] In another example, if the number of clusters is too large, such as exceeding the preset number, it means that the faces in the video are from different people, and the target video is a non-real person video with different people appearing in it. In this case, the representative feature vectors of each cluster can be concatenated to obtain the target face features. The identifier bit contained in the target face features is 0, which means that it is a non-real person appearing in it and the video is a copied video.

[0118] It should be noted that, Figure 4 For detailed information on S210, S230 to S240 shown, please refer to [link / reference]. Figure 2 S210, S230 to S240 shown will not be described again here.

[0119] In this embodiment, the process begins with face detection, which involves determining the face region, predicting key points of the face, extracting face features, and performing cluster analysis on the extracted features to remove duplicate or similar faces. This enables the server to effectively identify faces in the video and generate target face features to determine whether the video content is from a real person based on the representative feature vector of each cluster.

[0120] This application also provides another video account recognition method, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 5 As shown, in Figure 2 Based on what is shown, Figure 2 The S230 shown is expanded to S510. The second feature includes classification features used to describe the content classification of the target video. S510 is described in detail below:

[0121] S510. Multimodal features are extracted from the target video through the feature layer of the pre-trained classification model to obtain the classification features generated by the feature layer; wherein, the multimodal features include at least two of the following: image modality, text modality, and audio modality.

[0122] In this embodiment of the application, the classification model is pre-trained and used to determine the classification label of the video. The classification model includes a feature layer and a classification layer. The multimodal information of the target video is input into the feature layer of the classification model. The multimodal features of the target video are extracted through the feature layer. Classification features are generated based on the modal features of each modality. The classification features are input into the classification layer to obtain the classification label of the target video.

[0123] If a multimodal video includes both image and text modalities, image modal features are extracted through a feature layer. These image modal features include edge and texture information. Word vectors, which include semantic information, are also extracted from the text. Image content can be obtained through image modal features, and the expressed semantics can be obtained through word vectors. The extracted image modal features and word vectors are then mapped into a low-dimensional space to obtain classification features describing the content of the target video. Similarly, if a multimodal video also includes an audio modality, audio vectors, which include audio emotion information, are extracted through a feature layer. Based on the image and text, the audio emotion information is combined to further refine the content classification of the target video, resulting in classification features.

[0124] It should be noted that, Figure 5 For further details regarding S210-S220 and S240 shown, please refer to [link / reference]. Figure 2 S210 to S220 and S240 shown in the figure will not be described again here.

[0125] In this embodiment, multimodal features are extracted from the target video through the feature layer of the pre-trained classification model. The classification feature vector generated by the new feature layer can effectively reflect the classification category of the video, thus facilitating the determination of whether the video is a copied video by the consistency of the classification category.

[0126] In one embodiment of this application, another video account recognition method is also provided, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 6 As shown, this video account identification method is in Figure 2 Based on what is shown, Figure 2 S230 is extended to S610-S640. The second feature includes the content features of the target video, and S610-S640 are described in detail below:

[0127] S610. Obtain the video modal information and text modal information of each image frame of the target video.

[0128] In this embodiment of the application, the target video includes multiple image frames. The video modal information and text modal information of each image frame are obtained. The text modal information includes the subtitles included in the image frame and the text obtained by Automatic Speech Recognition (ASR).

[0129] S620. Obtain the pre-trained video encoder and text encoder. The video encoder and text encoder are obtained by training the intermediate video encoder and intermediate text encoder with sample videos and sample texts, and then fine-tuning the parameters of the intermediate video encoder and intermediate text encoder with positive sample pairs and negative sample pairs. Positive sample pairs include matching sample videos and sample texts, and negative sample pairs include unmatched sample videos and sample texts.

[0130] It is worth noting that in the embodiments of this application, the video encoder and the text encoder are pre-trained, including the intermediate video encoder and the intermediate text encoder obtained by pre-training based on sample videos and sample texts, and the intermediate video encoder and the intermediate text encoder obtained by fine-tuning based on positive sample pairs and negative sample pairs.

[0131] The sample videos and sample texts constitute a large dataset, while the positive and negative sample pairs constitute a small dataset. Positive sample pairs include matching sample videos and sample texts, such as a video and its correct descriptive text; negative sample pairs include mismatched sample videos and sample texts, such as a video and its irrelevant descriptive text.

[0132] In one example, the training steps for the intermediate video encoder and intermediate text encoder include:

[0133] The initial video encoder extracts features from the sample video to obtain sample video features, and the initial text encoder extracts features from the sample text to obtain sample text features. The sample video features and sample text features are then fused to obtain the first sample fused features, and a loss function is calculated based on the first sample fused features. The parameters of the initial video encoder and the initial text encoder are adjusted according to the loss function to obtain the intermediate video encoder and the intermediate text encoder.

[0134] like Figure 7 As shown, the initial video encoder and initial text encoder are obtained. Figure 7 As shown in the left figure, the sample video is input into the initial video encoder (Time encoder) for video feature extraction. It should be noted that the sample video, as well as the videos in the positive and negative samples, each consists of multiple video segments, each with corresponding time-series features. Therefore, the initial video encoder extracts not only the content features of the video but also its spatiotemporal features, thus obtaining the sample video features, such as 011001...; the sample text is input into the initial text encoder (Text encoder) for text feature extraction, resulting in the sample text features, such as 011000....

[0135] Feature fusion of sample video features and sample text features can be achieved by processing the sample video features and sample text features separately through transformation functions (such as fully connected layers or convolutional layers) to obtain new feature representations. The transformed features are then fused by summation or concatenation to obtain the first sample fused feature. The first sample fused feature is then input into a correspondence judgment module to determine whether the video and text match and to calculate the corresponding loss function. For example, the loss function can be constructed based on the matching result output by the correspondence judgment module and the difference between the matching labels between the sample video and sample text. Alternatively, the first loss can be constructed based on the difference between the matching result output by the correspondence judgment module and the matching labels, and the second loss can be constructed based on the similarity between the sample video features and sample text features. The first and second losses are then weighted and summed to obtain the loss function.

[0136] The parameters of the initial encoder and the initial text encoder are adjusted according to the loss function until the difference between the matching result and the matching label output by the correspondence determination module is less than the preset difference value, thus obtaining the intermediate video encoder and the intermediate text encoder.

[0137] The intermediate video encoder and intermediate text encoder are pre-trained on a large-scale dataset to obtain high-quality video and text feature representations. In order to optimize the intermediate video encoder and intermediate text encoder for short video application scenarios, the intermediate video encoder and intermediate text encoder are fine-tuned to better fit the needs of business scenarios.

[0138] In one example, the training steps for the video encoder and text encoder include:

[0139] For each positive and negative sample pair, video and text features are extracted using an intermediate video encoder and an intermediate text encoder. Feature fusion processing is then used to generate second sample fusion features for positive sample pairs and third sample fusion features for negative sample pairs. The original parameters of the intermediate video encoder and intermediate text encoder are kept frozen, and new parameters are introduced into the intermediate video encoder and intermediate text encoder. A contrastive loss function is constructed based on the second and third sample fusion features. The newly introduced parameters of the intermediate video encoder and intermediate text encoder are adjusted according to the contrastive loss function to obtain the video encoder and text encoder.

[0140] In this case, the number of positive sample pairs can be greater than the number of negative sample pairs. To enhance the robustness of positive sample pairs, in... Figure 7 As shown in the right-hand figure, for some positive sample pairs, such as positive sample pair A, the sample video in positive sample pair A is directly input into the intermediate video encoder to extract features of video content and video timing. The sample text in positive sample pair A is directly input into the intermediate text encoder to extract text features. Then, the features output by the intermediate video encoder and the features output by the intermediate text encoder are fused to obtain the second sample fusion feature corresponding to positive sample pair A. For some positive sample pairs, such as positive sample pair B, the sample video in positive sample pair B is first processed by random frame extraction. The randomly extracted sample video image is input into the intermediate video encoder to extract features of video content and video timing. The sample text in positive sample pair B is subjected to Easy Data Augmentation (EDA), including synonym replacement, random insertion, random swapping, and random deletion. The sample text after text data augmentation is input into the intermediate text encoder to extract text features. Then, the features output by the intermediate video encoder and the features output by the intermediate text encoder are fused to obtain the second sample fusion feature corresponding to positive sample pair B.

[0141] For negative sample pair C, the sample videos from negative sample pair C can be directly input into the intermediate video encoder to extract features from the video content and video timing. Similarly, the sample text from negative sample pair C can be directly input into the intermediate text encoder to extract text features. Then, the features output from the intermediate video encoder and the intermediate text encoder are fused to obtain the third sample fused feature corresponding to negative sample pair C. The feature fusion process for obtaining the second and third sample fused features is the same. Figure 7 The various fusion modules share parameters.

[0142] This application embodiment uses a Low-Rank Adaptation (LoRA) module to efficiently fine-tune the parameters of the intermediate video encoder and intermediate text encoder. LoRA introduces new parameters, such as a low-rank matrix, into the intermediate video encoder and intermediate text encoder. These new parameters are trainable, while the original parameters of the intermediate video encoder and intermediate text encoder remain fixed. That is, during training, only the new parameters are trained, while the original model parameters are frozen and not updated. In this way, fine-tuning of the model can be achieved while keeping the original model parameters unchanged.

[0143] In one example, by comparing positive and negative sample pairs, the similarity between similar sample pairs can be maximized while the similarity between dissimilar sample pairs can be minimized. Therefore, a contrastive loss function is constructed based on the second and third sample fusion features. For example, the first similarity between the second sample fusion features corresponding to positive sample pair A and the second sample fusion features corresponding to positive sample pair B is calculated, and the second similarity between the second sample fusion features corresponding to positive sample pair A and the third sample fusion features corresponding to negative sample pair C is calculated. A contrastive loss function is constructed based on the first and second similarities to maximize the first similarity and minimize the second similarity.

[0144] Then, the newly added parameters of the intermediate video encoder and intermediate text encoder are adjusted according to the contrastive loss function until the intermediate video encoder and intermediate text encoder converge, and the training is completed to obtain the final video encoder and text encoder.

[0145] S630. Video features are obtained by extracting video modal information from each image frame using a video encoder, and text features are obtained by extracting text information from each image frame using a text encoder.

[0146] S640. Content features are obtained by fusing video features and text features.

[0147] After fine-tuning, the video encoder and text encoder can obtain higher quality and richer video and text feature representations. Therefore, the video modal information of each image frame of the target video can be input into the video encoder to obtain video features, the text information of each image frame can be extracted to obtain text features, and the video features and text features can be fused, such as by weighted summation, to obtain the content features of the target video.

[0148] It should be noted that, Figure 6 For further details regarding S210-S220 and S240 shown, please refer to [link / reference]. Figure 2 S210 to S220 and S240 shown will not be described again here.

[0149] In this embodiment, an initial video / text encoder is first pre-trained on a large-scale dataset to obtain high-quality video and text feature representations. In the fine-tuning stage, contrastive learning is used to further optimize the intermediate video / text encoder on a small-scale dataset to make it more adaptable to the needs of specific tasks, so as to encode features more efficiently and accurately. Then, the video and text features of the target video are extracted through the video encoder and text encoder. Finally, the features are fused to obtain content features that can represent the video, so as to help the model better understand and distinguish the content of different accounts.

[0150] It is worth noting that, in one embodiment of this application, another video tag recognition method is also provided, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 8 As shown, this video tag recognition method is in Figure 2 Based on the example shown, S240 is extended to S810 to S820. S810 to S820 are described in detail below:

[0151] S810. Input the first feature and the second feature into the pre-trained account recognition model, so that the first feature and the second feature are mapped to the target feature space through the fusion layer of the account recognition model, and the features in the target feature space are clustered to obtain the target fused features.

[0152] S820. The target fusion features are classified and predicted through the classification layer of the account recognition model to obtain the recognition result.

[0153] In this application embodiment, the first feature includes a content detection feature for detecting whether the content of the target video is copied content and a target facial feature for detecting whether the target video features a real person; the second feature includes a classification feature for describing the content classification of the target video and the content feature of the target video itself.

[0154] The pre-trained account recognition model includes a fusion layer and a classification layer. The fusion layer includes a transformation function (Φ), which maps content inspection features, target face features, classification features, and content features to a new space. This transformation function can be any form of neural network, and its role is to transform the input data into a form more suitable for subsequent processing. The fusion layer also includes a symmetric function (ρ): which aggregates the transformed features (such as summation, averaging, etc.) to obtain the final output, which is the target fused feature. Due to the properties of the symmetric function, the aggregation result does not depend on the order of the input elements, thus enabling the handling of variable-size set inputs.

[0155] In one example, the first feature and the second feature are fused to obtain the target fused feature. Alternatively, the account attribute information of the target video account can be extracted first to obtain the account feature. The first feature, the second feature and the account feature are then input into the fusion layer of the model to perform feature fusion to obtain the target fused feature. The account attribute information includes the account creation time, the account's commonly used login address, the account signature, the avatar, etc.

[0156] The target fusion features output from the fusion layer are input into the classification layer. The classification layer performs classification prediction on the target fusion features to obtain the identification result of whether the target account is a copy account.

[0157] It should be noted that, Figure 8 For further details regarding S210 to S230 shown, please refer to [link to relevant documentation]. Figure 2 S210 to S230 shown will not be described again here.

[0158] In this embodiment of the application, the first feature and the second feature are transformed by the transformation function and the symmetric function in the fusion layer of the model. When processing multimodal data, the features of different modalities can be regarded as elements in the set and fused to obtain features with more robustness and representation ability. Then, the account recognition result can be accurately obtained through the classification layer of the model.

[0159] In other embodiments of this application, the first feature and the second feature are fused to obtain the target fused feature, including: extracting features from the account attribute information of the target video account to obtain the account feature; obtaining the weights corresponding to the account feature, the first feature and the second feature respectively, and performing feature fusion processing on the account feature, the first feature and the second feature according to the obtained weights to obtain the target fused feature.

[0160] It is worth noting that, in one embodiment of this application, another video tag recognition method is also provided, which can be applied to... Figure 1 The implementation environment shown can be executed by a terminal or a server, or by both. In this embodiment, the method is described using the server as an example. Figure 9 As shown, this video tag recognition method is in Figure 8 Based on the above, a training process for the account recognition model is added, namely S910 to S940. S910 to S940 are described in detail below:

[0161] S910. Obtain the labeled dataset. Use a data processing model trained in multiple rounds to clean the labeled dataset to obtain sample data. The sample data includes each sample video under the sample account and the account tags of the sample account.

[0162] The labeled dataset includes normal accounts labeled with normal account tags and sample videos under those normal accounts, as well as abnormal accounts labeled with abnormal account tags and sample videos under those abnormal accounts. The labeled dataset contains abnormal data such as noise and errors. In order to ensure the data quality of subsequent model training, the labeled dataset needs to be cleaned to obtain relatively clean and noise-free sample data. The sample data can include each sample video under a normal account and the normal account tag, and can also include each sample video under a normal account and the abnormal account tag.

[0163] In this embodiment, a data processing model trained in multiple rounds identifies anomalous data from the labeled dataset, and then performs data cleaning on the anomalous data, such as relabeling or removing lost sample data. The data processing model is a simple model such as a random forest model, and does not refer to a deep model.

[0164] In one example, a data processing model trained over multiple rounds is used to clean the labeled dataset to obtain sample data, including:

[0165] The labeled dataset is divided into training data and prediction data. Feature extraction is performed on the training data and prediction data respectively to obtain training sets and prediction sets. Both training sets and prediction sets include account attribute information and video attribute information of the training accounts. A data processing model is trained based on the training set to obtain a data processing model. The data processing model is used to identify abnormal data in the prediction set, and the abnormal data is cleaned to obtain the target prediction set. The steps of dividing the labeled dataset, training the data processing model, and cleaning the data are repeated until the data processing model is trained a preset number of times. Sample data is generated based on the target prediction set obtained from each data cleaning process.

[0166] Step 1: Divide the training data and prediction data; Step 2: Extract features to obtain the training set and prediction set; Step 3: Train the data processing model and clean the prediction set using the data processing model; Step 4: Perform training in steps 1, 2, and 3 k times to finally obtain relatively clean, noise-free data, i.e., sample data.

[0167] In step 1, the labeled dataset is randomly divided into k equal parts, with one part taken as the prediction data and the remainder as the training data. Feature extraction is performed on the training and prediction data to extract account attribute information of the training accounts, such as account name, signature, and avatar, as well as video attributes of the sample videos under the training accounts, such as video title, frame extraction results, entity recognition results, optical character recognition (OCR) and ASR results. When extracting video attribute information, the sample videos under the training account can be randomly shuffled first to further enhance the diversity of the data.

[0168] The obtained training set and prediction set both include account attribute information and video attribute information of the training accounts. The training set is used to train and set the model, such as the random forest model, to obtain the data processing model. The data processing model is then used to identify abnormal data in the prediction set and to clean the abnormal data to obtain the target prediction set. The abnormal data includes at least one of abnormal account attribute information and abnormal video attribute information. This completes one cycle until the number of training iterations of the data processing model reaches the preset number, such as k times. That is, the data is trained k times in a loop. Each time, steps such as data partitioning, feature extraction, prediction, and cleaning are performed. The target prediction sets obtained each time are then summarized to obtain the sample data.

[0169] S920. Extract the first and second features of each sample video corresponding to the sample account.

[0170] In one example, the first and second features of each sample video corresponding to the sample account are extracted by the feature extraction layer of the model to be trained. The specific process of extracting the first and second features is described in the aforementioned embodiment and will not be repeated here.

[0171] S930. The first feature and the second feature of the sample are mapped to the target feature space through the fusion layer of the model to be trained, and the features of the target feature space are clustered to obtain the sample fusion features. The sample fusion features are then classified and predicted through the classification layer of the model to be trained to obtain the sample account identification result.

[0172] The first and second features of the sample are input into the fusion layer of the model to be trained. For details on the process of fusing the sample features output by the fusion layer, please refer to [link to relevant documentation]. Figure 8 The example shown.

[0173] The sample account includes multiple sample videos. After obtaining the sample fusion features of each sample video, the sample fusion features of each sample video can be spliced ​​together and then input into the classification layer to obtain the recognition result of the sample account.

[0174] S940. Construct a loss function based on the sample account recognition results and the account tags of the sample accounts, and adjust the model parameters of the model to be trained according to the loss function to obtain the account recognition model.

[0175] Each sample account has a corresponding account tag, which indicates whether the sample account is a copycat account. The sample account identification result is the result of the model's prediction of whether the sample account is a copycat account. Therefore, a loss function can be calculated based on the difference between the sample account identification result and the sample account's account tag. This loss function can be such as a contrastive loss function or a mean squared error loss function. Then, the model parameters of the model to be trained are optimized based on the loss function until the difference between the sample account identification result and the sample account's account tag is less than a preset threshold, thus obtaining the account identification model.

[0176] It should be noted that, Figure 9 For further detailed information on S210-S230 and S810-S820 shown, please refer to [link to relevant documentation]. Figure 8 S210~S230 and S810~S820 shown will not be described again here.

[0177] For ease of understanding, this application also provides a video account identification method for identifying reposting accounts on a video platform. The video platform can be a video platform provided by an instant social application, such as... Figure 10 As shown, the video platform is Video Accounts. The instant social application displays the "Video Accounts" entry. The right image shows the recommendation feed interface after clicking on this entry. Users can experience different feeds (specific videos in Video Accounts) by swiping in the recommendation feed interface. Each feed is published by a specific account.

[0178] Some copied content may appear in the recommendation stream, such as... Figure 11 As shown, Figure 11 Three reselling accounts were shown. Figure 11 The account in (a) that reposted videos from celebrities or influential figures within the same video account, Figure 11 The video posted by account (b) does not feature the same person in the video. Figure 11 The categories of videos posted by the reposting account (c) are inconsistent.

[0179] In addition to manually identifying content plagiarism, related technologies can also use credential stuffing attacks. This involves searching the existing content library for similar feeds to identify content plagiarism within the video account. However, this credential stuffing attack method is limited to the results within the video account and cannot identify plagiarism patterns, thus it cannot identify content plagiarism from outside the video account.

[0180] The video account recognition method provided in this application uses a multimodal deep learning model to incorporate the content of account migration into the model, thus getting rid of the reliance on credential stuffing detection and starting more from the account content to identify the migration pattern and the migrated content.

[0181] like Figure 12 As shown, the training process of the account recognition model used in the video account recognition method includes cleaning dirty data, finding effective feed sub-features, and fusing the feed sub-features into an account model.

[0182] like Figure 13 As shown, cleaning dirty data yields sample data for training the model, including k-flod data partitioning, feature extraction, model calibration, and data cleaning.

[0183] Step 1: Perform k-flod decomposition on the manually labeled data (randomly divide it into k equal parts), and take 1 part of it as the prediction set each time, and use the rest as the training set.

[0184] The manually labeled data includes feed videos from both reposted and normal accounts, with tags manually assigned to them. First, the manually labeled data is checked to identify incorrectly labeled data, which is then modified or deleted to ensure data accuracy. The modified data is then randomly divided into k equal parts, with one part taken as prediction data each time and the remainder used as training data.

[0185] Step 2: Perform feature extraction on the data processed in Step 1. The feature extraction process is as follows: Figure 14 As shown, for each account in the training and prediction data, account attributes are extracted, including the account name and signature. For extracting video attributes, the feed list under each account is first randomly sorted to enhance data robustness. Then, video attributes are extracted, including the titles of recently published feeds, frame extraction results, entity recognition results, OCR and ASR results. Figure 14 Whether or not the content is reposted is an account tag.

[0186] Step 3: Using the training dataset from Step 2, use the model trained on the training set (such as a random forest model) to predict the prediction set, identify data in the prediction set that is highly likely to be problematic, and relabel or discard them.

[0187] Step 4: Repeat steps 1, 2, and 3 k times to obtain relatively clean, noise-free data.

[0188] The prediction set obtained by processing the abnormal data in step 3 each time is used as the sample data for training the model. With the training sample data, before training the deep model, it is necessary to identify which sub-features in the sample data are useful and which features are important. Through data analysis, the importance of features is identified, and sub-features that are positively correlated with account tags are found. These sub-features are then refined and made more accurate.

[0189] In the scenario of account reposting, the sub-features of the effective feeds are extracted from the feeds published by the accounts. Therefore, they reflect certain characteristics and behaviors of the accounts and are closely related to the account characteristics. By analyzing these sub-features, we can better understand and evaluate the account's behavior and content quality. Among them, the four most important sub-features are:

[0190] The results of each feed from an account (i.e., the aforementioned content detection features): It serves as the absolute basis for determining whether content is within or outside the video account, and the primary basis for determining content plagiarism. It refers to whether the feed published by the account matches the content in other known content libraries, in order to help identify whether the account has engaged in content plagiarism.

[0191] The presence of real people in each feed of an account (i.e., the aforementioned target facial features): Whether they are real people is an important basis for judging whether content is plagiarized. It refers to whether real people appear in the feed to help judge the authenticity of the account and the originality of the content.

[0192] The category of each feed for an account (the aforementioned category features) can determine whether the categories are similar or consistent. This refers to the content category features of the feed, which helps to understand the content direction of the account.

[0193] Each feed of an account has its own individual embedding (the aforementioned content features): This refers to the embedding feature of the feed, which is used to represent the content features of the feed to help the model better understand and distinguish the content of different accounts.

[0194] like Figure 15 As shown, the CLIP model is introduced to extract the credential stuffing results of each feed from the sample data. This includes extracting key text features and key image features of the feed through the CLIP model, followed by cross-modal alignment processing, matching the extracted features with the existing database, and then performing whitelist exemption to obtain the credential stuffing results.

[0195] like Figure 16 As shown, the optimization of real-person appearance in each feed of the account is as follows: face detection is performed on the feed through SSD, key point detection is performed through Dlib, face recognition is performed through ResNest to identify n faces, and face features are clustered through H-clustert to achieve deduplication of multiple faces, and then the face eMB is output.

[0196] like Figure 17 As shown, the classification of each feed for an account is as follows: a classification model is obtained through a pre-trained model, the feed is input into the classification model, and the classification model generates the classification emb.

[0197] Individual features of each feed for each account: Video and text features are extracted from the feed using a video encoder and a text encoder, and then these features are fused to obtain the feed's unique features. The video encoder and text encoder were pre-trained on a very large dataset and then fine-tuned through comparative learning on a smaller dataset. For details, please refer to [link to relevant documentation]. Figure 6 and 7 As shown.

[0198] This application embodiment requires four types of feed sub-features, such as Figure 18 As shown, the fusion layer of the account recognition model adopts the most suitable Deep-sets fusion. The target fusion features output by the fusion layer are input into the classification layer, which then performs classification prediction and outputs the result of whether the account is a copy account. Based on the output of the classification layer and the account label corresponding to the account, a loss function is constructed, and the parameters of the model are adjusted to obtain the account recognition model.

[0199] After the account model recognition training is completed, four sub-features of the new account's feed are obtained. These four sub-features are then input into the model, and the model outputs whether the new account is a copycat account.

[0200] Once a new account is identified as a content reposting account, rules can be strengthened in the recommendation feed to ban or limit the traffic of these accounts, preventing them from gaining access to traffic. This can optimize the user experience and boost the creative enthusiasm of video account creators.

[0201] The video account identification method provided in this application supplements existing feed-level content plagiarism identification. On the one hand, identification from the account dimension can increase the recall rate of plagiarized content identification and improve the identification of plagiarized traffic. On the other hand, account dimension identification can more easily identify cases of video plagiarism. Among them, the multimodal model is used to fully consider text, image and numerical features, which improves the accuracy of the model. By using four features—the brute-force attack results of each feed of the account, the appearance of real people in each feed of the account, the classification of each feed of the account, and the individual EMB of each feed of the account—the accuracy and recall rate of the final model results are improved.

[0202] This application describes an apparatus embodiment that can be used to execute the video account recognition method described above. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the video account recognition method described above.

[0203] This application provides a video account recognition device, such as... Figure 19 As shown, the device includes:

[0204] The acquisition module 1910 is used to acquire at least one target video uploaded by the target video account;

[0205] The extraction module 1920 is used to extract key features corresponding to key information of the target video, and perform feature matching processing on the key features to obtain a first feature for detecting the authenticity of the content of the target video;

[0206] The extraction module 1920 is further configured to perform multimodal feature extraction processing on the target video to obtain a second feature for describing the content characteristics of the target video;

[0207] The identification module 1930 is used to perform feature fusion processing on the first feature and the second feature to obtain a target fusion feature, and to identify the target video account based on the target fusion feature to obtain an identification result that characterizes whether the target video account is a reposting account.

[0208] In one embodiment of this application, based on the foregoing scheme, the first feature includes a content detection feature for detecting whether the content of the target video is plagiarized content; the extraction module is further used to extract key text features corresponding to key text in the target video, and key image features corresponding to key image frames in the target video; perform cross-modal alignment processing on the key text features and the key image features to obtain target text features and target key image features; search existing video feature libraries according to the target text features and target key image features respectively to obtain the feature similarity between the features of the target video and the features of the existing video feature library; and generate the content detection feature based on the feature similarity.

[0209] In one embodiment of this application, based on the foregoing scheme, the extraction module is further configured to generate the content detection feature based on the feature similarity if the feature similarity is greater than a first preset similarity threshold; obtain a preset exemption whitelist if the feature similarity is greater than a second preset similarity threshold and less than the first preset similarity threshold; match the video attributes of the target video with the video attributes in the exemption whitelist, and obtain the content detection feature based on the matching result; wherein, the video attributes include at least one of video account and video segment.

[0210] In one embodiment of this application, based on the aforementioned scheme, the target video includes a face, and the first feature includes a target face feature for detecting whether the target video is a real person appearing in the video; the extraction module is further configured to perform face detection on each image frame of the target video to obtain a face image region for each image frame; perform key point detection on the face image region of each image frame, and perform face alignment processing based on the detected key point coordinates to obtain a target face image for each image frame; perform face recognition on the target face image of each image frame to obtain face features for each face; and perform clustering processing on the face features of each face to obtain the target face features.

[0211] In one embodiment of this application, based on the aforementioned scheme, the extraction module is further configured to calculate the facial similarity between each pair of faces based on the facial features of each face; aggregate similar facial features into the same cluster based on a set similarity threshold and the facial similarity; perform feature fusion on the facial features in each cluster to obtain a representative feature vector of each cluster, and generate the target facial features based on the representative feature vector of each cluster.

[0212] In one embodiment of this application, based on the foregoing scheme, the second feature includes classification features for describing the content classification of the target video; the extraction module is used to perform multimodal feature extraction on the target video through the feature layer of a pre-trained classification model to obtain the classification features generated by the feature layer; wherein, the multimodality includes at least two of image modality, text modality, and audio modality.

[0213] In one embodiment of this application, based on the foregoing scheme, the second feature includes the content features of the target video; the extraction module is further configured to acquire video modal information and text modal information of each image frame of the target video; acquire a pre-trained video encoder and text encoder, wherein the video encoder and the text encoder are obtained by training an intermediate video encoder and an intermediate text encoder using sample videos and sample text, and then fine-tuning the parameters of the intermediate video encoder and the intermediate text encoder using positive sample pairs and negative sample pairs; the positive sample pairs include matching sample videos and sample text, and the negative sample pairs include unmatched sample videos and sample text; the video encoder extracts features from the video modal information of each image frame to obtain video features, and the text encoder extracts features from the text information of each image frame to obtain text features; and the content features are obtained by feature fusion based on the video features and the text features.

[0214] In one embodiment of this application, based on the foregoing scheme, the apparatus further includes a training module, configured to extract features from the sample video using an initial video encoder to obtain sample video features, and extract features from the sample text using an initial text encoder to obtain sample text features; fuse the sample video features and sample text features to obtain a first sample fusion feature, and calculate a loss function based on the first sample fusion feature; and adjust the parameters of the initial video encoder and the initial text encoder according to the loss function to obtain the intermediate video encoder and the intermediate text encoder.

[0215] In one embodiment of this application, based on the foregoing scheme, the training module is further configured to extract video and text features for each positive sample pair and each negative sample pair through the intermediate video encoder and the intermediate text encoder, and generate a second sample fusion feature corresponding to the positive sample pair and a third sample fusion feature corresponding to the negative sample pair through feature fusion processing; keep the original parameters of the intermediate video encoder and the intermediate text encoder frozen, and introduce new parameters into the intermediate video encoder and the intermediate text encoder; construct a contrastive loss function based on the second sample fusion feature and the third sample fusion feature; and adjust the new parameters corresponding to the intermediate video encoder and the intermediate text encoder respectively according to the contrastive loss function to obtain the video encoder and the text encoder.

[0216] In one embodiment of this application, based on the foregoing scheme, the recognition module is further configured to input the first feature and the second feature into a pre-trained account recognition model, so as to map the first feature and the second feature to a target feature space through the fusion layer of the account recognition model, and to perform clustering processing on the features of the target feature space to obtain target fusion features; and to perform classification prediction on the target fusion features through the classification layer of the account recognition model to obtain the recognition result.

[0217] In one embodiment of this application, based on the foregoing scheme, the device further includes a training module. The training module is used to acquire a labeled dataset, perform data cleaning on the labeled dataset through a multi-round trained data processing model to obtain sample data, the sample data including each sample video under a sample account, and the account tag of the sample account; extract a first sample feature and a second sample feature for each sample video corresponding to the sample account; map the first sample feature and the second sample feature to a target feature space through the fusion layer of the model to be trained, and perform clustering processing on the features of the target feature space to obtain sample fusion features, and perform classification prediction on the sample fusion features through the classification layer of the model to be trained to obtain the sample account recognition result of the sample account; construct a loss function based on the sample account recognition result and the account tag of the sample account, and adjust the model parameters of the model to be trained according to the loss function to obtain the account recognition model.

[0218] In one embodiment of this application, based on the foregoing scheme, the training module is further configured to divide the labeled dataset into training data and prediction data, perform feature extraction on the training data and prediction data respectively to obtain training sets and prediction sets, wherein both the training sets and the prediction sets include account attribute information and video attribute information of the training accounts; train a set model according to the training set to obtain the data processing model, identify abnormal data in the prediction set through the data processing model, and perform data cleaning on the abnormal data to obtain the target prediction set; repeat the steps of dividing the labeled dataset, training the data processing model, and data cleaning until the number of training iterations of the data processing model reaches a preset number, and generate the sample data based on the target prediction set obtained from each data cleaning process.

[0219] It should be noted that the apparatus provided in the above embodiments and the method provided in the above embodiments belong to the same concept, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0220] The device provided in the above embodiments can be located in a terminal or in a server.

[0221] Embodiments of this application also provide an electronic device, including one or more processors and a storage device, wherein the storage device is used to store one or more computer programs, which, when executed by one or more processors, cause the electronic device to implement the video account recognition method described above.

[0222] Figure 20 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0223] It should be noted that, Figure 20 The computer system 2000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0224] like Figure 20As shown, the computer system 2000 includes a central processing unit (CPU) 2001, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on a program stored in read-only memory (ROM) 2002 or a program loaded from storage portion 2008 into random access memory (RAM) 2003. The RAM 2003 also stores various programs and data required for system operation. The CPU 2001, ROM 2002, and RAM 2003 are interconnected via a bus 2004. An input / output (I / O) interface 2005 is also connected to the bus 2004.

[0225] In some embodiments, the following components are connected to the I / O interface 2005: an input section 2006 including a keyboard, mouse, etc.; an output section 2007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 2008 including a hard disk, etc.; and a communication section 2009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 2009 performs communication processing via a network such as the Internet. A drive 2010 is also connected to the I / O interface 2005 as needed. A removable medium 2011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 2010 as needed so that computer programs read from it can be installed into the storage section 2008 as needed.

[0226] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 2009, and / or installed from removable medium 2011. When the computer program is executed by processor (CPU) 2001, it performs various functions defined in the system of this application.

[0227] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory, flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0228] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0229] The units or modules described in the embodiments of this application can be implemented in software or hardware, and can also be located in a processor. The names of these units or modules do not necessarily limit the specific unit or module itself.

[0230] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the video account identification method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0231] Another aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the video account identification method as described above in the various embodiments.

[0232] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0233] Other embodiments of this application will readily conceive of by considering the specification and practicing the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0234] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for identifying video accounts, characterized in that, include: Obtain at least one target video uploaded by the target video account; Extract key features corresponding to the key information of the target video, and perform feature matching processing on the key features to obtain a first feature for detecting the authenticity of the content of the target video; Multimodal feature extraction processing is performed on the target video to obtain a second feature that describes the content characteristics of the target video; The first feature and the second feature are fused to obtain the target fused feature, and the target video account is identified based on the target fused feature to obtain the identification result used to characterize whether the target video account is a reposting account.

2. The method according to claim 1, characterized in that, The first feature includes content detection features for detecting whether the content of the target video is plagiarized content; the step of extracting key features corresponding to key information of the target video and performing feature matching processing on the key features to obtain a first sub-feature for detecting the authenticity of the content of the target video includes: Extract the key text features corresponding to the key text in the target video, and the key image features corresponding to the key image frames in the target video; Cross-modal alignment processing is performed on the key text features and the key image features to obtain target text features and target key image features; Based on the target text features and target key image features, the existing video feature library is retrieved to obtain the feature similarity between the features of the target video and the features of the existing video feature library. The content detection features are generated based on the feature similarity.

3. The method according to claim 2, characterized in that, The step of generating the content detection features based on the feature similarity includes: If the feature similarity is greater than a first preset similarity threshold, then the content detection feature is generated based on the feature similarity. If the feature similarity is greater than the second preset similarity threshold and less than the first preset similarity threshold, then a preset exemption whitelist is obtained. The video attributes of the target video are matched with the video attributes in the exemption whitelist, and the content detection features are obtained based on the matching results; wherein, the video attributes include at least one of video account and video segment.

4. The method according to claim 1, characterized in that, The target video includes a face, and the first feature includes target face features for detecting whether the target video is a real person appearing in the video; extracting key features corresponding to key information of the target video, and performing feature matching processing on the key features to obtain a first feature for detecting the authenticity of the content of the target video, including: Face detection is performed on each image frame of the target video to obtain the face image region of each image frame; Key point detection is performed on the face image region of each image frame, and face alignment processing is performed based on the detected key point coordinates to obtain the target face image of each image frame. Face recognition is performed on the target face image of each image frame to obtain the facial features of each face; Clustering is performed on the facial features of each face to obtain the target facial features.

5. The method according to claim 4, characterized in that, The process of clustering the facial features of each face to obtain the target facial features includes: Calculate the facial similarity between each pair of faces based on the facial features of each face. Based on the set similarity threshold and the face similarity, similar facial features are aggregated into the same cluster; Feature fusion is performed on the facial features in each cluster to obtain a representative feature vector for each cluster, and the target facial features are generated based on the representative feature vector of each cluster.

6. The method according to claim 1, characterized in that, The second feature includes classification features for describing the content classification of the target video; the step of performing multimodal feature extraction on the target video to obtain the second feature for describing the content characteristics of the target video includes: The target video is subjected to multimodal feature extraction through the feature layer of a pre-trained classification model to obtain the classification features generated by the feature layer; wherein, the multimodality includes at least two of image modality, text modality, and audio modality.

7. The method according to claim 1, characterized in that, The second feature includes the content features of the target video; the step of performing multimodal feature extraction on the target video to obtain the second feature used to describe the content characteristics of the target video includes: Obtain the video modal information and text modal information of each image frame of the target video; A pre-trained video encoder and text encoder are obtained. The video encoder and the text encoder are obtained by training an intermediate video encoder and an intermediate text encoder using sample videos and sample texts. The intermediate video encoder and the intermediate text encoder are then fine-tuned by using positive sample pairs and negative sample pairs. The positive sample pairs include matching sample videos and sample texts, and the negative sample pairs include unmatched sample videos and sample texts. The video encoder extracts video features from the video modal information of each image frame, and the text encoder extracts text features from the text information of each image frame. The content features are obtained by feature fusion based on the video features and the text features.

8. The method according to claim 7, characterized in that, The training steps for the intermediate video encoder and the intermediate text encoder include: The sample video features are obtained by extracting features from the sample video using an initial video encoder, and the sample text features are obtained by extracting features from the sample text using an initial text encoder. The sample video features and sample text features are fused to obtain the first sample fusion feature, and the loss function is calculated based on the first sample fusion feature; The intermediate video encoder and the intermediate text encoder are obtained by adjusting the parameters of the initial video encoder and the initial text encoder according to the loss function.

9. The method according to claim 7, characterized in that, The training steps for the video encoder and the text encoder include: For each positive sample pair and each negative sample pair, video and text features are extracted by the intermediate video encoder and the intermediate text encoder, and feature fusion processing is used to generate the second sample fusion feature corresponding to the positive sample pair and the third sample fusion feature corresponding to the negative sample pair. The original parameters of the intermediate video encoder and the intermediate text encoder are kept frozen, and new parameters are introduced into the intermediate video encoder and the intermediate text encoder. Construct a contrastive loss function based on the second sample fusion feature and the third sample fusion feature; The newly added parameters corresponding to the intermediate video encoder and the intermediate text encoder are adjusted according to the contrast loss function to obtain the video encoder and the text encoder.

10. The method according to any one of claims 1 to 9, wherein the step of performing feature fusion processing on the first feature and the second feature to obtain a target fused feature, and identifying the target video account based on the target fused feature, comprises: The first feature and the second feature are input into a pre-trained account recognition model, so that the first feature and the second feature are mapped to the target feature space through the fusion layer of the account recognition model, and the features of the target feature space are clustered to obtain the target fused feature. The target fusion features are classified and predicted by the classification layer of the account recognition model to obtain the recognition result.

11. The method according to claim 10, characterized in that, The training steps for the account recognition model include: Obtain the labeled dataset, and clean the labeled dataset using a data processing model trained in multiple rounds to obtain sample data. The sample data includes each sample video under the sample account, as well as the account tags of the sample account. Extract the first feature and the second feature of each sample video corresponding to the sample account; The first feature and the second feature of the sample are mapped to the target feature space through the fusion layer of the model to be trained, and the features of the target feature space are clustered to obtain the sample fusion feature. The sample fusion feature is then classified and predicted through the classification layer of the model to be trained to obtain the sample account identification result. A loss function is constructed based on the sample account identification results and the account tags of the sample accounts, and the model parameters of the model to be trained are adjusted according to the loss function to obtain the account identification model.

12. The method according to claim 11, characterized in that, The process of cleaning the labeled dataset using a data processing model trained through multiple rounds to obtain sample data includes: The labeled dataset is divided into training data and prediction data. Feature extraction is performed on the training data and prediction data respectively to obtain training set and prediction set. Both the training set and the prediction set include account attribute information and video attribute information of the training account. The data processing model is obtained by training the model based on the training set, and the abnormal data in the prediction set is identified by the data processing model. The abnormal data is then cleaned to obtain the target prediction set. Repeat the steps of dividing the labeled dataset, training the data processing model, and cleaning the data until the data processing model has been trained a preset number of times, and generate the sample data based on the target prediction set obtained from each data cleaning process.

13. A video account recognition device, characterized in that, include: The acquisition module is used to acquire at least one target video uploaded by the target video account; An extraction module is used to extract key features corresponding to key information of the target video, and perform feature matching processing on the key features to obtain a first feature for detecting the authenticity of the content of the target video; The extraction module is further configured to perform multimodal feature extraction processing on the target video to obtain a second feature for describing the content characteristics of the target video; The identification module is used to perform feature fusion processing on the first feature and the second feature to obtain a target fusion feature, and to identify the target video account based on the target fusion feature to obtain an identification result that characterizes whether the target video account is a reposting account.

14. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor of the electronic device, causes the electronic device to perform the method of any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from and executes the computer program to cause the electronic device to perform the method of any one of claims 1 to 12.

Images