Video call verification method and device, electronic equipment, medium and product

The initial and secondary verification of false call video recognition model trained with cost parameters and feature distance limitation is solved, and the problem of low efficiency of false video call recognition is achieved, achieving higher accuracy and reliability.

CN119091350APending Publication Date: 2024-12-06CHINA MOBILE GROUP ZHEJIANG +1

Patent Information

Application Number
CN202411193851.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, the processing efficiency of false video call recognition is low, mainly due to the complex processing process.

Method used

By obtaining the call video to be verified, initial verification is performed using a false call video recognition model trained based on the recognition cost parameter and feature distance limitation. If the initial verification is false, secondary verification is performed to improve accuracy.

Benefits of technology

It improves the accuracy and reliability of false video calls recognition, reduces the complexity of the processing process, and improves the recognition efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091350A_ABST
    Figure CN119091350A_ABST
Patent Text Reader

Abstract

The invention provides a video call verification method and device, electronic equipment, a medium and a product, and belongs to the technical field of Internet, and the method comprises the steps: inputting a to-be-verified call video into a false call video recognition model, and obtaining an initial verification result; the false call video identification model is obtained by training an initial model based on identification cost parameters and feature distance limitation; the identification cost parameter is determined based on the deviation between the real type and the identification type of the sample call video; the feature distance limitation is used for limiting the feature distance between the real visual feature and the false visual feature; the real visual features and the false visual features are obtained by performing feature extraction on the sample call video; the initial model is used for identifying whether the call video is real or false; and if the initial verification result is that the to-be-verified call video belongs to the false type, carrying out secondary verification on the to-be-verified call video to obtain a target verification result of the to-be-verified call video. According to the invention, the processing efficiency during false video call identification can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a video call verification method, device, electronic device, medium and product. Background Art

[0002] With the rapid development of communication technology, video calls have become an indispensable part of people's daily lives. However, with the continuous advancement of artificial intelligence technology, fake call videos, that is, call videos synthesized using artificial intelligence technology, have gradually surfaced, posing serious security risks to society.

[0003] In the prior art, fake call videos are identified through the use of multiple recognition models (such as voice feature recognition, facial expression feature recognition, human movement feature recognition, etc.) and special effect tag libraries. However, this method has a very complicated processing flow, resulting in low processing efficiency when identifying fake video calls. Summary of the invention

[0004] The present application provides a video call verification method, device, electronic device, medium and product, which are used to solve the problem that the existing method of identifying false call videos has a very complicated processing flow, resulting in low processing efficiency when identifying false video calls, thereby improving the processing efficiency when identifying false video calls.

[0005] The present application provides a video call verification method, which includes: obtaining a call video to be verified; inputting the call video to be verified into a false call video recognition model to obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training an initial model based on a recognition cost parameter and a feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false; if the initial verification result is that the call video to be verified is a false type, then the call video to be verified is secondary verified to obtain the target verification result of the call video to be verified.

[0006] According to a video call verification method provided by the present application, the false call video recognition model is obtained in the following manner: obtaining an initial model and a sample call video; the sample call video includes an actual call video and a synthetic call video; the synthetic call video is generated by artificial intelligence based on the actual call video; using the initial model to perform feature recognition on the actual call video and the synthetic call video in turn to obtain a real recognition type of the actual call video and a false recognition type of the synthetic call video; determining a recognition cost parameter based on the real deviation between the real recognition type of the actual call video and the real type of the actual video call, and the false deviation between the false recognition type of the synthetic call video and the real type of the synthetic call video; obtaining a feature distance limit; training the initial model based on the recognition cost parameter and the feature distance limit to obtain a false call video recognition model; wherein, the real visual feature is obtained by performing feature recognition on the actual call video using the initial model, and the false visual feature is obtained by performing feature recognition on the synthetic call video using the initial model.

[0007] According to a video call verification method provided by the present application, the initial model includes a feature recognition module; the feature distance limitation includes a real feature distance cost parameter, and the real feature distance cost parameter is determined in the following manner: according to the feature distance between the real visual feature and the false visual feature of each feature recognition module, the false feature distance cost parameter of the corresponding feature recognition module is determined respectively; the important coefficient associated with each feature recognition module is obtained; the important coefficient is determined by the position of the corresponding feature recognition module in the entire feature recognition module composed of each feature recognition module; according to the important coefficient associated with each feature recognition module and the false feature distance cost parameter, the weighted linear calculation parameter of the false feature distance cost parameter of each feature recognition module is determined; based on each weighted linear calculation parameter, the real feature distance cost parameter is determined.

[0008] According to a video call verification method provided by the present application, it also includes a primary recognition module for distinguishing whether a call video is an actual call video or a synthetic call video, and an advanced recognition module with a recognition ability higher than that of the primary recognition module; before performing a secondary verification on the call video to be verified, it also includes: obtaining a training instance, the training instance including a sample pending call video and a matching actual classification identifier and a sample confidence; performing feature recognition on the sample pending call video through the primary recognition module to obtain a sample video frame feature corresponding to the sample pending call video; and performing a secondary verification on the sample pending call video according to the sample video frame feature. The method comprises the steps of: inferring the video call classification result of the sample video frame feature based on the sample video frame feature to obtain the inferred classification identifier and the inferred confidence; calculating the sample reliability coefficient of the video call classification result inferred by the sample video frame feature for the sample pending call video according to the sequence information of the sample video frame feature and the inferred confidence; wherein the sample video frame feature comprises a plurality of pixel-level sample video frame features corresponding to each feature extractor obtained by performing feature recognition on the sample pending call video in sequence according to a plurality of feature extractors of the primary recognition module; determining a target feature extractor from the plurality of feature extractors; extracting a sample video frame feature based on the sample video frame feature extracted by the target feature extractor; The sample reliability coefficient corresponding to the video frame feature is clustered on the multiple pixel-level sample video frame features of the target feature extractor to generate a sample summary content for describing the classification of the sample pending call video into the inferred classification identifier; wherein the training instance also includes the video call classification result summary content corresponding to the sample pending call video; a summary content set is generated according to the video call classification result summary content and the sample summary content; a target summary content is determined from the summary content set; a sample feature map indicating the sample pending call video is generated according to the target summary content; the sample pending call video and the sample feature map are integrated to obtain an integrated pending call video; an advanced video call classification result inference is performed on the sample pending call video according to the integrated pending call video through the advanced recognition module to obtain a target inferred classification identifier; the error between the inferred classification identifier and the actual classification identifier is determined to obtain an actual surrogate parameter, and the error between the target inferred classification identifier and the actual classification identifier is determined to obtain an inferred surrogate parameter; the network architecture of the primary recognition module is optimized according to the actual surrogate parameter, and the network architecture of the advanced recognition module is optimized according to the inferred surrogate parameter until a preset state is reached.

[0009] According to a video call verification method provided by the present application, the secondary verification of the call video to be verified is performed to obtain a target verification result of the call video to be verified, including: performing feature recognition on the call video to be verified to obtain video frame features; the video frame features include at least one of facial expression features, eye features, and audio abnormality features; using a primary recognition module to classify the video according to the video frame features to obtain a classification result; the classification result includes a classification type and its classification coefficient; and classifying the video according to the video frame features and the classification coefficient to obtain a target verification result of the call video to be verified.

[0010] According to a video call verification method provided by the present application, video classification is performed according to the video frame features and the classification coefficient to obtain a target verification result of the call video to be verified, including: determining the reliability coefficient of the video frame features for the classification of the call video to be verified according to the sequence information of the video frame features and the classification coefficient; clustering the video frame features according to the reliability coefficient and generating summary content for describing the call video to be verified as the corresponding video call classification result based on the clustering result; and using an advanced recognition module to classify the call video to be verified according to the summary content to obtain the target verification result of the call video to be verified.

[0011] According to a video call verification method provided by the present application, after obtaining the target verification result of the call video to be verified, it also includes at least one of the following: according to the target verification result, determining the target pending call video corresponding to the classification result that the call video to be verified is identified as a target video call, and marking the target pending call video; according to the target verification result, sending the call video to be verified to an isolation server corresponding to the classification result that the call video to be verified is identified as a target video call; according to the target verification result, generating an alarm pop-up window corresponding to the classification result that the call video to be verified is identified as a target video call.

[0012] The present application also provides a video call verification device, including: an acquisition module, used to obtain a call video to be verified; a first classification module, used to input the call video to be verified into a false call video recognition model, and obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training the initial model based on a recognition cost parameter and a feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false; a second classification module, used to perform a secondary verification on the call video to be verified if the initial verification result is that the call video to be verified is a false type, and obtain a target verification result of the call video to be verified.

[0013] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the video call verification method described in any one of the above methods is implemented.

[0014] The present application also provides a medium, which is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the video call verification method as described in any one of the above is implemented.

[0015] The present application also provides a product, which is a computer program product, including a computer program, and when the computer program is executed by a processor, it implements any of the above-mentioned video call verification methods.

[0016] The video call verification method, device, electronic device, medium and product provided by the present application obtain a false call video recognition model by pre-training an initial model for identifying whether a call video is real or false based on recognition cost parameters and feature distance limitations, wherein the recognition cost parameters are determined based on the deviation between the real type and the recognition type of the sample call video, and the feature distance limitation is used to limit the feature distance between the real visual features and the false visual features, and the real visual features and the false visual features are both obtained by feature extraction of the sample call video; and then, after obtaining the call video to be verified, the call video to be verified is input into the false call video recognition model. Since the false call video recognition model is based on the recognition cost parameters and feature distance limitations, It is obtained through distance limited training, so it can more accurately identify real visual features and false visual features, and thus obtain an accurate initial verification result of the false call video recognition model output; further, when the initial verification result of the present application is that the call video to be verified is of a false type, the call video to be verified can be further verified for the second time to obtain a more accurate target verification result of the call video to be verified, thereby improving the accuracy and reliability of false video call recognition, since there is no need for the use of multiple recognition models and special effects label libraries, the complexity of the processing flow is effectively reduced, so the complexity of the processing flow can be reduced while ensuring the recognition accuracy, thereby improving the processing efficiency of false video call recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flowchart of the video call verification method provided by this application.

[0019] Figure 2 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0022] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0023] Combine the following Figure 1-Figure 2 The present invention describes the video call verification method, device, electronic device, medium and product.

[0024] Figure 1 is a flowchart of the video call verification method provided by this application, such as Figure 1 As shown, the method includes the following steps 100 to 300.

[0025] Step 100: Obtain the call video to be verified.

[0026] Step 200: input the call video to be verified into the false call video recognition model to obtain the initial verification result output by the false call video recognition model; wherein, the false call video recognition model is obtained by training the initial model based on the recognition cost parameter and the feature distance limitation; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limitation is used to limit the feature distance between the real visual features and the false visual features; the real visual features and the false visual features are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false.

[0027] Step 300: If the initial verification result is that the call video to be verified is of a false type, a secondary verification is performed on the call video to be verified to obtain a target verification result of the call video to be verified.

[0028] It should be noted that the execution subject of the video call verification method provided in the embodiment of the present application may be a server, a computer device, etc., such as a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. The present application may use a server as an example of the execution subject of the video call verification method, but does not limit the execution subject of the video call verification method of the present application to only a server.

[0029] The server or computer device of the present application may be provided with or connected to a video call verification device. The video call verification device may be controlled in the present application to execute the video call verification method of the present application by controlling the video call verification device.

[0030] The present application can pre-train an initial model for identifying whether a call video is real or fake based on recognition cost parameters and feature distance limitations to obtain a fake call video recognition model, wherein the recognition cost parameters are determined based on the deviation between the real type and the recognition type of the sample call video, and the feature distance limitation is used to limit the feature distance between the real visual features and the fake visual features, and both the real visual features and the fake visual features are obtained by feature extraction of the sample call video.

[0031] Therefore, the fake call video recognition model can infer the video call classification results based on the data stream of the input video, and determine whether the video is a fake type or a real type, that is, whether the video is a real call video or a call video synthesized by artificial intelligence (AI).

[0032] Specifically, a video call set including actual call videos and synthesized call videos can be constructed as sample call videos, and an initial model for identifying the type of call videos as real or fake can be constructed. Further, the initial model is used to perform preliminary feature recognition on the videos in the set. By comparing the deviation between the real recognition type of the actual call video and the real actual video call category (i.e., the real type of the actual call video), and the deviation between the false recognition type of the synthesized call video and the false actual video call category (i.e., the real type of the synthesized call video), the recognition cost parameter is obtained. Then, the recognition cost parameter and the feature distance are used to limit the initial model so that it can more accurately identify real visual features and false visual features.

[0033] Thus, we obtain a fake call video recognition model, which is designed to detect whether AI is used to generate fake videos in video calls.

[0034] It should be noted that the fake call video recognition model in the present application can identify the probability that the input call video is of a real type and the probability that it is of a fake type, and can use the type with a higher probability between the two as the verification result of the input call video.

[0035] Furthermore, the server may receive a video call request from the client, the request including the target user's information and video stream data. Based on the target user's information, the server locates the user's current video call in the corresponding system and defines the video of the current video call as the call video to be verified. For example, the target user A is having a video call with user B, and the server obtains the ongoing video call and defines it as the call video to be verified. Furthermore, the server may input the obtained call video to be verified (specifically, the data stream of the call video to be verified) into a pre-trained fake call video recognition model.

[0036] It should be noted that after processing and analyzing the fake call video recognition model, the server can obtain the first verification result, which can be defined as the initial verification result in this application. If the initial verification result shows that the call video to be verified is of a real type, the server will continue to process the call normally.

[0037] According to the initial verification result, if the server determines that the current video call may be a false video generated by AI, that is, the call video to be verified is of a false type, the call video to be verified can be marked as a pending call video to be classified.

[0038] Furthermore, the server can initiate a more stringent second verification process to perform a second verification on the call video to be verified, and a more complex algorithm or model, or manual review, can be used. After the second verification, the server obtains the final recognition result as the target verification result.

[0039] Furthermore, if the final recognition result confirms that the video call is a fake video generated by AI, the server can take corresponding measures, such as interrupting the call, issuing a warning, or recording the violation, etc. If the final recognition result shows that the video call is real, the server will continue to process the call normally.

[0040] This application proposes a fake call video recognition solution based on deep learning and multi-feature fusion, which can effectively improve the recognition accuracy and reliability of fake call videos and provide strong technical support for protecting personal privacy and corporate security.

[0041] The video call verification method provided in the embodiment of the present application obtains a false call video recognition model by pre-training an initial model for identifying whether a call video is real or false based on a recognition cost parameter and a feature distance limitation, wherein the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video, and the feature distance limitation is used to limit the feature distance between the real visual feature and the false visual feature, and the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; and then, after obtaining the call video to be verified, the call video to be verified is input into the false call video recognition model. Since the false call video recognition model is trained using the recognition cost parameter and the feature distance limitation Therefore, it can more accurately identify real visual features and false visual features, so that an accurate initial verification result of the false call video recognition model output can be obtained; further, when the initial verification result of the present application is that the call video to be verified is of a false type, the call video to be verified can be further verified for the second time to obtain a more accurate target verification result of the call video to be verified, thereby improving the accuracy and reliability of false video call recognition, since there is no need for the use of multiple recognition models and special effects tag libraries, the complexity of the processing flow is effectively reduced, so the complexity of the processing flow can be reduced while ensuring the recognition accuracy, thereby improving the processing efficiency of false video call recognition.

[0042] In one embodiment, a false call video recognition model is obtained in the following manner: an initial model and a sample call video are obtained; the sample call video includes an actual call video and a synthetic call video; the synthetic call video is generated by artificial intelligence based on the actual call video; the initial model is used to perform feature recognition on the actual call video and the synthetic call video in turn to obtain a real recognition type of the actual call video and a false recognition type of the synthetic call video; a recognition cost parameter is determined based on the real deviation between the real recognition type of the actual call video and the real type of the actual video call, as well as the false deviation between the false recognition type of the synthetic call video and the real type of the synthetic call video; a feature distance limit is obtained; the initial model is trained based on the recognition cost parameter and the feature distance limit to obtain a false call video recognition model; wherein, the real visual feature is obtained by performing feature recognition on the actual call video using the initial model, and the false visual feature is obtained by performing feature recognition on the synthetic call video using the initial model.

[0043] Furthermore, the initial model includes a feature recognition module; the feature distance limitation includes a real feature distance cost parameter, and the real feature distance cost parameter is determined in the following manner: according to the feature distance between the real visual feature and the false visual feature of each feature recognition module, the false feature distance cost parameter of the corresponding feature recognition module is determined respectively; the important coefficient associated with each feature recognition module is obtained; the important coefficient is determined by the position of the corresponding feature recognition module in the entire feature recognition module composed of each feature recognition module; according to the important coefficient associated with each feature recognition module and the false feature distance cost parameter, the weighted linear calculation parameter of the false feature distance cost parameter of each feature recognition module is determined; based on each weighted linear calculation parameter, the real feature distance cost parameter is determined.

[0044] Specifically, the server of the present application can obtain a video call set as a sample call video from the database. This set contains two types of video call data: one is the actual call video, that is, the video call between real users; the other is the synthetic call video synthesized by AI, which is a fake video generated by AI technology based on the actual call video. This set is used as a data source for training and optimizing the fake call video recognition model.

[0045] More specifically, the server of the present application can obtain actual call videos, and use a real sample AI synthetic call video production strategy to generate a first synthetic call video corresponding to each actual call video; and use a false sample AI synthetic call video production strategy to generate a second synthetic call video corresponding to each actual call video.

[0046] Among them, the real sample AI synthetic call video production strategy is used to generate visual impact factors in actual call videos, the false sample AI synthetic call video production strategy is used to generate audio impact factors in actual call videos, and the multiple sample AI synthetic call video production strategy is used to generate different impact factors in actual call videos.

[0047] Furthermore, the actual call video and multiple synthetic call videos corresponding to each actual call video are taken as a video call set.

[0048] In one embodiment, the server retrieves and obtains multiple actual call videos from a database. These videos are calls between real users and are stored by the server for subsequent use. Each actual call video contains real information such as the user's facial features, actions, background, etc. The server applies two different AI synthetic call video production strategies to each actual call video obtained. First, the server uses the real sample AI synthetic call video production strategy to generate visual impact factors in each actual call video. These visual impact factors can be slight changes in facial expressions, eye movements, or adjustments to background details, aiming to make the generated AI synthetic call video visually closer to the real call, but still retain some recognizable false features. Secondly, the server uses the false sample AI synthetic call video production strategy to generate audio impact factors in each actual call video. These audio impact factors can be changes in voice pitch, addition of background noise, or traces of voice synthesis, aiming to make the generated synthetic call video present the characteristics of a false call in audio. Through these two strategies, the server generates a corresponding first synthetic call video (mainly changing the visual aspect) and a second synthetic call video (mainly changing the audio aspect) for each actual call video. The server combines the acquired actual call video and the generated first synthetic call video and second synthetic call video to form a complete video call collection. This collection contains real call videos and various types of synthetic call videos for training and optimizing the fake call video recognition model. In this way, the server can provide a diverse dataset to help the model better learn and distinguish the feature differences between real call videos and fake call videos.

[0049] Furthermore, the server uses the constructed initial model to perform preliminary feature recognition on each video in the video call set. For actual call videos, it identifies them as real calls; for synthetic call videos, it identifies them as fake calls. This recognition process is based on the model's analysis and judgment of the visual features in the video.

[0050] Specifically, the server can use the initial model to perform preliminary feature recognition on the actual call video and the synthetic call video in turn to obtain a true recognition type and a false recognition type; wherein the true recognition type and the false recognition type are used to identify the acquired call video as a sample actual call video, or to identify the sample synthetic call video production strategy corresponding to the acquired call video.

[0051] In one embodiment, the server loads the trained initial model and prepares to process the video call collection constructed in the previous step. This collection includes actual call videos and synthetic call videos generated by different strategies. First, the server uses the initial fake call video recognition model (i.e., the initial model) to perform preliminary feature recognition on the actual call video. In this process, the initial model analyzes the visual and audio features in the video, such as facial expressions, movements, voice tones, etc., and determines the authenticity of the video based on these features. If the initial model recognizes that the features in the video match the features of the real call video, it will mark these videos as real recognition types.

[0052] Next, the server continues to use the same initial model to perform preliminary feature recognition on the synthesized call videos. For the first synthesized call video (mainly changing the visual aspect), the initial model may detect some visual features that are inconsistent with the real call video, such as stiff facial expressions, unnatural eye movements, etc. For the second synthesized call video (mainly changing the audio aspect), the initial model may detect false features such as abnormal audio quality, sudden changes in voice tone, or unnatural background noise. Based on these recognition results, the initial model will mark the corresponding synthesized call video as a false recognition type.

[0053] In addition, while obtaining the true identification type and the false identification type, the server can also identify whether these call videos belong to the sample actual call video or are generated by a specific sample AI synthetic call video production strategy. This is because different AI synthesis strategies will leave unique traces or features in the generated videos, which can be learned and recognized by the model.

[0054] In this way, the server can accurately identify the type of each video in the video call collection, providing an accurate data basis for subsequent training and optimization work.

[0055] Furthermore, the server evaluates the recognition results of the initial model and calculates the recognition cost parameter. This parameter is calculated based on the deviation between the recognition results of the initial model for the actual call video and the synthesized call video and the true type. For example: if the recognition result of the initial model for the actual call video is exactly the same as the true type (ie, the true category), then the true deviation is 0; if the recognition result of the initial model for the synthesized call video is exactly the same as the true type (also called the false category), then the false deviation is 0. The recognition cost parameter in this application comprehensively considers the size of the true deviation and the false deviation, and is used to measure the recognition performance of the model.

[0056] In one embodiment, the server first focuses on the real identification type of the actual call video. If the initial model correctly identifies an actual call video as the real type, then the deviation is small or non-existent. However, if the initial model incorrectly marks an actual call video as the fake type, then there will be a significant deviation, which reflects the initial model's inadequacy in identifying the real call video.

[0057] For example, the server has a real call video in which two people are talking naturally, with real facial expressions and movements. If the initial model correctly identifies it as a real call video, then the real deviation is small. But if the initial model incorrectly identifies it as a fake call video, then the real deviation will be large, indicating that the initial model has problems in identifying the characteristics of real calls.

[0058] Next, the server processes the false identification type of the synthetic call video. For each synthetic call video, the server compares the identification results of the initial model with the specific AI synthesis strategy used to generate the video. If the initial model correctly identifies that the video is a fake call video generated by a specific strategy, then the false deviation is small. However, if the initial model fails to correctly identify the fake call video or its generation strategy, then there will be a large false deviation.

[0059] For example, the server has a synthetic call video in which a specific audio influence factor is used to change the voice pitch. If the initial model correctly identifies this video as a fake call video and is also able to identify the use of a specific audio influence factor, then the false deviation is small. However, if the initial model fails to identify this as a fake call video, or incorrectly identifies other influence factors, then the false deviation will be large.

[0060] Finally, the server takes into account the real deviation and the false deviation and calculates the recognition cost parameter. This parameter reflects the overall performance of the model in recognizing real call videos and synthetic call videos, including the accuracy of correct recognition and the cost of incorrect recognition. The server can use this parameter to further optimize the model and improve its recognition performance on different types of call videos.

[0061] The server trains and optimizes the initial model based on the recognition cost parameter and the feature distance limit. The feature distance limit is a constraint used to ensure that the model can accurately distinguish between real visual features and false visual features during the recognition process.

[0062] It should be further noted that, when the initial model includes multiple feature recognition modules in a sequential relationship, the server first analyzes each feature recognition module in the entire initial model. For each module, the server calculates the feature distance between the real visual features and the fake visual features. This feature distance measures the model's ability to distinguish the features extracted by the module when identifying real call videos and synthetic call videos.

[0063] Based on these feature distances, the server calculates the false feature distance cost parameter of each feature recognition module, which reflects the recognition performance of the model on this module. The server assigns an importance coefficient to each feature recognition module based on its position and role in the entire model. This importance coefficient reflects the degree of influence of the module on the overall recognition performance of the model. Generally speaking, feature recognition modules at the front end of the model can have larger importance coefficients because they process more basic and critical feature information.

[0064] Furthermore, the server multiplies the false feature distance cost parameter of each feature recognition module by its corresponding importance coefficient to obtain a weighted false feature distance cost parameter.

[0065] Then, the server performs linear calculation on all weighted false feature distance cost parameters to obtain a comprehensive true feature distance cost parameter, which reflects the overall recognition performance of the initial model in the entire feature recognition process.

[0066] Furthermore, the server uses the recognition cost parameter and the real feature distance cost parameter as training data to train and optimize the initial model. By continuously adjusting the parameters and structure of the initial model, the server enables the initial model to achieve higher accuracy and stability when recognizing actual call videos and synthetic call videos. This training process is an iterative process. The server will continuously evaluate the performance of the model and make corresponding adjustments until a satisfactory recognition effect is achieved. Finally, the trained and optimized fake call video recognition model will be used to verify and identify the current video call of the target user.

[0067] The present application obtains a false call video recognition model by pre-training an initial model for identifying whether a call video is real or fake based on recognition cost parameters and feature distance limitations, wherein the recognition cost parameters are determined based on the deviation between the real type and the recognition type of the sample call video, and the feature distance limitation is used to limit the feature distance between the real visual features and the false visual features, and both the real visual features and the false visual features are obtained by feature extraction of the sample call video; and then after obtaining the call video to be verified, the call video to be verified is input into the false call video recognition model. Since the false call video recognition model is trained using recognition cost parameters and feature distance limitations, its It is able to more accurately identify real visual features and false visual features, and thus obtain an accurate initial verification result of the false call video recognition model output; further, when the initial verification result is that the call video to be verified is of a false type, the present application can further perform a secondary verification on the call video to be verified, and obtain a more accurate target verification result of the call video to be verified, thereby improving the accuracy and reliability of false video call recognition, and since there is no need for the use of multiple recognition models and special effects tag libraries, the complexity of the processing flow is effectively reduced, and the complexity of the processing flow can be reduced while ensuring the recognition accuracy, thereby improving the processing efficiency of false video call recognition.

[0068] In one embodiment, the server of the present application also includes a primary recognition module for distinguishing whether a call video is an actual call video or a synthetic call video, and an advanced recognition module with a recognition ability higher than that of the primary recognition module; before performing secondary verification on the call video to be verified, it also includes: obtaining a training instance, the training instance includes a sample pending call video and a matching actual classification identifier and a sample confidence; performing feature recognition on the sample pending call video through the primary recognition module to obtain a sample video frame feature corresponding to the sample pending call video; performing video call classification result inference on the sample pending call video based on the sample video frame feature to obtain an inferred classification identifier and an inferred confidence; calculating a sample reliability coefficient of the sample video frame feature for the video call classification result inference of the sample pending call video based on the sequence information and inferred confidence of the sample video frame feature; wherein the sample video frame feature includes a plurality of pixel-level sample video frame features corresponding to each feature extractor obtained by performing feature recognition on the sample pending call video in sequence according to multiple feature extractors of the primary recognition module; determining a target feature extractor from multiple feature extractors; and extracting a sample video frame based on the sample video frame feature extracted by the target feature extractor. The sample reliability coefficient corresponding to the video frame feature is clustered on multiple pixel-level sample video frame features of the target feature extractor to generate a sample summary content for describing the classification of the sample pending call video into the inferred classification identifier; wherein the training instance also includes the video call classification result summary content corresponding to the sample pending call video; a summary content set is generated according to the video call classification result summary content and the sample summary content; the target summary content is determined from the summary content set; according to the target summary content, a sample feature map indicating the sample pending call video is generated; the sample pending call video and the sample feature map are integrated to obtain the integrated pending call video; the advanced video call classification result is inferred for the sample pending call video according to the integrated pending call video through the advanced recognition module to obtain the target inferred classification identifier; the error between the inferred classification identifier and the actual classification identifier is determined to obtain the actual surrogate parameter, and the error between the target inferred classification identifier and the actual classification identifier is determined to obtain the inferred surrogate parameter; the network architecture of the primary recognition module is optimized according to the actual surrogate parameter, and the network architecture of the advanced recognition module is optimized according to the inferred surrogate parameter until a preset state is reached.

[0069] Specifically, if the server collects a batch of call video samples from multiple sources (such as surveillance cameras, social media platforms, user uploads, etc.), some of these videos are real call records, and some are fake videos synthesized by AI. Each video sample is accompanied by an actual classification label, such as "real" or "AI synthesis", and a sample confidence, indicating the reliability of this classification label. These videos and related information constitute training instances.

[0070] The server uses the primary recognition module to process these video samples. First, it performs second-order feature recognition on each video to extract key video frame features, such as facial expressions, sound features, background environment, etc. Then, based on these features, the primary recognition module makes a preliminary classification inference for each video sample, giving an inferred classification label (such as "real" or "AI synthesis") and an inference confidence.

[0071] Furthermore, the server will calculate a sample reliability coefficient based on the sequence information and inferred confidence of the key video frame features of each video sample. This coefficient reflects the confidence level of the primary recognition module in the classification result. For example, if the features of a video sample clearly and consistently point to the category of "AI synthesis", then its sample reliability coefficient will be very high.

[0072] The primary recognition module may contain multiple feature extractors for extracting different features. The server will evaluate the performance of these feature extractors and select those that can effectively extract features that are helpful for classification as target feature extractors. For example, some feature extractors are good at extracting facial expression features, while others are good at extracting sound features. The server will select the most suitable feature extractor based on their performance.

[0073] The server will cluster the key video frame features extracted by the target feature extractor and their reliability coefficients to generate a sample summary. This summary contains the key information of the video and the basis for classification. For example, it can include facial expression features and sound features that appear in the video, as well as the frequency and importance of these features in the video.

[0074] Then, the server selects the most representative target summary content from all summary contents for subsequent classification tasks.

[0075] Furthermore, the server generates a sample feature map based on the target summary content. This feature map highlights the key features in the video, which helps the advanced recognition module to better understand and classify the video. For example, it can show the importance and distribution of facial expression features or sound features in the video in the form of a heat map.

[0076] Then, the server will integrate this sample feature map with the original video sample to obtain an integrated pending call video that contains both the original video information and highlights the key features.

[0077] Finally, the server will use the advanced recognition module to infer the advanced video call classification results of the integrated pending call videos and obtain a target inferred classification identifier. This identifier is the final judgment of the advanced recognition module on the video classification.

[0078] Then, the server calculates the error between the inferred classification mark and the actual classification mark to obtain an actual parameter and an inferred parameter. These two parameters reflect the performance of the primary recognition module and the advanced recognition module in the classification task.

[0079] The server will optimize the network architecture of the primary recognition module and the advanced recognition module based on the actual and inferred generation parameters. For example, it can adjust the parameters or structure of the feature extractor to improve the accuracy and efficiency of feature extraction; or adjust the decision boundary or weight distribution of the classifier to improve the accuracy and generalization of classification. This optimization process will continue until a preset state or performance standard is reached.

[0080] Furthermore, a secondary verification is performed on the call video to be verified to obtain a target verification result of the call video to be verified, including: feature recognition is performed on the call video to be verified to obtain video frame features; the video frame features include at least one of facial expression features, eye features, and audio abnormality features; a primary recognition module is used to perform video classification based on the video frame features to obtain a classification result; the classification result includes a classification type and a classification coefficient thereof; the video is classified based on the video frame features and the classification coefficient to obtain a target verification result of the call video to be verified.

[0081] Furthermore, video classification is performed according to the video frame features and the classification coefficient to obtain a target verification result of the call video to be verified, including: determining the reliability coefficient of the video frame features for the classification of the call video to be verified according to the sequence information and the classification coefficient of the video frame features; clustering the video frame features according to the reliability coefficient and generating summary content for describing the call video to be verified as the corresponding video call classification result based on the clustering result; and utilizing the advanced recognition module to classify the call video to be verified according to the summary content to obtain a target verification result of the call video to be verified.

[0082] Specifically, the server of this application has identified a batch of pending call videos that may be synthetic call videos (i.e., the type of initial verification results are false) through preliminary verification. Therefore, the server can perform more in-depth classification processing on these videos.

[0083] The server retrieves the pending call videos that are initially marked as "possible AI call videos" from the storage system (i.e., the pending call videos of the type whose initial verification results are false). Next, the server processes these videos using a more advanced second-order feature recognition algorithm. This algorithm focuses on identifying subtle features in the video, such as slight changes in facial expressions, unnatural eye movements, abnormal patterns in audio, etc., which may not be fully captured or analyzed in the initial verification.

[0084] The server inputs the key video frame features extracted by the second-order feature recognition algorithm into a specially trained classifier. This classifier performs a preliminary classification of the pending call video based on these features and the patterns learned from a large number of known real and synthetic call videos. The classification result includes the probability that the video is identified as a real call video or a synthetic call video, and a classification coefficient, which indicates the classifier's confidence level in the result.

[0085] Specifically, a primary recognition module is deployed in the server. After being trained with a large number of actual call videos and synthetic call videos, the module has the ability to distinguish between the two. When the server obtains the key video frame features of the pending call video to be classified, it will input these features into the primary recognition module.

[0086] After receiving these video frame features, the primary recognition module will start working. It contains a series of complex algorithms and models, which will perform calculations and analyses based on the input features. During the calculation and analysis process, the primary recognition module will compare the video frame features of the pending call video with the feature patterns of the known actual call video and the synthesized call video to find similarities and differences.

[0087] After a series of operations and analyses, the primary recognition module will output a first recognition result as the classification result. This first recognition result is a preliminary judgment, which indicates the possibility that the pending call video is recognized as an actual call video or a synthetic call video. This result is usually accompanied by a confidence score to indicate the reliability of the recognition result.

[0088] It should be noted that this first recognition result is only a preliminary classification result, and it may not be 100% accurate. Therefore, in practical applications, a comprehensive judgment is usually made in combination with the results of other steps and modules to improve the accuracy of classification. For example, the server may decide whether to further process the pending call video based on the confidence score of the first recognition result. If the confidence score is high, the server may directly classify the pending call video as an actual call video or a synthesized call video; if the confidence score is low, the server may send the pending call video to a more advanced recognition module for further processing and analysis.

[0089] To further improve the accuracy of classification, the server analyzes the sequence information of key video frame features, such as the stability and consistency of features over time. At the same time, combined with the classification coefficient, the server calculates a reliability coefficient to evaluate the reliability of the current classification result. If the feature sequence shows a high degree of consistency and stability, and the classification coefficient is high, the reliability coefficient will be increased accordingly.

[0090] The server clusters key video frame features based on the reliability coefficients. This means that video frames with similar features and reliability coefficients are grouped together to form different clusters. Each cluster represents a specific pattern or feature combination in the video.

[0091] The server extracts the most representative information from these clusters and generates a summary content that concisely describes the basis and reasons for classifying the pending call video as a real or synthetic call video.

[0092] Finally, the server uses the generated summary content and other available information (such as video metadata, source, etc.) to perform advanced real / AI classification on the pending call video. This process may involve the collaborative work of multiple models and algorithms to ensure the accuracy and reliability of the classification results.

[0093] Specifically, the server can perform advanced real / AI classification on the pending call video according to the summary content through the advanced recognition module to obtain the final recognition result of the pending call video.

[0094] Specifically, in the server, there is an advanced recognition module that is designed to process the pending call videos after preliminary classification, especially those that require more sophisticated analysis to determine their authenticity. This advanced recognition module not only considers basic video features, but also incorporates complex pattern recognition and deep learning algorithms.

[0095] Therefore, when the server obtains a summary content, since the summary content contains the key features and classification basis of the pending call video, the server can then pass the summary content to the advanced recognition module.

[0096] After receiving the summary content, the advanced recognition module starts working. It first parses the key information in the summary content, such as the feature sequence of video frames, spectral analysis of audio signals, etc. Then, it uses the internal deep learning model to perform higher-level abstraction and representation learning of these features. In this process, the model captures more subtle temporal dependencies between video frames, subtle changes in facial expressions, naturalness of voice, and other features that are difficult to capture with traditional methods. As the deep learning model operates, the advanced recognition module makes a judgment on the authenticity or AI synthesis of the pending call video based on the information in the summary content. This judgment is not only based on the intuitive features of the video and audio, but also takes into account the changing patterns and inherent logical relationships of these features in the time dimension.

[0097] Finally, the advanced recognition module will output a final recognition result. This result is the final judgment on the authenticity of the pending call video, indicating whether the video is recognized as a real call or an AI-synthesized call. Compared with the primary recognition result, this final recognition result has undergone more sophisticated processing and deeper analysis, so it has higher accuracy and reliability.

[0098] Finally, the server will output a final recognition result, which clearly indicates whether the pending call video is identified as an actual call video or a synthetic call video, and can be accompanied by a confidence level indicator. This result can be used in subsequent application scenarios, such as content review, security monitoring, user feedback, etc.

[0099] In this application, when the initial verification result indicates that the call video to be verified is of a false type, the accuracy and reliability of recognition can be improved by further performing a second verification. Since there is no need to use multiple recognition models and special effect tag libraries, the complexity of the processing flow is effectively reduced. Therefore, the complexity of the processing flow can be reduced while ensuring the accuracy of recognition, thereby improving the processing efficiency when identifying false video calls.

[0100] In one embodiment, after obtaining the target verification result of the call video to be verified, at least one of the following is further included: According to the target verification result, determine the target pending call video corresponding to the classification result that the call video to be verified is identified as the target video call, and mark the target pending call video; According to the target verification result, the call video to be verified is sent to the isolation server corresponding to the classification result that the call video to be verified is identified as the target video call; According to the target verification result, an alarm pop-up window corresponding to the classification result that the call video to be verified is identified as a target video call is generated.

[0101] After obtaining the final recognition result of the pending call video, the server of this application can determine whether the video belongs to a real call or an AI synthesized call based on this result. Then, the server marks the video identified as a specific category so that it can be quickly identified and classified later. Among them, this application can define the AI ​​synthesized call as the target video call.

[0102] Assume that the server processes a pending call video and identifies it as an AI-synthesized call through the advanced recognition module. The server will add a tag, such as "AI-synthesized", to the video data, so that any system or user who needs to identify the nature of the video can quickly learn this information during future retrieval or processing.

[0103] At the same time, in order to ensure network security and the protection of user data, the server can send the pending call video to the isolation server corresponding to the recognition result based on the final recognition result. This can prevent potential malicious video content from spreading in the network. Continuing with the AI ​​synthetic call video above as an example, after the server determines the nature of the video, it will send it to an isolation server dedicated to storing and processing AI synthetic content. This isolation server can have stricter access control and security measures to prevent these videos from being improperly used or spread.

[0104] In addition, in order to promptly notify relevant personnel to pay attention and take action, the server can also generate an alarm pop-up window corresponding to the recognition result based on the final recognition result. This pop-up window usually contains key information about the recognition result and suggestions for actions that need to be taken. When the server identifies a pending call video as an AI synthetic call and believes that this may pose a security threat, it will generate an alarm pop-up window and send it to the network administrator. This pop-up window can contain a screenshot of the video, detailed information on the recognition result, and recommended measures (such as further checking the source of the video, strengthening network security measures, etc.). In this way, network administrators can quickly understand this potential security issue and take appropriate actions.

[0105] This application can perform processing operations such as marking, isolating or alarming the call video to be verified based on the target verification result, thereby improving the security of the video call process.

[0106] The video call verification device provided by the present application is described below. The video call verification device described below and the video call verification method described above can be referenced to each other.

[0107] Furthermore, the present application also provides a video call verification device.

[0108] The video call verification device may also include: an acquisition module, used to obtain a call video to be verified; a first classification module, used to input the call video to be verified into a false call video recognition model to obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training the initial model based on a recognition cost parameter and a feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false; a second classification module, used to perform a secondary verification on the call video to be verified if the initial verification result is that the call video to be verified is a false type, so as to obtain a target verification result of the call video to be verified.

[0109] In one embodiment, the second classification module is further used to: obtain training examples, the training examples including sample pending call videos and matching actual classification identifiers and sample confidences; perform feature recognition on the sample pending call videos through the primary recognition module to obtain sample video frame features corresponding to the sample pending call videos; infer video call classification results of the sample pending call videos based on the sample video frame features to obtain inferred classification identifiers and inferred confidences; calculate sample reliability coefficients of the sample video frame features for the inferred video call classification results of the sample pending call videos based on sequence information of the sample video frame features and the inferred confidences; wherein the sample video frame features include multiple pixel-level sample video frame features corresponding to each feature extractor obtained by performing feature recognition on the sample pending call videos in sequence according to multiple feature extractors of the primary recognition module; determine a target feature extractor from the multiple feature extractors; and perform a sample reliability coefficient on the multiple pixel-level sample video frame features of the target feature extractor based on the sample video frame features extracted by the target feature extractor. A clustering operation is performed to generate a sample summary content for describing the classification of the sample pending call video into an inferred classification identifier; wherein the training instance also includes a video call classification result summary content corresponding to the sample pending call video; a summary content set is generated according to the video call classification result summary content and the sample summary content; a target summary content is determined from the summary content set; a sample feature map indicating the sample pending call video is generated according to the target summary content; the sample pending call video and the sample feature map are integrated to obtain an integrated pending call video; an advanced video call classification result inference is performed on the sample pending call video according to the integrated pending call video through an advanced recognition module to obtain a target inferred classification identifier; an error between the inferred classification identifier and the actual classification identifier is determined to obtain an actual generation parameter, and an error between the target inferred classification identifier and the actual classification identifier is determined to obtain an inferred generation parameter; a network architecture of a primary recognition module is optimized according to the actual generation parameter, and a network architecture of a high-level recognition module is optimized according to the inferred generation parameter until a preset state is reached.

[0110] In one embodiment, the second classification module is specifically used to: perform feature recognition on the call video to be verified to obtain video frame features; the video frame features include at least one of facial expression features, eye features, and audio abnormality features; use the primary recognition module to classify the video according to the video frame features to obtain a classification result; the classification result includes the classification type and its classification coefficient; classify the video according to the video frame features and the classification coefficient to obtain a target verification result of the call video to be verified.

[0111] In one embodiment, the second classification module includes a classification unit, which is used to: determine the reliability coefficient of the video frame feature for the classification of the call video to be verified based on the sequence information of the video frame feature and the classification coefficient; cluster the video frame features according to the reliability coefficient and generate summary content for describing the call video to be verified as the corresponding video call classification result based on the clustering result; use the advanced recognition module to classify the call video to be verified according to the summary content to obtain the target verification result of the call video to be verified.

[0112] In one embodiment, the second classification module is also used to: determine the target pending call video corresponding to the classification result that the call video to be verified is identified as a target video call according to the target verification result, and mark the target pending call video; send the call video to be verified to the isolation server corresponding to the classification result that the call video to be verified is identified as a target video call according to the target verification result; generate an alarm pop-up window corresponding to the classification result that the call video to be verified is identified as a target video call according to the target verification result.

[0113] The video call verification device provided in the embodiment of the present application obtains a false call video recognition model by pre-training an initial model for identifying whether a call video is real or false based on a recognition cost parameter and a feature distance limitation, wherein the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video, and the feature distance limitation is used to limit the feature distance between the real visual feature and the false visual feature, and the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; and then after obtaining the call video to be verified, the call video to be verified is input into the false call video recognition model. Since the false call video recognition model is trained using the recognition cost parameter and the feature distance limitation Therefore, it can more accurately identify real visual features and false visual features, so that an accurate initial verification result of the false call video recognition model output can be obtained; further, when the initial verification result of the present application is that the call video to be verified is of a false type, the call video to be verified can be further verified for the second time to obtain a more accurate target verification result of the call video to be verified, thereby improving the accuracy and reliability of false video call recognition, since there is no need for the use of multiple recognition models and special effects tag libraries, the complexity of the processing flow is effectively reduced, so the complexity of the processing flow can be reduced while ensuring the recognition accuracy, thereby improving the processing efficiency of false video call recognition.

[0114] Figure 2 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 2As shown, the electronic device may include: a processor (processor) 210, a communication interface (Communications Interface) 220, a memory (memory) 230 and a communication bus 240, wherein the processor 210, the communication interface 220, and the memory 230 communicate with each other through the communication bus 240. The processor 210 can call the logic instructions in the memory 230 to execute the video call verification method, which includes: obtaining a call video to be verified; inputting the call video to be verified into a false call video recognition model to obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training the initial model based on the recognition cost parameter and the feature distance limitation; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limitation is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false; if the initial verification result is that the call video to be verified is of a false type, the call video to be verified is secondary verified to obtain the target verification result of the call video to be verified.

[0115] In addition, the logic instructions in the above-mentioned memory 230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0116] On the other hand, the present application also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video call verification method provided by the above methods, which method includes: obtaining a call video to be verified; inputting the call video to be verified into a false call video recognition model to obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training an initial model based on a recognition cost parameter and a feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or false; if the initial verification result is that the call video to be verified is a false type, the call video to be verified is secondary verified to obtain the target verification result of the call video to be verified.

[0117] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the video call verification method provided by the above-mentioned methods, the method comprising: obtaining a call video to be verified; inputting the call video to be verified into a false call video recognition model to obtain an initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training an initial model based on a recognition cost parameter and a feature distance limitation; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limitation is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether a call video is real or false; if the initial verification result is that the call video to be verified is a false type, then the call video to be verified is subjected to a secondary verification to obtain a target verification result of the call video to be verified.

[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video call verification method, It is characterized in that The method comprises: Get the video of the call to be verified; The call video to be verified is input into the false call video recognition model to obtain the initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training the initial model based on the recognition cost parameter and the feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or fake; If the initial verification result is that the call video to be verified is of a false type, a secondary verification is performed on the call video to be verified to obtain a target verification result of the call video to be verified.

2. The video call verification method according to claim 1, It is characterized in that The fake call video recognition model is obtained in the following way: Obtaining an initial model and a sample call video; the sample call video includes an actual call video and a synthesized call video; the synthesized call video is generated by artificial intelligence based on the actual call video; Using the initial model, sequentially perform feature recognition on the actual call video and the synthesized call video to obtain a true recognition type of the actual call video and a false recognition type of the synthesized call video; Determine an identification cost parameter according to a real deviation between a real identification type of the actual call video and a real type of the actual video call, and a false deviation between a false identification type of the synthetic call video and a real type of the synthetic call video; Get feature distance limit; The initial model is trained based on the recognition cost parameter and the feature distance limit to obtain a fake call video recognition model; wherein the real visual feature is obtained by using the initial model to perform feature recognition on the actual call video, and the false visual feature is obtained by using the initial model to perform feature recognition on the synthetic call video.

3. The video call verification method according to claim 2, It is characterized in that The initial model includes a feature recognition module; The feature distance limitation includes a real feature distance cost parameter, and the real feature distance cost parameter is determined in the following manner: According to the feature distance between the real visual feature and the false visual feature of each feature recognition module, respectively determine the false feature distance cost parameter of the corresponding feature recognition module; Obtaining an important coefficient associated with each feature recognition module; the important coefficient is determined by the position of the corresponding feature recognition module in the entire feature recognition module composed of each feature recognition module; Determine the weighted linear calculation parameters of the false feature distance cost parameters of each feature recognition module according to the important coefficients associated with each feature recognition module and the false feature distance cost parameters; Based on each weighted linear calculation parameter, the true feature distance cost parameter is determined.

4. The video call verification method according to claim 1, It is characterized in that Also included is a primary recognition module for distinguishing a call video as an actual call video or a synthesized call video, and an advanced recognition module having a higher recognition capability than the primary recognition module; Before performing secondary verification on the call video to be verified, the method further includes: Obtaining a training instance, wherein the training instance includes a sample pending call video and a matching actual classification identifier and a sample confidence; Performing feature recognition on the sample pending call video through a primary recognition module to obtain sample video frame features corresponding to the sample pending call video; Inferring a video call classification result of the sample pending call video according to the sample video frame features to obtain an inferred classification identifier and an inferred confidence degree; According to the sequence information of the sample video frame features and the inferred confidence, a sample reliability coefficient of the sample video frame features inferred for the video call classification result of the sample pending call video is calculated; wherein the sample video frame features include a plurality of pixel-level sample video frame features corresponding to each feature extractor obtained by sequentially performing feature recognition on the sample pending call video according to a plurality of feature extractors of the primary recognition module; determining a target feature extractor from a plurality of feature extractors; According to the sample reliability coefficient corresponding to the sample video frame feature extracted by the target feature extractor, a clustering operation is performed on multiple pixel-level sample video frame features of the target feature extractor to generate a sample summary content for describing the classification of the sample pending call video into the inferred classification identifier; wherein the training instance also includes the video call classification result summary content corresponding to the sample pending call video; Generate a summary content set according to the video call classification result summary content and the sample summary content; Determine target summary content from the summary content set; Generating a sample feature graph indicating the sample pending call video according to the target summary content; Integrate the sample pending call video and the sample feature graph to obtain an integrated pending call video; Inferring the advanced video call classification result of the sample pending call video according to the integrated pending call video by the advanced recognition module to obtain a target inferred classification identifier; Determine the error between the inferred classification identifier and the actual classification identifier to obtain the actual proxy parameter, and determine the error between the target inferred classification identifier and the actual classification identifier to obtain the inferred proxy parameter; The network architecture of the primary recognition module is optimized according to the actual generation parameters, and the network architecture of the advanced recognition module is optimized according to the inferred generation parameters until a preset state is reached.

5. The video call verification method according to claim 4, It is characterized in that The performing secondary verification on the call video to be verified to obtain a target verification result of the call video to be verified includes: Perform feature recognition on the call video to be verified to obtain video frame features; the video frame features include at least one of facial expression features, eye features, and audio abnormality features; Using a primary recognition module to classify the video according to the video frame features to obtain a classification result; the classification result includes a classification type and a classification coefficient; Video classification is performed according to the video frame features and the classification coefficients to obtain a target verification result of the call video to be verified.

6. The video call verification method according to claim 5, It is characterized in that The performing video classification according to the video frame features and the classification coefficient to obtain the target verification result of the call video to be verified includes: Determining, according to the sequence information of the video frame feature and the classification coefficient, a reliability coefficient of the video frame feature for the classification of the call video to be verified; Performing a clustering operation on the video frame features according to the reliability coefficient and generating summary content for describing the call video to be verified as a classification result of the corresponding video call based on the clustering result; The advanced recognition module is used to classify the call video to be verified according to the summary content to obtain a target verification result of the call video to be verified.

7. The video call verification method according to any one of claims 1 to 6, It is characterized in that After obtaining the target verification result of the call video to be verified, at least one of the following items is also included: According to the target verification result, determining a target pending call video corresponding to the classification result that the call video to be verified is identified as a target video call, and marking the target pending call video; According to the target verification result, sending the call video to be verified to an isolation server corresponding to the classification result that the call video to be verified is identified as a target video call; According to the target verification result, an alarm pop-up window corresponding to the classification result that the call video to be verified is identified as a target video call is generated.

8. A video call verification device, It is characterized in that include: An acquisition module is used to obtain the call video to be verified; The first classification module is used to input the call video to be verified into the false call video recognition model to obtain the initial verification result output by the false call video recognition model; wherein the false call video recognition model is obtained by training the initial model based on the recognition cost parameter and the feature distance limit; the recognition cost parameter is determined based on the deviation between the real type and the recognition type of the sample call video; the feature distance limit is used to limit the feature distance between the real visual feature and the false visual feature; the real visual feature and the false visual feature are both obtained by feature extraction of the sample call video; the initial model is used to identify whether the call video is real or fake; The second classification module is used to perform a secondary verification on the call video to be verified if the initial verification result is that the call video to be verified is of a false type, so as to obtain a target verification result of the call video to be verified.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the video call verification method according to any one of claims 1 to 7 is implemented.

10. A medium, the medium being a non-transitory computer-readable storage medium, having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the video call verification method according to any one of claims 1 to 7 is implemented.

11. A product, the product being a computer program product, comprising a computer program, It is characterized in that When the computer program is executed by a processor, the video call verification method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Living body detection method and device, computer device and storage medium

    CN109858381A

  • Image recognition and neural network model training method, device and system

    CN110188641A

  • Model training method and device, image retrieval method and device, equipment and medium

    CN117216309A

  • Forgery video detection method, device and equipment

    CN117649621A

Cited By

  • AI false information network propagation scene analysis method and system based on large model

    CN120567706A

  • Real-time security assistant for unified communications as a service (UCaaS)

    US12777026B1