Video call method and mobile terminal

Abstract

The application discloses a video call method and a mobile terminal, and relates to the technical field of mobile terminals. The method comprises the following steps: for a video call scene, when a first condition is met, detecting whether a first face in a video call interface has a face changing risk, and displaying a first prompt after detecting that the face changing risk exists; when a second condition is met, displaying a second prompt for detecting the face changing risk, detecting whether a second face in the video call interface has the face changing risk and prompting a detection process in response to a first operation on the second prompt, and displaying a third prompt after obtaining a detection result, wherein the detection result comprises the face changing risk or the face changing risk does not exist. In this way, the face changing detection can be triggered in different ways under different conditions, so that different detection experiences can be provided.

Description

[0001] This application claims priority to Chinese Patent Application No. 202410844074.7, filed with the State Intellectual Property Office of China on June 26, 2024, entitled "Video Call Method and Mobile Terminal", the entire contents of which are incorporated herein by reference. This application is a divisional application of Chinese Patent Application No. 202411380212.7, filed with the State Intellectual Property Office of China on September 29, 2024, entitled "Video Call Method and Mobile Terminal". Technical Field

[0002] This application relates to the field of mobile terminal technology, and in particular to a video call method and a mobile terminal. Background Technology

[0003] Mobile devices such as smartphones and tablets typically offer video calling functionality. For example, if a social chat application is installed on the mobile device, it can provide video calling capabilities through that application.

[0004] During video calls, both parties can see each other's faces, making it easier to verify each other's identities and build mutual trust. However, some individuals exploit this vulnerability, using artificial intelligence (AI) face-swapping technology to replace the face of the person on the other end of the video call with that of a familiar person, such as a relative, friend, or superior, in order to gain the other party's trust. They then request money transfers or payment assistance, which can easily lead to financial losses and a negative user experience. Summary of the Invention

[0005] This application provides a video call method and a mobile terminal. During a video call, if there is a risk of face swapping, the user can be promptly notified, thus improving the call experience.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: Firstly, this application provides a video call method applied to a mobile terminal. Specifically, after establishing a video call, it can detect whether there is a risk of face swapping in the face displayed on the video call interface. Face swapping detection can be triggered in different ways under different conditions. Specifically, when a first condition (the detection condition in the active detection section below) is met, it detects whether there is a risk of face swapping in the first face displayed on the video call interface. If a face swapping risk is detected, a first prompt (prompt e in the section below) is displayed. In other words, if the first condition is met, it actively detects whether there is a risk of face swapping and provides a prompt when a risk is detected. Therefore, the active detection method, except when a face swapping risk is detected, will not affect the video call.

[0007] If the second condition is met (as indicated by prompts g or h below), a second prompt for detecting face-swapping risk is displayed (as indicated by prompts g or h below). In response to the first action on the second prompt (such as triggering the detection control), the system checks for face-swapping risk and displays the detection process. After obtaining the detection result, a third prompt (as indicated by prompt j below) is displayed, indicating whether face-swapping risk exists or not. In other words, if the second condition is met, a prompt for detecting face-swapping risk can be displayed first, and then passive detection for face-swapping risk is triggered only after user confirmation. During passive detection, the detection process is displayed, such as showing the animation of the detection in progress, and prompts are displayed regardless of whether there is a risk or not. Therefore, in passive detection, both the detection process and the detection result are visible, allowing users to understand the detection process and result after confirming the detection.

[0008] The first and second conditions mentioned above differ. Generally, the first condition has lower requirements, while the second condition has higher requirements. In other words, the first condition is easier to meet, while the second condition is more difficult. Therefore, in most cases, braking detection can be used to detect face-swapping risks, reducing the impact on video calls. In a few cases, passive detection can be used to detect face-swapping risks, allowing users to understand the detection process and results.

[0009] In summary, the solution proposed in this application can trigger face-swapping detection and prompts in different ways under different conditions, thereby providing different detection experiences. For example, in the active detection method, there is no risk of face-swapping, while in the passive detection method, there is awareness throughout the process.

[0010] In one possible design approach of the first aspect, the first condition or the second condition includes at least one of the following: The time difference between adding the first user as a friend and the start of the video call is within a first duration (such as duration 5, 6, or 10 in the following text), meaning the first user is a friend added within a short period. The first user is an account logged in on the other end device that established the video call with the mobile terminal.

[0011] The first user is the first friend added. For example, the first friend added after a chat application is installed on a mobile device.

[0012] The video call is the first video call with the first user.

[0013] The video call is initiated by the first user.

[0014] The first user is in the detection list of the mobile terminal (list 1 below). The detection list records the list of the first number of users recently added to the mobile terminal (number 1 below). That is, the first user is one of the certain number of friends recently added to the mobile terminal.

[0015] Mobile devices may experience risk events, such as receiving risky information, receiving overseas calls, downloading unknown applications and performing operations such as entering bank account numbers, transferring money, or sharing screens.

[0016] The second moment at which the risk event is detected on the mobile terminal is within a second duration, which is the time difference between the start time of the video call and the second moment at which the risk event is detected on the mobile terminal. That is, if a video call is received shortly after the risk event is detected on the mobile terminal, face-swapping detection can be triggered for that video call. The second duration may be the same as or different from the first duration.

[0017] The mobile terminal is deemed to pose a risk based on multiple related risk behavior factors. For example, if a user receives an overseas call and then downloads unknown applications and shares their screen shortly afterward, face-swapping detection can be triggered for such video calls.

[0018] The time difference between the third moment and the start time of the video call is within a third duration; wherein, the third moment is the moment when the mobile terminal is determined to be at risk based on multiple related risk behavior factors. That is, if a video call is received shortly after determining that the mobile terminal is at risk based on a series of related risk behavior factors, face-swapping detection can be triggered for that video call. The third duration may be the same as or different from the second or first duration.

[0019] In other words, mobile terminals can detect or prompt for face-swapping risks in video calls that may involve face swapping, thereby improving the targeting of face-swapping detection and avoiding a large number of invalid detections.

[0020] In one possible design approach of the first aspect, the aforementioned display of the first prompt includes: displaying the first prompt on the video call interface when the video call has not ended while a face-swapping risk is detected, the first prompt including a re-detection control and / or a screen recording control; and displaying the first prompt on the interface after the video call has ended while a face-swapping risk is detected, the first prompt not including the re-detection control and the screen recording control; wherein the re-detection control is used to trigger the mobile terminal to re-execute face-swapping detection; and the screen recording control is used to record the screen of the video call interface after being triggered.

[0021] In this way, a primary warning will be displayed whenever a face-swapping risk is detected, thus indicating the presence of face-swapping. Furthermore, since re-detection and screen recording cannot be performed after the video call ends, if the video call has ended when a face-swapping risk is detected, re-detection and screen recording controls will not be provided in the primary warning to avoid providing invalid controls. In addition, providing screen recording controls to record the video call interface allows for the preservation of recordings of video calls with face-swapping risks, which can be used as evidence when necessary, or for updating the face-swapping detection model.

[0022] In one possible design approach of the first aspect, the detection result indicates no risk of face swapping. After obtaining the detection result, a third prompt is displayed, including: if the video call is still ongoing when the detection result is obtained, a third prompt is displayed on the video call interface, including a re-detection control and / or a screen recording control. If the video call has ended when the detection result is obtained, a third prompt is displayed on the interface after the video call ends, excluding the re-detection control and screen recording control. For details, please refer to the aforementioned explanation of the first prompt; it will not be repeated here.

[0023] In one possible design approach of the first aspect, the video call interface displays a prompt including a re-detection control (such as the first or third prompt mentioned above). This includes displaying the prompt including the re-detection control on the video call interface if the number of times the video call is checked for face-swapping risk does not exceed the first count (the count 1 in the following text). This avoids malicious detection targeting the same video call.

[0024] In one possible design approach of the first aspect, if the number of times the video call is checked for the risk of face swapping exceeds the first count, a prompt without a re-detection control is displayed on the video call interface.

[0025] In one possible design approach of the first aspect, the mobile terminal may display a "detection incomplete" message if the second condition is met and the second face is not detected; or, if the second condition is met and the video call ends before a video frame is acquired, the mobile terminal may display a "detection incomplete" message on the interface indicating the end of the video call. In other words, in the passive detection method, for abnormal situations such as failure to detect a face or acquisition of image frames that prevent detection from being completed, a "detection incomplete" message can be displayed, allowing the user to clearly identify the abnormality.

[0026] Conversely, if the first condition is met and the first face is not detected, or if the video call ends before a video frame is acquired, no notification of incomplete detection will be displayed. In other words, in the active detection method, no notification will be displayed for abnormal situations such as failure to detect a face or acquisition of image frames that prevent detection from being completed, to avoid interfering with the user.

[0027] In one possible design approach of the first aspect, the detection process is indicated by displaying at least one of the following: a prompt to start detection (e.g., "Start Detection"), a prompt that is detecting (e.g., "Detection in progress"), and a detection animation.

[0028] In one possible design approach of the first aspect, displaying a second prompt regarding the risk of face-swapping includes: When the second prompt is displayed for the first to Nth times, it is displayed in card form, including a description of the face-swapping detection function and / or a confirmation control (such as the detection control, activation and detection control, etc., described below). When the second prompt is displayed for the (N+1)th time, it is displayed in capsule form, excluding the description of the face-swapping detection function and the confirmation control. Here, N is a natural number greater than 1, such as N=3.

[0029] This allows the user to be shown the details of the prompt during the first few displays of the second prompt, so that the user understands the content of the prompt. When the second prompt is displayed again, a capsule is shown to the user, reducing the impact on the call.

[0030] In one possible design approach of the first aspect, the second condition includes a first sub-condition and a second sub-condition. When the second condition is met, a second prompt indicating the risk of face-swapping is displayed, including: when the first sub-condition (as in prompt g below) is met, displaying the second prompt indicating the risk of face-swapping (as in prompt g below), where the first sub-condition includes the face-swapping detection switch on the mobile terminal being enabled; and when the second sub-condition (as in prompt h below) is met, displaying a second prompt indicating that the face-swapping detection function is enabled and detected (as in prompt h below), where the second sub-condition includes the face-swapping detection switch on the mobile terminal being disabled.

[0031] Furthermore, the first sub-condition has lower requirements, while the second sub-condition has higher requirements. In other words, the first sub-condition is easier to meet, while the second sub-condition is more difficult. In practice, mobile terminals have risk protection enabled by default, which corresponds to enabling the face-swapping detection function by default. If the face-swapping detection switch is off, meaning the face-swapping detection function is not enabled, it indicates that the user has likely manually disabled the face-swapping detection function. In this case, the second sub-condition can prompt the user to enable the face-swapping detection function at a lower frequency, avoiding frequent prompts for unnecessary functions and causing user annoyance. However, when the face-swapping detection function is enabled, the first sub-condition can prompt the user to use the face-swapping detection function to detect face-swapping risks at a slightly higher frequency, thereby increasing the usage rate of the face-swapping detection function.

[0032] In one possible design approach, the mobile terminal can, under the condition of satisfying a first condition, capture video frames from the video call interface; perform face region detection on the captured video frames, identify the face regions in the detected face regions that meet the face constraints of the other end as the first face; and perform face-swapping risk detection based on the first face in the video frames.

[0033] In the above scheme, by intercepting video frames in a video call, the face region detection can accurately identify the face region of the other end that needs to be detected for face swapping risk, thereby improving the accuracy of subsequent face swapping risk detection.

[0034] In one possible design approach, the face constraints at the other end include preset positional constraints and / or preset size constraints.

[0035] In the above scheme, the mobile terminal can accurately identify the face region of the other end based on preset position constraints and / or preset size constraints.

[0036] In one possible design approach, before identifying a face region in the detected face region that satisfies the other party's face constraint as the first face, the mobile terminal can obtain the number of window switching times when capturing the video frame; the number of window switching times refers to the number of times the display windows of the two parties in the video call interface are exchanged when capturing the video frame; based on the number of window switching times, the other party's face constraint corresponding to the video frame is determined.

[0037] The above scheme takes into account the situation of switching windows during video calls. Based on the statistical number of window switching, it is more accurate to determine the face constraint conditions of the other end for identifying the face region, thereby enabling accurate identification of the other end's face region.

[0038] In one possible design approach, multiple video frames are captured. The mobile terminal can perform face-swapping recognition on each captured video frame based on the first face in the video frame to obtain the face forgery confidence level corresponding to the video frame. The face forgery confidence levels corresponding to the multiple video frames are fused to obtain a target confidence level. The presence of face-swapping risk is determined based on the target confidence level.

[0039] In the above scheme, the mobile terminal can more accurately identify the face-swapping risk of the current video call based on the fusion result of the face forgery confidence of multiple video frames.

[0040] In one possible design approach, the plurality of video frames are extracted from the video call according to at least one of the preset interception parameters: interception timing, interception frequency, or total interception duration.

[0041] In the above scheme, during video calls, video frames are truncated based on preset truncation parameters, which can obtain video frames that meet the requirements for face-swapping detection, thereby improving the accuracy of subsequent face-swapping risk detection.

[0042] In one possible design approach, the mobile terminal can filter the face forgery confidence scores corresponding to a portion of the video frames from the face forgery confidence scores corresponding to the multiple video frames respectively; wherein the filtered face forgery confidence scores are higher than the unfiltered face forgery confidence scores; and the filtered face forgery confidence scores are weighted and fused to obtain the target confidence score.

[0043] The above scheme can obtain the face-swapping risk identification result without detecting all frames, saving system resources and improving detection efficiency.

[0044] In one possible design approach, after the mobile terminal captures a video frame and calculates the confidence level of face spoofing for that video frame, it updates and stores the confidence level of face spoofing for that video frame in an array. Then, it performs a weighted fusion of the currently stored confidence levels in the array to obtain a target confidence level. Furthermore, the mobile terminal can identify the risk level of face-spoofing based on the target confidence level; if the risk level is medium risk, it continues to capture the next video frame; if the risk level is low risk or high risk, it stops capturing video frames, and in the case of a high risk level, it determines that face-spoofing risk exists.

[0045] In the above scheme, by combining an array, the comprehensive target confidence level corresponding to each captured video frame can be calculated to identify the risk level of the video call. Only when the risk is medium will video frames continue to be captured. If the risk is low or high, video frames do not need to be captured. This saves system resources and can accurately identify the risk of face swapping.

[0046] Secondly, this application also provides a mobile terminal, which includes a display screen, a memory, and one or more processors. The display screen, memory, and processors are coupled. The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, they cause the mobile terminal to perform the methods described in the first aspect and any of its possible designs.

[0047] Thirdly, this application provides a chip system applied to a mobile terminal including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the interface circuits are used to receive signals from the memory of the mobile terminal and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the mobile terminal performs the method as described in the first aspect and any of its possible design embodiments.

[0048] Fourthly, this application provides a computer-readable storage medium including computer instructions that, when executed on a mobile terminal, cause the mobile terminal to perform a method as described in the first aspect and any of its possible design embodiments.

[0049] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method as described in the first aspect and any of its possible design methods.

[0050] Understandably, the beneficial effects achieved by the mobile terminal of the second aspect, the chip system of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0051] Figure 1A A scenario diagram provided for an embodiment of this application; Figure 1B A hardware structure diagram of a mobile phone provided in an embodiment of this application; Figure 1C A software structure diagram of a mobile phone and a flowchart of a video call method based on the software structure diagram are provided for embodiments of this application. Figure 1D This is one of the schematic diagrams of the interface for enabling face-swapping detection provided in the embodiments of this application; Figure 2 This is a second schematic diagram of the interface for enabling face-swapping detection provided in an embodiment of this application; Figure 3This is a schematic diagram of the interface for enabling the system manager service provided in an embodiment of this application; Figure 4 A schematic diagram of an interface for understanding face-swapping detection details provided in an embodiment of this application; Figure 5 A schematic diagram of the interface for viewing face-swapping detection records provided in an embodiment of this application; Figure 6 A schematic diagram of the interface for security settings provided in an embodiment of this application; Figure 7 The third schematic diagram of the interface for enabling face-swapping detection provided in the embodiments of this application; Figure 8 A schematic diagram of prompt a and its interface response provided in the embodiments of this application; Figure 9 A schematic diagram of the disappearance process of prompt 'a' provided in an embodiment of this application; Figure 10 A schematic diagram illustrating the prompt d and its interface response provided in the embodiments of this application; Figure 11 A schematic diagram of the active detection scheme provided in the embodiments of this application; Figure 12 A schematic diagram of the interface for the active detection scheme provided in the embodiments of this application; Figure 13 A schematic diagram of the interface for adding friends and starting a video call provided in an embodiment of this application; Figure 14 A schematic diagram of an interface prompting the use of face-swapping detection during a video call, provided in an embodiment of this application; Figure 15A This application provides an embodiment of an interface that prompts users to enable face-swapping detection during a video call. Figure 15B A schematic diagram of another interface prompting the user to enable face-swapping detection during a video call, provided as an embodiment of this application; Figure 16 A schematic diagram of the passive detection scheme provided in the embodiments of this application; Figure 17 One of the interface diagrams of the passive detection scheme provided in the embodiments of this application; Figure 18 A second schematic diagram of the interface of the passive detection scheme provided in the embodiments of this application; Figure 19 The third schematic diagram of the interface of the passive detection scheme provided in the embodiments of this application; Figure 20 The fourth schematic diagram of the interface of the passive detection scheme provided in the embodiments of this application; Figure 21Fifth schematic diagram of the interface of the passive detection scheme provided in the embodiments of this application; Figure 22 A sixth schematic diagram of the interface of the passive detection scheme provided in the embodiments of this application; Figure 23 This is one of the interface diagrams provided in this application embodiment for situations where there is a risk of face swapping; Figure 24 The second schematic diagram of the interface provided in this application embodiment for situations where there is a risk of face swapping; Figure 25A The third schematic diagram of the interface provided in this application embodiment for situations where there is a risk of face swapping; Figure 25B The fourth schematic diagram of the interface provided in this application embodiment for situations where there is a risk of face swapping; Figure 26 One of the schematic diagrams of the interface for disabling face-swapping detection provided in the embodiments of this application; Figure 27 This is a second schematic diagram of the interface for disabling face-swapping detection provided in an embodiment of this application; Figure 28 A flowchart illustrating a video call method provided in this application embodiment; Figure 29 A schematic diagram of a video call interface provided in an embodiment of this application; Figure 30 This is a schematic diagram illustrating the interface changes when adding a friend triggers face-swapping risk detection, as provided in an embodiment of this application. Figure 31 A schematic diagram illustrating the interface changes for triggering passive detection when adding a friend, as provided in an embodiment of this application. Figure 32 A flowchart illustrating a software-based video call method provided in an embodiment of this application; Figure 33 A timing diagram of a video call method provided in an embodiment of this application; Figure 34 A timing diagram illustrating face-swapping detection and processing steps provided in an embodiment of this application; Figure 35 A timing diagram of a passive detection-based video call method provided in an embodiment of this application; Figure 36 A timing diagram for another passive detection-based video call method provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0053] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. The terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," and "third," etc., may explicitly or implicitly include one or more of that feature.

[0054] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0055] In video call scenarios, such as Figure 1AAs shown, user X on the other end (e.g., a PC) adds Lisa (the user's device user) as a friend and then uses face-swapping software installed on the PC. The PC uses a face-swapping AI algorithm to forge a face in the video stream captured by the camera and a "voice changer" AI algorithm to forge the voice in the audio stream captured by the microphone. The forged face and voice are then combined using AI deep synthesis to impersonate Lisa's friend David during a video call. For example... Figure 1A In this video call scenario on the PC, the facial features displayed on the screen are different from those of Lisa's real face (such as the mouth, nose, eyebrows, and eyes). Instead, they resemble the face of Lisa's friend, David. This is achieved using a facial spoofing AI algorithm. After the fake face initiates a video call to Lisa on the PC, it sends the forged video stream to the user's device based on video encoding standards such as H.265 or H.264 and transmission protocols such as RTP or RTCP. The user's device then displays the fake face in the video call interface. The user believes that David is initiating the video call. Thus, the user (Lisa) is easily misled into believing that she is having a video call with a fake friend, potentially suffering financial losses or privacy breaches due to this misunderstanding.

[0056] To avoid the aforementioned adverse effects, this application proposes a video call method applied to a mobile terminal, enabling the mobile terminal to be used by a user (e.g., Figure 1A When Lisa (from the video) is used for video calls, it can detect the risk of face-swapping in real time and provide a warning. For example, Figure 1A As shown, the detection mechanism for face-swapping risk may include extracting images from the video stream (i.e., video frames), detecting whether the other end's face in the video frame has been swapped, for example, by detecting face-swapping boundaries, eye expressions, eyebrows, lips, and other synthetic defects in the video frame, and / or performing inter-frame coherence, image flickering, and jitter detection to identify face-swapping risk. It should be noted that this is only a simplified illustrative description of face-swapping risk detection, and this application does not limit the specific means of detecting face-swapping risk. For example, a mobile terminal can detect the existence of face-swapping risk based on an AI model. Another example is that a mobile terminal can compare a face with a user's whitelist of faces to detect the existence of face-swapping risk. Yet another example is that a mobile terminal can detect the existence of face-swapping risk by analyzing facial expressions and body movements.

[0057] For example, a mobile terminal may have a video call application (APP) installed, which can be referred to as a "video call application". The mobile terminal can make video calls through these APPs. The video call scenarios further include scenarios where the user actively initiates and enters a video call, and scenarios where the user passively receives and joins a video call. Using the video call method provided in this application embodiment, the mobile terminal can detect and alert users in real time to the risk of face-swapping during a video call using a video call application.

[0058] In some embodiments, the methods of this application embodiment can be executed for specific video calling applications. Specifically, the mobile terminal can obtain a pre-configured application whitelist (hereinafter referred to as the "whitelist"). For video calling applications within the whitelist, the methods of this application embodiment can be executed. That is, when a user uses a video calling application within the whitelist for a video call, the methods of this application embodiment can be executed to detect face-swapping risks and provide a prompt. This allows for targeted and more flexible face-swapping risk control in video calls. Furthermore, no complex operations or version updates are required; by modifying the whitelist, face-swapping risk control for video calling applications can be more conveniently implemented.

[0059] It should be understood that traditional forgery detection methods typically rely on the cloud, resulting in poor processing speed and inadequate user privacy, thus posing security risks. Alternatively, they may require high-end hardware, which is costly and unsuitable for mobile video call scenarios. This makes it impossible to perform timely risk detection during mobile video calls, limiting the security protection offered by these methods.

[0060] Using the video call method provided in this application, the mobile terminal can detect face-swapping risks in a timely and real-time manner during video calls and provide alerts, improving the timeliness of risk detection. Furthermore, face-swapping detection (face-swapping risk detection) can be implemented on the mobile terminal itself, without requiring additional large amounts of memory or other system resources. In addition, since face-swapping detection is performed by the mobile terminal itself during the video call, user privacy data can be processed locally on the mobile terminal, improving security and protecting user privacy.

[0061] Specifically, the video call method provided in this application mainly involves performing face-swapping detection on the face image of the other party during a video call on a mobile terminal. This allows for real-time identification of whether the face image of the other party is forged or synthesized, i.e., identifying whether there is a risk of face-swapping in the face image of the other party. Furthermore, if a face-swapping risk exists, a risk warning can be issued; if no face-swapping risk exists, no warning can be issued or a warning of no risk can be displayed.

[0062] For example, the aforementioned mobile terminal can be a mobile phone, tablet, PC, or other mobile terminal that can provide video calling functionality, such as a mobile terminal with an app that supports video calling installed. This application does not impose any special limitations on the specific form of the mobile terminal. The following mainly uses a mobile phone as an example to illustrate the solution of this application.

[0063] See Figure 1B See Figure 1B The mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0064] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the mobile phone. In other embodiments of this application, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0065] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0066] In some embodiments, the mobile phone can use the processor 110 to complete the video call method, thereby detecting the risk of face swapping and providing a prompt during the video call.

[0067] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0068] The mobile phone implements its display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0069] The display screen 194 is used to display images, videos, etc. In some embodiments, the mobile phone can display a video call interface, a warning message about the risk of face swapping, etc. on the display screen 194.

[0070] Mobile phones can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0071] Mobile phones can perform audio functions, such as music playback and recording, through components like the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0072] Button 190 may include a power button, volume buttons, etc. Button 190 may be a mechanical button or a touch button. The mobile phone can receive button input and generate key signal inputs related to the phone's user settings and function control. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 may be an indicator light, used to indicate charging status, battery level changes, or messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the mobile phone.

[0073] Mobile phone software systems can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses a layered architecture software system as an example to illustrate the software structure of a mobile phone. It should be noted that this application is not limited to the software structure of mobile phones.

[0074] Figure 1C This is a software structure block diagram of a mobile phone provided in an embodiment of this application.

[0075] As we can understand, a layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 1CAs shown, a software system may include an application layer, an application framework layer, and a native service layer.

[0076] The application layer can include a series of application packages. For example... Figure 1C As shown, the application package may include a third-party video calling application, a perception module, and a detection module. The perception module is used for Activity behavior awareness; for example, it can detect incoming video call events and preset risk events. The detection module includes a face-swapping detection module and a face-swapping risk identification and handling module.

[0077] The face-swapping detection module is used to identify whether the face displayed on the other end of a video call is forged using AI face-swapping technology. The face-swapping risk identification and handling module is used to confirm whether there is a risk based on the identification results of the face-swapping detection module and to take appropriate measures.

[0078] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0079] The application framework layer can include a window manager (WindowManager). The window manager manages window applications. It can obtain the screen size, determine if a status bar is present, lock the screen, take screenshots, and record the screen (e.g., recording the screen of a video call).

[0080] The application framework layer may also include Figure 1C Other modules not shown include, for example, the view system and the notification manager.

[0081] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including notification icons can include views for displaying text and views for displaying images.

[0082] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, in this embodiment, the notification manager can output prompts to the user, such as alerting the user that there is a risk of face-swapping on the other end of a video call, or prompting the user to turn on the detection switch.

[0083] The local service layer resides between the application framework layer and the kernel layer. This layer may include a Surface Flinger. The Surface Flinger manages, composites, and renders the system's graphical interface, ensuring that the application's and system's interfaces are displayed correctly and efficiently on the screen. For example, in this embodiment, the Surface Flinger provides video frames during a video call.

[0084] It should be noted that the software structure is not limited to the above-mentioned hierarchical structure, and may also include the kernel layer, driver layer (such as display driver), etc., which will not be elaborated further.

[0085] To facilitate understanding, we will now combine Figure 1C The software structure of the video call method in this application embodiment is briefly described as follows: 1. The sensing module senses video call events in video call applications.

[0086] 2. The perception module notifies the detection module of the video call event perception result.

[0087] 3. During a video call, the face-swapping detection module in the detection module captures video frame layer data from the graphics synthesizer through the screenshot interface.

[0088] 4. The image synthesizer returns the truncated video frames to the face-swapping detection module.

[0089] 5. The face-swapping detection module performs face forgery detection inference on video frames and sends the confidence result of the inference to the face-swapping risk identification and handling module.

[0090] The face-swapping risk identification and handling module can identify face-swapping risks and handle them accordingly based on the confidence results of face forgery detection inference. It should be understood that "handling" in the embodiments of this application refers to processing the face-swapping risk identification results.

[0091] The video call method provided in this application can be implemented in a mobile phone with the above-described hardware and software structure. The video call method provided in this application is further described in detail below with reference to the accompanying drawings.

[0092] In some embodiments, after detecting an event that enables face-swapping detection, the mobile phone can activate the face-swapping detection function. When face-swapping detection is enabled, the phone can detect the risk of face-swapping during video calls and provide a warning. Detecting an activation event indicates that the user has granted the necessary permissions and data for face-swapping detection. In this way, the phone can execute the appropriate permissions and obtain data with the user's permission.

[0093] The activation event can be an operation event that skips risk protection settings in the out-of-box experience (OOBE) (Method 1 below), an operation event that turns on the risk protection switch in the settings app (Method 2 below), an operation event that turns on the face-swapping detection switch in the settings app (Method 3 below), or an operation event that performs activation detection on prompt a prompt to turn on risk protection (Method 4 below), etc. This application embodiment does not specifically limit this.

[0094] The following sections will provide a detailed explanation of the various activation events listed above and the process by which they trigger the face-swapping detection function.

[0095] Method 1: Enable the operation event of skipping risk protection configuration in OOBE.

[0096] OOBE refers to the process of configuring a mobile phone during its first boot after leaving the factory, such as configuring the language, region, and network. OOBE can also be interpreted as a boot wizard or boot process.

[0097] In OOBE (Out-of-Body Environment), the phone typically has risk protection enabled by default. Risk protection offers various risk detection functions, such as risky call detection, risky information detection, and face-swapping detection. Enabling risk protection is equivalent to enabling multiple risk detection functions within it.

[0098] In OOBE, the phone detects an operation event where the user skips the risk protection settings, such as triggering the confirmation skip control. This indicates that the user supports enabling risk protection, and the phone can enable risk protection, thereby enabling the face-swapping detection function under risk protection.

[0099] See Figure 1D The configuration interface for risk protection in OOBE is... Figure 1D As shown in the interface 101, the risk protection switch 1011 in interface 101 is turned on by default. In this case, if the user does not turn off the switch 1011, but directly triggers the confirmation skip control "Next" 1012 in interface 101, the phone can skip the risk protection settings and turn on risk protection by default in response to the triggering operation of "Next" 1012.

[0100] Method 2: The activation event is the operation event of turning on the risk protection switch in the settings application.

[0101] The phone's settings app provides a risk protection setting 1. In one specific implementation, the phone further provides setting 1 within the security and / or privacy settings of the settings app.

[0102] See Figure 2The phone can display interface 201, which is the settings application interface. The "Security" option 2011 in interface 201 is a security setting. In response to the user's triggering of "Security" 2011, the phone can display interface 202. Interface 202 is the security settings interface. The "Risk Protection" option 2021 in interface 202 is risk protection setting 1. That is, taking setting 1 as an example within the security settings... In the settings interface of the security settings item shown in interface 202, "Risk Protection" 2021 is located in the area near the top of the interface and is larger than other settings such as "SOS Emergency Help" 2022 and "Emergency Warning Notification" 2023, so that setting item 1 can be placed in a more conspicuous position.

[0103] The phone detects an operation event where the risk protection switch is turned on in setting 1. If a trigger operation is detected on the risk protection switch which is in a closed state, risk protection can be turned on, thereby enabling the face-swapping detection function under risk protection.

[0104] See also Figure 2 In response to the user's triggering of "Risk Protection 2021", the phone can display interface 203, which is the risk protection settings interface. The "Risk Detection" switch 2031 in interface 203 is the risk protection switch. When the "Risk Detection" switch 2031 in interface 203 is in the off state, it indicates that risk protection is not enabled.

[0105] In response to the user's triggering of the "Risk Detection" switch 2031 in interface 203, the phone can enable risk protection. For example, after enabling risk protection, the phone can display interface 204, which is also the risk protection settings interface. Interface 204 also includes the "Risk Detection" switch 2031, and the "Risk Detection" switch 2031 in interface 204 is in the "on" state, indicating that risk protection has been enabled.

[0106] In one specific implementation, risk protection is provided by the system manager service in the mobile phone.

[0107] If the phone does not have the system manager service enabled, and detects an operation event of enabling the risk protection switch in setting item 1, the phone can first prompt the user to enable the system manager service (denoted as prompt b). In response to the user's operation of enabling the system manager service in prompt b, such as the triggering operation of the confirmation control in prompt b, the phone can enable the system manager service and further enable risk protection.

[0108] See also Figure 2When the system manager service is not enabled, in response to the user's triggering of the "Risk Detection" switch 2031 in interface 203, the phone can display interface 205. Interface 205 adds a display prompt 2051 to the risk protection settings interface. Prompt 2051 is prompt a. For example, prompt 2051 includes the text "When enabling risk detection, the 'System Manager Service' must be enabled simultaneously," prompting the user to enable the system manager service. The "Agree" button 2052 in prompt 2051 is a control to confirm enabling the system manager service. In response to the user's triggering of the "Agree" button 2052, the phone can enable the system manager service and risk protection, as shown in interface 204.

[0109] In response to the user's action of canceling the system manager service in prompt b, such as the triggering of the confirmation cancellation control in prompt b (such as "Cancel" 2053 in interface 205), the phone can cancel the system manager service, and correspondingly, risk protection cannot be enabled.

[0110] If the phone has the system manager service enabled, and detects the operation event of turning on the risk protection switch in setting item 1, then risk protection can be enabled, as shown in the response process from interface 203 to interface 204 above.

[0111] Normally, the system manager service is enabled by default in OOBE. Alternatively, if the system manager service is not enabled, in response to starting the system manager service, the phone can display a prompt b to enable the system manager service. In response to the user's operation of prompt b, the phone can enable the system manager service.

[0112] See Figure 3 When the system manager service is not enabled, in response to the application launching the system manager, the phone can display interface 301, with prompt 3011 (prompt b) in interface 301. Prompt 3011 mentions various capabilities provided by the system manager service, such as "the system manager service provides notification management, battery management, privacy and security, application launch management, permission management, cleanup and acceleration, data usage management, harassment blocking, virus scanning, and risk protection functions." Furthermore, prompt 3011 includes an "Agree" control 3012 to enable the system manager service. In response to the user's "Agree" 3012, the phone can enable the system manager service.

[0113] In addition, the risk protection settings interface can also include settings for various risk detection functions, including the on / off status of the corresponding risk detection functions.

[0114] When risk protection is disabled, the default on / off state of various risk detection functions is off. For example, interface 203 includes: the setting "Risk Call Interception" 2032 for risk call detection, the setting "Risk Information Interception" 2033 for risk information detection, and the setting "AI Face Swap Reminder" 2034 for face swap detection, all of which are off.

[0115] With risk protection enabled, the on / off status of various risk detection functions is always "on" by default. For example, in interface 204, the on / off status of "Risk Call Interception" (2032), "Risk Information Interception" (2033), and "AI Face Swap Reminder" (2034) is all "on".

[0116] Furthermore, in response to a user's triggering of any risk detection function setting in the risk protection settings interface, the mobile phone can display the settings interface for that risk detection function.

[0117] For example, after enabling risk protection, in response to the user's request... Figure 2 The "AI Face Swap Reminder" 2034 trigger operation in interface 204 shown can be displayed on the phone. Figure 4 The interface shown is 401, which is the settings interface for face-swapping detection. The "AI Face-Swapping Detection" switch 4011 in interface 401 is the face-swapping detection switch. Since risk protection is enabled at this time, the switch for "AI Face-Swapping Detection" 4011 is in the "on" state.

[0118] The face-swapping detection settings interface can also include a description of the face-swapping detection function, such as the text description in interface 401: "During video calls, the system can detect in real time whether the other party is an AI face-swapping device to prevent the risk of face-swapping," so that users can clearly understand the face-swapping detection function.

[0119] The settings interface for face-swapping detection can also include a details entry, such as "Learn More" (4012) in interface 401. In response to the user's triggering of this details entry, the phone can display more detailed information about face-swapping detection, such as scenarios where face-swapping occurs and formulas to warn of face-swapping risks, allowing users to understand face-swapping detection in more detail. See also... Figure 4 In response to the user's triggering operation of "Learn More" 4012 in interface 401, the mobile phone can display interface 402, which includes scenarios where face swapping occurs 4021, formulas to warn of face swapping risks 4022, etc.

[0120] Additionally, the risk protection settings interface can also include protection records for various risk detection functions within a recent preset time period (such as 60 days or 1 month), including the number of risk occurrences. See also... Figure 5The phone can display interface 501 (the same as interface 204 mentioned above). In area 5011 of interface 501, there is a record of the protection against face swapping, such as the number of risk counts being 0.

[0121] In response to a user's trigger action on the protection record of any risk detection function, the mobile phone can display the detailed detection record of that risk detection function. See also Figure 5 In response to the user's trigger operation on area 5011 in interface 501, the mobile phone can display interface 502, which is the face-swapping detection record interface, containing detailed detection records of face-swapping. However, since the number of risk counts is 0 at this time, there are no detailed detection records in interface 502.

[0122] At this point, it should be noted that the above... Figures 2-5 The names and layouts of the settings items shown in the application are merely illustrative. In actual implementation, they are not limited to these examples. For instance, the settings interface for security settings can also be as shown... Figure 6 As shown in interface 601, “Risk Protection” 6011 in interface 601 is the first setting item for risk protection. The layout of the settings items in interface 601 is quite different from that in interface 202 above. For example, the display position and size of “Risk Protection” 6011 in interface 601 are different from those of “Risk Protection” 2021 in interface 202.

[0123] Method 3: The activation event is the operation event of turning on the face-swapping detection switch in the settings application.

[0124] The phone offers a face-swapping detection setting (option 2). If the phone detects an operation that turns on the face-swapping detection switch in setting 2, such as a triggering operation on a switch that is currently off, the phone can enable face-swapping detection.

[0125] See Figure 7 Without risk protection enabled, the phone displays interface 701 (same as interface 203 above), where "AI Face Swap Reminder" 7011 is setting item 2. In response to the user's triggering of "AI Face Swap Reminder" 7011, the phone displays the face swap detection settings interface shown in interface 702, where the "AI Face Swap Detection" switch 7021 is the face swap detection switch. Since risk protection is not enabled at this time, the "AI Face Swap Detection" switch 7021 in interface 702 is in the off state. In response to the user's triggering of the "AI Face Swap Detection" switch 7021, the phone can enable face swap detection, such as displaying interface 703 (same as interface 401 above), where the "AI Face Swap Detection" switch 7021 is in the on state, indicating that face swap detection is enabled.

[0126] As explained above, the risk protection feature includes face-swapping detection. Therefore, if risk protection is not enabled on the phone, upon detecting an action triggered by setting item 2, the phone can first prompt the user to enable risk protection (denoted as prompt c). In response to prompt c, the phone can then enable both risk protection and face-swapping detection.

[0127] See also Figure 7 In response to the user's triggering operation on the "AI Face Swap Reminder" 7011 in interface 701, the phone can first display interface 704. The prompt 7041 in interface 704 is prompt c, which includes the text "When AI face swap detection is enabled, the 'Risk Detection' switch will be enabled simultaneously," used to prompt the user to enable risk protection. Prompt 7041 also includes an "Enable" control 7042 for enabling risk protection. Enabling risk protection can be a triggering operation on "Enable" 7042. In response to the user's triggering operation on "Enable" 7042, the phone can enable face swap detection, as shown in interface 703.

[0128] If risk protection is enabled on the phone, after detecting an operation event where the face-swapping detection switch is turned on in setting 2, the phone can enable face-swapping detection without displaying prompt c. It should be noted that face-swapping detection is enabled by default when risk protection is enabled; however, the phone can also independently disable face-swapping detection in response to a user's action of turning it off. Therefore, it is possible for risk protection to be enabled but face-swapping detection to be disabled. For details on how to independently disable face-swapping detection, please refer to the explanation of method 3 below; it will not be elaborated upon here.

[0129] Similar to Method 2 mentioned above, risk protection is provided by the system manager service in the phone. Based on this, if the system manager service is not enabled on the phone, and an operation event of enabling the face-swapping detection switch in setting 2 is detected, the phone can first prompt the user to enable the system manager service (denoted as prompt d). In response to the user's operation of enabling the system manager service in prompt d, the phone can enable the system manager service and face-swapping detection.

[0130] See also Figure 7Without the system manager service enabled, in response to the user's triggering action on the "AI Face Swap Reminder" 7011 in interface 701, the phone can display interface 705, where prompt 7051 is prompt d. Prompt 7051 includes the text "When enabling face swap detection, the 'System Manager Service' must be enabled simultaneously," prompting the user to enable the system manager service. Prompt 7051 includes an "Agree" control 7052 for enabling the system manager service; enabling the system manager service can be triggered by clicking "Agree" 7052. In response to the user's triggering action on "Agree" 7052, the phone can enable the system manager service and face swap detection, as shown in interface 703.

[0131] If the phone's system manager service is enabled, and it detects the operation event of turning on the face-swapping detection switch in setting item 2, it can enable face-swapping detection without displaying prompt d, as shown in the response process from interface 702 to interface 703 above.

[0132] In practice, not enabling the system manager service also means that risk protection is not enabled. Therefore, when the system manager service is not enabled, the phone's response process can be interface 701, interface 705, interface 704, and interface 703, thus ensuring that face-swapping detection is enabled only when both the system manager service and risk protection are enabled. Conversely, even when the system manager service is enabled, risk protection may not be enabled. In this case, the phone's response process can also be interface 701, interface 704, and interface 703.

[0133] Method 4: The activation event is the operation event for performing the activation detection when prompted to enable risk protection.

[0134] Without risk protection enabled, the phone can send a push notification (a) to prompt the user to enable face-swapping detection. See also... Figure 8 Without risk protection enabled, the phone can display interface 801, with prompt 8011 being prompt a. Prompt 8011 includes the text "Added 'AI Face Swap Detection' to detect AI face swapping during video calls," indicating the addition of the face swap detection function. Prompt 8011 also includes "Enable Risk Detection" 8012, prompting the user to enable risk protection. In other words, prompt 8011 can prompt the user to enable risk protection.

[0135] In response to the user's action of enabling risk detection on prompt 'a', the mobile phone can display the risk protection settings interface. In one specific implementation, prompt 'a' includes a control for enabling risk protection, such as "Enable Risk Detection" 8012 in interface 801, and the action of enabling detection can be a trigger operation on the control for enabling risk protection.

[0136] See also Figure 8The control for enabling risk protection is "Enable Risk Protection" 8012 in interface 801. In response to the user's trigger operation of "Enable Risk Protection" 8012, the mobile phone can display interface 802 (the same as interface 203 mentioned above). Interface 802 is the risk protection settings interface.

[0137] After entering the risk protection settings interface, the phone can respond to the operation event of turning on the risk protection switch, thereby enabling risk protection and face-swapping detection. For details, please refer to the explanation in Method 2 above, which will not be repeated here.

[0138] Of course, if risk protection is enabled but face-swapping detection is disabled, in response to the user's action of enabling detection in prompt 'a', the risk protection settings interface displayed on the phone will show the risk protection switch as 'on' instead of 'off' as shown in interface 801. However, the face-swapping settings will also be 'off'. In this case, the phone can enable face-swapping detection in response to the action of enabling the face-swapping switch, as explained in Method 3 above, and will not be repeated here.

[0139] After displaying prompt 'a', the phone can cancel displaying prompt 'a' in response to an operation event that enables detection, or in response to a user's triggering operation on an area outside of prompt 'a', or in response to the display duration of prompt 'a' reaching a duration of 1 (e.g., 10 seconds).

[0140] See also Figure 8 In response to the user's trigger operation of "Enable Risk Protection" 8012, the interface 802 displayed on the mobile phone no longer includes prompt 8011.

[0141] Furthermore, prompt 'a' can be in the form of a card as shown in prompt 8011 above. After the display duration of prompt 'a' reaches duration 2, such as after 5 seconds, the phone can first shrink prompt 'a' into capsule 1 to avoid interfering with the user's use of the phone. After the display duration of capsule 1 reaches duration 3, the phone can cancel the display of capsule 1, thereby canceling the display of prompt 'a'.

[0142] Taking duration 2 as 5 seconds and duration 3 as 10 seconds as examples, after the display duration of prompt 8011 in interface 801 reaches 5 seconds, the phone can shrink prompt 8011 to... Figure 9 The capsule 9011 in interface 901 is Capsule 1. After Capsule 9011 has been displayed for 10 seconds, the phone can cancel the display of Capsule 9011, as shown in interface 902.

[0143] After shrinking prompt 'a' to capsule 1, in response to the user's triggering action on capsule 1, the phone can restore the display of prompt 'a'. For example, in response to the user's triggering action on capsule 9011 in interface 901, the phone can restore the display of 'a'. Figure 8 The interface shown is 801.

[0144] Therefore, it should be noted that the above description of method four mainly uses the example of displaying prompt 'a' on the phone's home screen. In practice, the phone can also display prompt 'a' on other screens. For example, when the phone detects that the push conditions for prompt 'a' are met, it will display prompt 'a' on the phone's current screen.

[0145] This application embodiment exemplarily illustrates the push conditions for prompt a: The first condition for the push notification is that the system detects access to the aforementioned setting item 1 without enabling risk protection.

[0146] When the phone detects access to Settings 1, such as the aforementioned interface 202, it can push notification a. This allows notification a to be pushed even when risk protection is not enabled, but the Risk Protection Settings 1 is viewed.

[0147] The second condition for pushing the notification is that a risk event has been detected.

[0148] Risk events include receiving risky phone calls, receiving risky information, receiving overseas phone calls, downloading unknown applications and performing actions such as entering bank account numbers, transferring money, and sharing screens.

[0149] When the phone detects a risk event, it sends a push notification (a). This allows the user to be promptly prompted to enable face-swapping detection when the phone faces security risks.

[0150] The third condition for the push notification is that face-swapping detection is not enabled, and the interval between the last time the notification is displayed is 4 days, such as 7 days, 20 days, or 60 days.

[0151] If the phone does not detect any operation event to enable face-swapping detection after displaying prompt 'a', it will not enable face-swapping detection. Subsequently, if face-swapping detection is not enabled and a considerable amount of time has passed since the last display of prompt 'a', the phone can display prompt 'a' again to remind the user.

[0152] In practice, the above-mentioned various push conditions can be combined.

[0153] For example, the phone can first display prompt 'a' after either push condition one or push condition two arrives, and then display prompt 'a' again after push condition three arrives.

[0154] For example, the phone can first display prompt a after either push condition one or push condition two arrives, and then display prompt a again when push condition three arrives and push condition one or push condition two is met.

[0155] Additionally, if risk protection is enabled in OOBE, the phone can push a notification with the option "d," which prompts the user to understand face-swapping detection. As explained earlier regarding Method 1, risk protection is enabled by default in OOBE. If risk protection is enabled in OOBE, the user is highly likely unaware of the risk protection and various risk detection functions. In this case, after OOBE completes, the phone can prompt the user with "d" to understand face-swapping detection.

[0156] See Figure 10 When risk protection is enabled in OOBE, the phone can display interface 1001, with prompt 10011 as prompt d. Prompt 10011 includes the text "Added 'AI Face Swap Detection' to detect AI face swapping during video calls," indicating the addition of the face swap detection function. Furthermore, prompt 10011 also includes the text "Learn more" 10012, prompting the user to learn more about face swap detection. In other words, prompt 10011 can suggest learning about face swap detection.

[0157] In response to a user's action to "learn more" in prompt d, such as triggering a control within prompt d, the phone can display the face-swapping detection settings interface for the user to understand face-swapping detection. See also... Figure 10 Taking the "Go to Learn" control in interface 10012 as an example, in response to the user's trigger operation on "Go to Learn" 10012, the phone can display interface 1002 (the same as interface 401 mentioned above). Interface 1002 is the settings interface for face swap detection.

[0158] The phone can display an introduction to the face-swapping detection function in the settings interface. Furthermore, the face-swapping detection settings interface includes a details entry point to trigger the phone to display detailed information about face-swapping detection. This allows users to understand face-swapping detection. For details, please refer to the explanation in Method Two above; it will not be repeated here.

[0159] The disappearance, shrinking, and push conditions of prompt d can be referred to in the previous explanation of prompt a, which will not be repeated here.

[0160] After enabling face-swapping detection, the phone can perform face-swapping detection and provide a prompt during video calls.

[0161] Furthermore, the phone can perform face-swapping detection either actively or passively. Active detection means that detection can begin automatically without user intervention. Passive detection means that detection can begin passively based on user-triggered actions. These will be explained below: Option 1: Active detection.

[0162] In response to the start of a video call, the phone can proactively begin face-swapping detection.

[0163] If a face-swapping risk is detected, the phone displays a warning (e) to indicate the risk, thus providing risk feedback. If no face-swapping risk is detected, no warning is displayed. This way, the phone only alerts when there is a risk of face-swapping, minimizing interference with video calls.

[0164] See Figure 11 Regarding proactive detection: In risk-free scenarios (i.e., no risk of face swapping), the user is completely unaware of the process before, during, and after detection. In risky scenarios (i.e., no risk of face swapping), the user is unaware of the process before and during detection; only after detection will the phone display a notification (e) to provide risk feedback, and the user will then be aware of this process.

[0165] If a risk of face-swapping is detected, the phone can display a warning message on the video call screen. (See also...) Figure 12 During a video call, the phone can display interface 1201. If the phone detects a risk of face swapping, it can display interface 1202. The prompt 12021 in interface 1202 is prompt e, which includes the text "The other party is suspected of using AI face swapping to impersonate someone," to warn of the risk of face swapping.

[0166] Among them, prompt e can include information indicating that there is a security risk, such as the prompt text in prompt 12021, "The other party is suspected of using AI face-swapping to impersonate someone."

[0167] Furthermore, prompt e can also include information indicating the level of risk, such as the text "AI face-swapping probability 95%" in prompt 12021, used to indicate the level of risk.

[0168] Furthermore, prompt e can also include information instructing verification of the other party's identity, such as the text "Please be sure to verify the authenticity of their identity through other means" in prompt 12021, which is used to prompt for identity verification.

[0169] Furthermore, prompt e can also include an entry point for understanding face-swapping detection, such as the entry point "Learn more about the risks of AI face-swapping" 12022 in prompt 12021, which is used to trigger the display of the face-swapping detection settings interface. In response to the user's triggering operation of the entry point for understanding face-swapping detection, the phone can display the face-swapping detection settings interface, such as interface 401 mentioned above.

[0170] Furthermore, prompt 'e' can also include a re-detection control (also known as a "re-detection control"), such as "re-detection" 12023 in prompt 12021, which triggers the phone to re-perform face-swapping detection. In response to the user's triggering of the re-detection control, the phone can re-perform face-swapping detection. It should be noted that when the phone triggers face-swapping detection due to the user's triggering of the re-detection control, this is a case of the phone passively performing face-swapping detection. This will be explained in detail in the section on passive face-swapping detection below, and will not be elaborated upon here.

[0171] Furthermore, prompt 'e' can also include screen recording controls (which can be called "screen recording controls"), such as "Record and save video" 12024 in prompt 12021, which is used to trigger the phone to record the screen of a video call. In response to the user's triggering operation of the screen recording control, the phone can record the screen of the video call until the video call ends, at which point recording stops and the screen recording is saved to the phone.

[0172] In practice, the phone may have already ended the video call after detecting a risk of face-swapping. To address this, the phone can display a warning (e.g., "f") on the screen after the video call ends, such as in the chat list, to indicate the risk of face-swapping.

[0173] See also Figure 12 During a video call, the phone can display interface 1201. After detecting a risk of face swapping and having exited the video call, the phone can display interface 1203. The prompt 12031 in interface 1203 is prompt f, which includes the text "The other party is suspected of using AI face swapping to impersonate someone," used to warn of the risk of face swapping.

[0174] Similar to prompt e, prompt f may also include information indicating the existence of a security risk, information indicating the degree of risk, etc. The instructions provide information on verifying the other party's identity and accessing the face-swapping detection system. For details, please refer to the previous explanation regarding prompt e.

[0175] It should be noted that the "f" message is displayed only after the video call is exited. At this point, it is no longer possible to re-detect and record the video call screen. Consequently, the "f" message usually does not include controls for re-detection or screen recording.

[0176] Furthermore, prompt f may include a control to cancel the display, such as "OK" 12032 in prompt 12031, which is used to trigger the phone to cancel the display of prompt f. In response to the user's triggering operation of the control to cancel the display, the phone can cancel the display of control 6.

[0177] In practice, the phone can capture screenshots of video calls for face-swapping detection. In some cases, the video call may end before the phone captures the necessary screenshots for detection. In such cases, the phone cannot obtain a detection result. However, considering that the phone is actively performing the detection, it will not indicate that the detection is incomplete, thus avoiding interference with the user.

[0178] In practice, the phone may not detect a face and therefore cannot obtain a detection result. Similarly, considering that the phone actively detects the face, it will not display a notification that no face was detected, thus avoiding interference with the user.

[0179] Furthermore, in response to the start of a video call, the phone can proactively begin detecting face-swapping risks if the detection conditions are met. This allows for targeted detection of face-swapping risks, rather than performing face-swapping detection for every video call, thereby reducing the phone's power consumption.

[0180] The following are some typical testing conditions: Detection condition 1 The other party in the video call is a friend added within the last 5 hours. Specifically, for the same newly added friend, if a video call is made with that friend within 5 hours of adding them, the phone will perform face-swapping detection; if a video call is made with that friend after 5 hours of adding them, the phone will not perform face-swapping detection.

[0181] See Figure 13 The phone can display interface 1301, which is the management interface for new friend requests in the chat application. Interface 1301 includes David's friend request 13011. In response to the user's "accept" action 13012 on the friend request 13011, the phone can add David as a friend. For example, after adding him and returning to the chat list interface, the phone can display interface 1302. Interface 1302 includes the conversation options 13021 for the newly added friend David.

[0182] See also Figure 13 Within 5 hours of adding David as a friend, the phone receives a video call invitation from David and displays interface 1303. Interface 1303 includes a call answering control 13031. In response to the user's triggering of the call answering control 13031, the phone can start a video call, as shown above. Figure 12The screen is at position 1201. At this point, the phone can begin face-swapping detection.

[0183] Conversely, if you do not call David within 5 hours of adding him as a friend, but later receive a video call invitation from David, screen 1303 can still be displayed, and the phone can avoid performing face-swapping detection after starting the video call.

[0184] Typically, face-swapping risks occur during video calls with recently added friends. Therefore, the phone can perform face-swapping detection on video calls with recently added friends, thus increasing the likelihood of detecting video calls with face-swapping risks.

[0185] Detection condition 2 The first video call is with the other party in this video call. In other words, for the same friend, if it is the first video call with that friend, the phone will perform face-swapping detection; if it is not the first video call with that friend, the phone will not perform face-swapping detection.

[0186] Typically, the risk of face-swapping occurs during the first video call with a friend. If there is no risk of face-swapping during the first call, it is highly unlikely that subsequent video calls with that friend will also be free of face-swapping risk. Therefore, a mobile phone can detect the risk of face-swapping during the first video call with a friend, thus enabling the detection of face-swapping risks in subsequent video calls with that friend.

[0187] Detection condition 3 The other party in the video call is in list 1, which records the number of newly added friends (1).

[0188] In practice, for newly added friends, the phone can store relevant information, such as name and location, in a specific area for face-swapping detection. Meanwhile, some users, due to the nature of their profession, may add a large number of friends in a short period, such as dozens in a single day. Consequently, the storage space required to store this information on the phone is substantial.

[0189] To address this, the phone can store information for a maximum of one friend. After storing information for one friend, if a new friend is added, the phone can delete the information stored in that specific area using a first-in, first-out (FIFO) approach before storing the new friend's information. Accordingly, "list 1" refers to the list of friends whose information is stored in that specific area. The phone can perform face-swapping detection on the stored friends. In this way, the phone consistently performs face-swapping detection on the most recently added friend (number of friends) while avoiding excessive storage space consumption.

[0190] Detection condition 4 The detection criteria for face-swapping risks include the other party initiating the video call. Typically, users who gain the phone owner's trust and then guide them to perform actions that could lead to financial loss or information leakage will initiate the video call. Therefore, mobile phones can perform face-swapping detection on video calls initiated by the other party, increasing the likelihood of detecting video calls with face-swapping risks.

[0191] In practice, at least two of the above detection conditions can be combined. For example, detection conditions 1-4 can be combined so that the mobile phone can perform face-swapping detection in video calls initiated by newly added friends in list 1 within a duration of 5.

[0192] Of course, in actual implementation, the detection conditions for proactive detection are not limited to those listed above. For example, the detection condition can also be that the mobile phone has a risk event. Please refer to the explanation of push condition two above for details.

[0193] Option 2: Passive detection.

[0194] During a video call, the phone can passively initiate face-swapping detection based on the user's actions within the video call interface. This allows the phone to assess the potential for face-swapping based on the user's specific needs.

[0195] The possible scenarios for the detection operation include at least one of the following: In scenario one, the detection operation is a user-triggered operation on the re-detection control in the aforementioned prompt e.

[0196] The prompt e is the aforementioned Figure 12 In the interface 1202 shown, the prompt 12021 and the re-detection control are the "re-detection" 12023 in prompt 12021. The detection operation can be the user's trigger operation on "re-detection" 12023.

[0197] Scenario 2: With face-swapping detection enabled, the phone can display a "g" prompt on the video call screen to indicate the use of the face-swapping detection function. Correspondingly, the detection action can be the user's response to the "g" prompt.

[0198] In one specific implementation, the prompt g includes a detection control, and the detection operation can be a user-triggered operation on the detection control.

[0199] See Figure 14The phone can display interface 1401, where the "AI Face Swap Reminder" setting 14011 for face swap detection is set to "On," indicating that face swap detection is enabled. In this case, during a video call, the phone can display interface 1402, which is the video call interface. The prompt 14021 on interface 1402 is prompt g. Prompt 14021 includes the text "AI Face Swap Detection can detect whether the other party's face in the video is impersonating someone else" and "Detect Now" 14022, thus prompting the user to use face swap detection. "Detect Now" 14022 is a detection control; the detection operation can be triggered by the user.

[0200] Furthermore, in addition to enabling face-swapping detection, the push conditions for notification g may also include one or more of the aforementioned push condition two, detection condition 1, detection condition 2, detection condition 3, and detection condition 4. This application does not impose specific limitations on this.

[0201] The phone can present different notifications in different formats on different occasions.

[0202] Specifically, when the above prompt g is displayed from the 1st to the Nth time (N>1, where N is an integer), the mobile phone can first present it in the form of a card, as shown in prompt 14021 in interface 1402 above. When the above prompt g is displayed for the N+1th time and thereafter, the mobile phone can first present it in the form of a capsule, as shown in capsule 14031 in interface 1403 above.

[0203] In this way, the phone can present the details of the prompt to the user during the first few times the prompt "g" is displayed, so that the user can clearly understand the content of the prompt. When the prompt "g" is displayed again, a capsule is presented to the user to reduce the impact on the call.

[0204] Furthermore, the card format and capsule format can be switched between each other.

[0205] Specifically, after the display duration of the card-style notification "g" reaches a duration of 7 seconds, such as 5 seconds, the phone can shrink the notification "g" into a capsule-like form. See also... Figure 14 After the phone displays the screen 1402 for 5 seconds, the prompt 14021 in the screen 1402 can be collapsed into the capsule 14031 in the screen 1403. That is, the capsule 14031 is the collapsed prompt g.

[0206] Specifically, in response to a user's tap on the capsule-shaped prompt g, the phone can expand the prompt g into a card format. See also... Figure 14 After displaying interface 1403, in response to the user's click on capsule 14031, the phone can expand capsule 14031 into prompt 14021 on interface 1402.

[0207] Scenario 3: When face-swapping detection is not enabled, the phone can display a prompt "h" on the video call interface to remind the user to enable and use the face-swapping detection function. Correspondingly, the detection action can be triggered by the user's response to the prompt "h".

[0208] In one specific implementation, the prompt 'h' includes an enable detection control, and the detection operation can be a user-triggered operation to enable the detection control.

[0209] See Figure 15A The phone can display interface 1501, where the "AI Face Swap Reminder" setting 15011 for face swap detection is set to "Not Enabled," indicating that face swap detection is not enabled. In this case, during a video call, the phone can display interface 1502, which is the video call interface. Interface 1502 contains prompt 15021, which includes the text "Enabling 'AI Face Swap Detection' can detect whether the other party's face in the video is impersonating someone else" and "Enable and Detect" 15022, thus prompting the user to enable and use face swap detection. The "Enable and Detect" 15022 is an enable / detection control, meaning the detection operation can be triggered by the user's interaction with "Enable and Detect" 15022.

[0210] Therefore, it's important to note that after enabling risk protection on the phone, multiple risk detection functions under risk protection will be enabled by default. Afterward, the user can individually disable one or more of these risk detection functions, such as disabling face-swapping detection. In this case, the situation shown in interface 1501 above will occur, meaning risk protection is enabled, but face-swapping detection is not. See the explanation of method 3 below for details; it will not be elaborated upon here. In this way, the phone can push a notification (h) even when the user disables face-swapping detection, specifically prompting the user to enable and use face-swapping detection.

[0211] It should be noted that if face-swapping detection is not enabled due to the lack of risk protection, i.e., both risk protection and face-swapping detection are not enabled, the phone will usually push the aforementioned prompt a, but will not push prompt h. For details, please refer to the relevant explanation of method four above, which will not be repeated here.

[0212] Furthermore, in addition to not enabling face-swapping detection, the conditions for pushing the notification h may also include one or more of the aforementioned push condition two, detection condition 1, detection condition 2, detection condition 3, and detection condition 4. This application does not impose specific limitations on this.

[0213] Generally, the conditions for pushing notifications for "h" are stricter than those for "g". This allows the phone to push notifications for "h" less frequently when face-swapping detection is disabled, reducing the impact on users; and to push notifications for "g" more frequently when face-swapping detection is enabled, increasing the usage rate of face-swapping detection.

[0214] For example, a typical set of push conditions for prompt g and prompt h are shown below: The conditions for push notification g include the existence of security risks (i.e., push condition two) and the other party in the video call being a friend added within the last 5 minutes (i.e. detection condition one).

[0215] The conditions for push notification h include the existence of security risks (i.e., push condition two), the other party in the video call being the first friend added, and the other party initiating the video call within 6 hours (less than 5 hours) after adding the friend.

[0216] Additionally, similar to the aforementioned notification g, the phone can present notifications h in different formats (such as card and capsule formats) at different times. Furthermore, the card and capsule formats can be switched between each other. See also... Figure 15B After the phone displays the screen 1502 for 5 seconds, the prompt 15021 (i.e., the card-shaped prompt h) in the screen 1502 can be collapsed into the capsule 15031 in the screen 1503. That is, the capsule 15031 is the collapsed prompt h.

[0217] In response to a user's tap on the capsule-shaped prompt "h", the phone can expand "h" into a card format. See also... Figure 15B After the phone displays interface 1503, in response to the user's click on capsule 15031, the phone can expand capsule 15031 into prompt 15021 on interface 1502.

[0218] Furthermore, the detection conditions that trigger the aforementioned active detection are usually more lenient than the push conditions for prompts h and g. This allows for a higher frequency of triggering active detection with a lower level of awareness (visible only when a face-swapping risk is detected), and a lower frequency of triggering passive detection with a higher level of awareness (visual display throughout).

[0219] In passive detection solutions, the mobile phone can provide a fully visual display in response to the detection operation, allowing users to clearly understand the detection process.

[0220] See Figure 16In the passive detection solution: before detection, the phone can prompt the user to start the detection. During detection, the phone can display the detection process. After detection, the phone can provide feedback on the detection results. The visual presentation before and during detection allows the user to clearly perceive that the phone is detecting, giving them a sense of certainty. The visual presentation after detection allows the user to understand the risks of face-swapping, giving them a sense of control.

[0221] The response process before, during, and after detection is described below: First, before detection. "Before detection" can be understood as the stage before the phone obtains the data used for face-swapping detection. For example, if the phone needs to obtain at least 10 frames of screenshots from a video call for face-swapping detection, then the period before obtaining these 10 frames falls under the "before detection" stage.

[0222] Before the test, the phone can display a prompt "i" to indicate when the test should begin.

[0223] See Figure 17 In response to the detection operation, the mobile phone can display interface 1701, which is a video call interface. Capsule 17011 in interface 1701 is a prompt i, which includes the prompt text "Start Detection" to prompt the start of the detection.

[0224] Second, during the detection process. This can be understood as the stage between when the phone obtains the data used for face-swapping detection and when the detection result is obtained.

[0225] During the test, the phone can display a prompt "j" and a test animation. The prompt "j" indicates that the test is in progress.

[0226] See also Figure 17 After displaying interface 1701, the phone can then display interface 1702, which remains the video call interface. The capsule 17021 in interface 1702 serves as a prompt, containing the text "Detecting," indicating that detection is in progress. Interface 1702 also includes a scanning animation from top to bottom (as shown by the arrow in interface 1702).

[0227] It should be noted that the form of the detection animation can be diverse and is not limited to that shown in interface 1702. Furthermore, a single detection process can include multiple effects. For example, in addition to the top-to-bottom scanning detection animation in interface 1702, the animation shown in interface 1703, which scans the other party's face region (represented in a dot matrix form on face 17031 in the figure), can also be displayed.

[0228] Third, post-detection. Post-detection can be understood as the stage after the phone's detection results are obtained. These results include whether there is a risk of face-swapping or not.

[0229] After the test, the phone can display a notification "j", which indicates that the test is complete.

[0230] See also Figure 17 After the phone displays interface 1702 and receives the test results, it can then display interface 1704, which remains the video call interface. Capsule 17041 in interface 1704 is a prompt, containing the text "Detection Complete," indicating that the test is complete.

[0231] It should be noted that in the passive detection scheme, the phone can display a prompt regardless of whether a face-swapping risk is detected or not, allowing the user to obtain all detection results.

[0232] The above description of the visualization before, during, and after testing primarily uses capsule-shaped prompts as an example, such as prompts h, i, and j, all of which are capsules. However, this is not a limitation in actual implementation.

[0233] For example, hints h-j can also be in card form. See also Figure 18 The prompt 'h' can also be prompt 18011 in interface 1801, and the prompt 'i' can also be prompt 18021 in interface 1802.

[0234] Among them, the card-style prompt j can be divided into two cases: there is a risk of face swapping and there is no risk of face swapping.

[0235] See also Figure 18 In cases where there is a risk of face swapping, the prompt j can be prompt 18031 in interface 1803, used to indicate the risk of face swapping. The specific content of prompt j in cases of face swapping risk is basically the same as that of prompt e mentioned earlier, and can be found in the previous explanation of prompt e, which will not be repeated here.

[0236] In practice, to avoid malicious detection, if the number of times the phone repeatedly performs face-swapping detection for the video call is less than the maximum (e.g., 3 times, 5 times, etc.), the phone can provide a re-detection control in a card-style prompt j, such as "Re-detect" 18032 in prompt 18031 of interface 1803. If the number of times the phone repeatedly performs face-swapping detection for the video call reaches the maximum, the phone may not provide a re-detection control in the card-style prompt j, such as prompt 18041 in interface 1804, which does not include a re-detection control.

[0237] See also Figure 18 In the absence of face-swapping risk, prompt j can also be prompt 18051 in interface 1805, which includes the text "No AI face-swapping risk detected" to indicate that there is no face-swapping risk. Furthermore, in the absence of security risk, the card-type prompt j can also include controls for re-detection, screen recording, etc., and this application does not make specific limitations on this.

[0238] In one specific implementation, the phone can default to presenting prompts h-j in capsule format. Subsequently, in response to the user's click on the capsule-style prompt, the phone can switch to presenting it in card format.

[0239] In another specific implementation, the mobile phone can present prompts h and i in capsule form by default, and prompt j in card form by default, so as to facilitate viewing the specific detection results.

[0240] If the detection result indicates no risk of face swapping, the phone can automatically shrink the card-style prompt "j" into a capsule-like form after the display duration reaches 8 seconds (e.g., 5 seconds). Figure 19 The card prompt 19011 in interface 1901 shrinks into capsule 19021 in interface 1902. Subsequently, after the capsule-shaped prompt j has been displayed for a duration of 9 seconds (e.g., 5 seconds), the phone can automatically cancel the display of prompt j, such as automatically canceling the display of capsule 19021 in interface 1902 and then displaying interface 1903. In this way, without the risk of face swapping, the phone can automatically shrink and then cancel the display of prompt j, avoiding any impact on call quality.

[0241] Of course, when displaying a card-style prompt j, in response to the user's swipe up on prompt j or tap on an area outside prompt j, the phone can also passively shrink prompt j into a capsule shape.

[0242] In practice, the phone may have already ended the video call by the time the test results are received. In this case, the phone can display a "k" icon on the screen after the video call ends, such as the chat list, to indicate that the test is complete.

[0243] Similar to the prompt j mentioned earlier, prompt k can also be in capsule or card form. Furthermore, the content of the card-form prompt k differs depending on whether there is a risk of face swapping or not. It can also further indicate the detection result, such as whether there is a risk of face swapping or not.

[0244] See Figure 20After the video call ends, the phone receives the test results, and then interface 2001 is displayed. The prompt 20011 in interface 2001 is a capsule-shaped prompt. Prompt 20011 includes the text "Detection Complete," indicating that the test is complete.

[0245] Given the risk of face-swapping, please continue to see Figure 20 After the video call ends, if the phone detects a risk of face swapping, it will display interface 2002. The prompt 20021 in interface 2002 is a card-style prompt k, containing the text "The other party is suspected of using AI face swapping to impersonate someone," indicating that the detection has been completed and the result is that there is a risk of face swapping.

[0246] Assuming there is no risk of face swapping, continue to see Figure 20 After the video call ends, if the phone detects no risk of face swapping, it will display interface 2003. The prompt 20031 in interface 2003 is a card-style prompt k, containing the text "No AI face swapping risk detected," indicating that the detection has been completed and the result is that there is no risk of face swapping.

[0247] Regarding the specific content of hint k, please refer to the previous explanation of hint j; it will not be repeated here.

[0248] It should be noted that the "k" prompt is only displayed after the video call ends. At this point, it is no longer possible to re-detect and record the video call screen. Consequently, the "k" prompt usually does not include controls for re-detection or screen recording.

[0249] In practice, the phone can capture screenshots of video calls for face-swapping detection. In some cases, the phone may not have captured the necessary image for detection before the video call ends. In this situation, the phone cannot obtain a detection result. Furthermore, considering a passive detection approach, user feedback is needed; the phone could indicate that the detection is incomplete. Similarly, the phone could display this notification in capsule or card format.

[0250] See Figure 21 After the video call ends, if the phone has not yet acquired the image for face-swapping detection, it can display the chat list interface shown in interface 2101, which includes prompt 21011 containing the text "Detection incomplete," indicating that the detection was not completed. Alternatively, the phone can display the chat list interface shown in interface 2102, which includes prompt 21021 containing the text "Incomplete," indicating that the detection was not completed.

[0251] In practice, the phone might not even detect the other person's face, in which case no detection result can be obtained. Furthermore, considering a passive detection approach, user feedback is necessary; the phone could indicate that the detection was incomplete. Similarly, the phone could display a capsule or card notification indicating incomplete detection, and the card notification could further specify that the incomplete detection was due to the failure to detect a face.

[0252] It should be noted that if no face is detected during a video call, a notification can be displayed on the video call interface. If no face is detected after the video call ends, a notification can be displayed on the interface after the video call ends, such as the chat list interface.

[0253] For example, see the prompt in the video call interface. Figure 22 During a video call, if the phone detects that the other party's face is not present, it can display the video call interface shown in interface 2202. Interface 2202 includes a prompt 22021, which contains the text "Incomplete," indicating that the detection was not completed. Alternatively, the phone can display the video call interface shown in interface 2201, which includes a prompt 22011. Prompt 22011 contains the text "Detection Incomplete" and "No Face Detected," indicating that the detection was incomplete because no face was detected.

[0254] Regarding Scheme 1 and Scheme 2 above, if a face-swapping risk is detected, the mobile phone can adopt at least one of the following response strategies to make users aware of the face-swapping risk.

[0255] Response strategy one: During video calls, the phone does not remove the warning about the risk of face swapping, such as warning e in the active detection and warning j in the passive detection, and the same applies below. In this way, the phone can continuously display the risk of face swapping throughout the video call.

[0256] In the proactive detection solution, the phone can continuously display the prompt "e" during a video call.

[0257] In the passive detection scheme, during a video call, the phone can respond to user actions by switching between capsule-shaped and card-shaped prompts (j), but the phone will always display prompt j. For example, in response to a user swiping up on prompt j or tapping an area outside of prompt j, the phone can passively shrink prompt j into a capsule shape, but will continue to display the capsule-shaped prompt j. This is unlike the previous method... Figure 19 As shown in the case where there is no risk of face swapping, the prompt j is proactively shrunk from card format to capsule format and finally canceled from display.

[0258] Of course, in response to the user's action of canceling the display, the phone can passively cancel the display of the warning about the risk of face swapping. This cancellation action can be a swipe up action on the warning, or a trigger action on an area other than the warning.

[0259] The second response strategy involves triggering a face-swapping risk warning during a video call. In response, the phone will access the notification center and display Notification 1, indicating that a face-swapping risk has been detected. This way, even after accessing the notification center, the phone will continue to display the face-swapping risk warning.

[0260] Taking a passive detection scheme as an example, see [link / reference]. Figure 23 Upon detecting a potential face-swapping risk, the phone can display interface 2301, with prompt 23011 being prompt j. Responding to a user's swipe-down action on interface 2301, such as a swipe in the direction indicated by the arrow, the phone can display interface 2302. Interface 2302 is a pull-down notification center, displaying various notifications received by the phone (not shown in the diagram). Notification 23021 is notification 1, containing the text "The other party is suspected of using AI face-swapping to impersonate someone," thus notifying the user of the detected face-swapping risk.

[0261] In addition, the content in notification 1 can be consistent with the content in prompt j. For example, notification 1 can also include controls for re-detection, screen recording, etc., so that users can easily trigger re-detection, screen recording, etc. in the pull-down notification center.

[0262] The third response strategy involves the phone displaying Notification 2 on the screen after a video call ends, such as in the chat list. Notification 2 can again indicate that a face-swapping risk has been detected and provide an entry point to the detection record to view the risk information. In response to the user's triggering of the detection record entry, the phone can display the face-swapping detection record interface. Thus, after detecting a face-swapping risk and ending the video call, the phone provides a quick access to the face-swapping detection record interface so that the user can view the risk information.

[0263] Taking the passive detection scheme as an example, see... Figure 24Upon detecting a potential face-swapping risk, the phone can display interface 2401, with prompt 24011 (prompt j) within it. After ending the video call, the phone can display interface 2402, which includes prompt 24021 (notification 2). Prompt 24021 includes the text "The video call participant is suspected of being impersonated by someone using AI face-swapping. Please verify their identity through other means," thus re-notifying the user of the face-swapping risk. Prompt 24021 also includes "Learn More" 24022, which is the entry point for the detection record. In response to the user's triggering of "Learn More" 24022, the phone can display interface 2403, which is the face-swapping detection record interface, including detection record 24031.

[0264] Additionally, if Notification 2 is displayed on the screen after a video call ends, the phone can display Notification 2 in the pull-down notification center in response to the notification being accessed. This way, even when the phone enters the pull-down notification center, it can still provide a warning about the risk of face-swapping and offer a quick access to the face-swapping detection log.

[0265] See also Figure 24 In response to the user's swipe down on interface 2402, the phone can display interface 2404. Interface 2404 is the pull-down notification center, which includes notification 24041, which is notification 2. Notification 24041 contains a "Learn More" option 24042, which is the entry point for the detection record. In response to the user's triggering of "Learn More" 24042, the phone can also display interface 2403.

[0266] The fourth response strategy is to provide a second warning about the risk of face-swapping within a 9-hour window (e.g., 12 hours, 24 hours) after detecting a face-swapping risk and then detecting related risky behavior. This strengthens the warning after a risky behavior occurs.

[0267] Among the associated risky behaviors are accessing money transfer interfaces and receiving website links from friends in video calls that may involve deepfakes.

[0268] Taking the passive detection scheme as an example, see... Figure 25A Upon detecting a potential face-swapping risk, the phone displays interface 2501, with prompt 25011 (prompt j) within interface 2501. If the phone then enters the transfer interface within the next 9 hours, interface 2502 will be displayed. Interface 2502 includes the text "The system detected a face-swapping risk in your recent video call; continuing the payment may result in financial loss," thus reiterating the face-swapping risk.

[0269] In addition, after detecting a face-swapping risk or when the risk level reaches a preset threshold, such as a probability of 80%, the phone can send the risk information to a protection device. For example, the phone can provide a settings entry for a protection device (e.g., in the risk protection settings interface) and send the risk information to the designated protection device.

[0270] See Figure 25B The phone can display interface 2511, which is the risk protection settings interface. Interface 2511 includes "Send a message to the guardian" 25111 and "Remote guardian reminder" 25112. "Send a message to the guardian" 25112 is a toggle option to send face-swapping risk information to the guardian device. "Remote guardian reminder" 25112 is the settings entry for the guardian device. After the "Send a message to the guardian" 25112 switch is turned on, if the phone detects face-swapping risk or the risk level of face-swapping reaches a preset level value, it can send risk information to the guardian device set in "Remote guardian reminder" 25112.

[0271] Furthermore, the monitoring device can display a risk warning after receiving risk information. See also... Figure 25B After receiving risk information, the protection device can display interface 2512, which includes risk warning 25121, indicating that it has received risk information about face swapping.

[0272] Similar to enabling the face-swapping detection function mentioned earlier, users can also disable it once they no longer need it. Specifically, the phone can disable the face-swapping detection function after detecting an event indicating that it has been disabled. When the face-swapping detection function is disabled, the phone will no longer detect the risk of face-swapping and provide any warnings during video calls.

[0273] The shutdown event can be an operation event that disables risk protection settings in OOBE, an operation event that disables the risk protection switch in the settings application, or an operation event that disables the face-swapping detection switch in the settings application, etc. This application embodiment does not specifically limit this.

[0274] The following sections will explain in detail the process of triggering the face-swapping detection function for the various events listed above.

[0275] Method 1: The shutdown event is the operation event for disabling risk protection settings in OOBE.

[0276] In OOBE (Out-of-Body Environment), mobile phones typically have risk protection enabled by default. The system detects when a user disables risk protection settings in OOBE, such as the previously mentioned... Figure 1D If the default switch 1011 in the middle interface 101 is turned off, it indicates that the user does not support enabling risk protection, and the phone can turn off risk protection.

[0277] Method 2, the shutdown event is the operation event of turning off the risk protection switch in the settings application.

[0278] The phone detects a trigger operation on the risk protection switch that is currently enabled, and can then disable risk protection, thereby disabling the face-swapping detection function under risk protection.

[0279] See Figure 26 With risk protection enabled, the phone displays interface 2601, which is the risk protection settings interface. The risk protection switch 26011 in interface 2601 is enabled. In response to the user's activation of the risk protection switch 26011 in interface 2601, the phone displays interface 2602. In interface 2602, the risk protection switch 26011 is disabled, meaning risk protection is off. At this time, the "AI Face Swap Reminder" setting 26021 in interface 2602 is also disabled, indicating that the face swap detection function is also turned off.

[0280] Furthermore, disabling risk protection will disable various risk detection functions under risk protection. Based on this, in response to the user's triggering of the risk protection switch (which is currently enabled), the phone can first display prompt 'm', such as prompt 26031 in interface 2603, to indicate that disabling risk protection will render various risk detection functions unavailable. In response to the user's confirmation of prompt 'm', such as the triggering of "Still Off" 26032 in prompt 26031, the phone will then disable risk protection, thereby disabling face-swapping detection, as shown in interface 2602 after disabling.

[0281] Method 3, the off event is the operation event of turning off the face-swapping detection switch in the settings application.

[0282] The phone detects a trigger operation on the face-swapping detection switch, which is currently enabled, and can disable face-swapping detection.

[0283] See Figure 27 When face-swapping detection is enabled, the phone displays interface 2701, which is the face-swapping detection settings interface. In interface 2701, the face-swapping detection switch 27011 is turned on. In response to the user's activation of switch 27011 in interface 2701, the phone displays interface 2702. In interface 2702, the face-swapping detection switch 27011 is turned off, meaning face-swapping detection is disabled.

[0284] It should be noted that using method 3 only disables the face-swapping detection function; the risk protection function remains enabled. This can result in situations where risk protection is on but face-swapping detection is off. For example, in the risk protection settings interface, the risk protection switch may be on, but the face-swapping detection setting may be off.

[0285] See also Figure 27 In response to the trigger operation of the return control 27021 in interface 2702, the phone can display interface 2703 (the same as interface 1501 mentioned above). Interface 2703 is the risk protection settings interface. In interface 2703, the risk protection switch 27031 is in the on state, indicating that risk protection has been enabled, and the switch state of the face-swapping detection setting item "AI face-swapping reminder" 27032 is off, indicating that face-swapping detection has not been enabled.

[0286] Please see Figure 28 In one specific embodiment, a video call method is provided. Taking the application of this method to a mobile phone as an example, it includes at least the following steps S2801 to S2804: Step S2801: During the video call, intercept the video stream and repeatedly acquire video frames.

[0287] It should be understood that Figure 28 The illustrative embodiment is that, with the detection switch turned on (i.e., user authorization obtained), face-swapping detection is performed through steps S2801 to S2804 using an active detection method.

[0288] The detection switch may include at least one of the following: a risk protection switch or a face-swapping detection switch.

[0289] It should be understood that in some scenarios, the face-swapping detection switch can be integrated into the risk protection switch, and the face-swapping detection function can be activated simply by turning on the risk protection switch. For example, in some scenarios, the risk protection switch can be decoupled from the face-swapping detection switch, so only turning on the face-swapping detection switch is needed to activate the face-swapping detection function. Furthermore, in some scenarios, both the risk protection switch and the face-swapping detection switch must be turned on simultaneously to activate the face-swapping detection function. In summary, in the embodiments of this application, as long as the detection switch can activate the face-swapping detection function, it is not limited to whether there is one or multiple detection switches, nor is it limited to the specific naming of the detection switches.

[0290] The timing for triggering the face-swapping detection switch can be found in the preceding description. Specifically, as mentioned earlier, after the phone detects the face-swapping detection activation event, it can enable the face-swapping detection function (i.e., turn on the detection switch). Activation events can include skipping risk protection settings in the Out-of-box experience (OOBE), enabling the risk protection switch in the Settings app, enabling the face-swapping detection switch in the Settings app, and performing the activation detection operation in response to prompt 1 indicating the need to enable risk protection. The process of triggering the face-swapping detection function using the above-mentioned activation events can be found in the preceding description, and the relevant interfaces can be referred to above. Figures 2 to 8 as well as Figure 10 This will not be elaborated upon here.

[0291] For example, the mobile phone can intercept video during a video call according to preset interception timing and / or interception frequency and other interception parameters to obtain video frames. For each video frame, steps S2802 to S2804 can be executed to obtain the face forgery detection result corresponding to each video frame.

[0292] The preset timing for data interception refers to the timing at which data interception is triggered. For example, data interception can start immediately after a video call is connected, or it can start 3 seconds after a video call is connected.

[0293] The throttling rate indicates the number of frames throttled per second. For example, 24 frames are throttled per second.

[0294] Furthermore, the preset throttling parameters can also include the total throttling duration. For example, a total throttling time of 3 seconds or 5 seconds.

[0295] It should be noted that all parameters related to face-swapping detection in the embodiments of this application can be configured, such as the aforementioned throttling parameters, the triggering parameters used for triggering detection (e.g., preset durations of 5 or 6), and the confidence threshold used below. Parameter configuration enables more flexible and accurate face-swapping detection.

[0296] Understandably, video calls that are passively established (i.e., the local end is called) are generally riskier. Since passively established video calls are based on video call events (i.e., video call events initiated by the other end of the call), mobile phones can identify the risks of video call events after receiving them. For video call events with risks, face-swapping detection can be initiated using either active or passive detection methods. For video call events without risks, face-swapping detection can be omitted.

[0297] In other embodiments, the mobile phone may also perform face-swapping detection for any video call, without being limited to the called party scenario, or limited to starting face-swapping detection only for video calls that pose a risk.

[0298] In some examples, the mobile phone can actively detect face-swapping for high-risk (which can be denoted as the first risk level) video call events (i.e., suspicious video calls with relatively high risk). For medium-risk (which can be denoted as the second risk level) video call events (i.e., suspicious video calls with moderate risk), a passive detection method is used for face-swapping, that is, interaction between the front end and the user is added during the detection process.

[0299] It should be noted that in other examples, the mobile phone can also use a passive detection method for face-swapping detection for high-risk (which can be denoted as the first risk level) video call events (i.e., suspicious video calls with relatively high risk). For medium-risk (which can be denoted as the second risk level) video call events (i.e., suspicious video calls with moderate risk), an active detection method can be used. Furthermore, the mobile phone can also uniformly use a default method (such as active or passive detection) for face-swapping detection for video call events of any risk level. There are no limitations on this.

[0300] For details regarding the differences in interface presentation between active and passive detection methods, please refer to the section above. Figures 11 to 22 The interface diagram description, in which, Figures 11 to 13 This is used to describe the interface diagram under the active detection method. Figures 14 to 22 This is a diagram used to describe the interface in a passive detection mode. It should be noted that... Figures 23 to 25B Although this example uses passive detection, it can also be applied to active detection methods. That is, when using active detection for face-swapping detection, if a risk is detected, it can also be addressed. Figures 23 to 25B The described solution includes risk warnings. Furthermore, the differences between active and passive detection at the data processing level will be described step-by-step below.

[0301] In some embodiments, the conditions for triggering face-swapping detection mentioned above may include meeting preset conditions and detecting a video call event within a preset time period after starting monitoring. Specifically, the mobile phone can start monitoring suspicious video calls (i.e., risky video call events) when the preset conditions are met. If a risky video call event is detected within a preset time period after starting monitoring, face-swapping risk detection is performed on the video call established based on that video call event. In other words, if the time difference between the time when the preset conditions are met and the start time of the video call is within the preset time period, it is determined that the conditions for triggering face-swapping detection are met. The start time of the video call may include the time when the video call event is received or the time when the video call is established.

[0302] Optionally, meeting the preset conditions may include at least one of the following: adding a friend event, detecting a preset risk event, or detecting a full-link trigger risk.

[0303] In combination with the above Figure 13 The following example illustrates the process of adding a friend. (See also...) Figure 13 After receiving David's friend request and adding him as a friend, the system will start listening for video call requests initiated by David within a 5-minute period. If a video call request from David is received and a video call is established within 5 minutes (i.e., a video call event is detected), it is determined that David's video call may pose a risk. In other words, if the time difference between adding a friend and the start of the video call is within 5 minutes, the conditions for triggering face-swapping detection are met. Therefore, face-swapping detection can be performed on this video call, i.e., face-swapping detection and handling can be carried out during the video call through steps S2801 to S2804.

[0304] In some embodiments, the preset risk events may include at least one of the following: receiving a risky phone call, receiving a risky text message, or downloading a risky application.

[0305] For example, after detecting at least one risk event, such as receiving a risky phone call, receiving a risky SMS message, or downloading a risky application, monitoring of suspicious video call events can be initiated within a duration of 10. Any video call event received within this duration, or even the first video call event, can be considered a risky video call event. In other embodiments, it is not limited to video call events; any video call established within the duration of 10 (e.g., a video call passively established through a video call event and / or an actively initiated video call) can also be considered risky. That is, if the time difference between the moment a risky event is detected on the phone and the start time of the video call is within a duration of 10, it is determined that the conditions for triggering face-swapping detection are met. Then, the phone can be triggered to perform face-swapping detection on the video call. It should be understood that a duration of 10 can be equal to or different from a duration of 5; it is only used to represent a preset duration.

[0306] Furthermore, risk events are not limited to the events listed above, and may also include receiving overseas calls / text messages, receiving unknown calls / text messages, downloading unknown applications and entering bank account numbers, etc., without limitation.

[0307] In some examples, risky phone calls, risky text messages, and risky apps have been identified or confirmed as posing a risk. Overseas phone calls / text messages, unknown phone calls / text messages, and unknown apps may pose a potential risk, but it has not yet been confirmed that a risk definitely exists.

[0308] End-to-end triggered risk: This refers to the determination of the existence of risk based on end-to-end risk detection. In other words, end-to-end triggered risk is an end-to-end risk detection result that indicates the presence of risk.

[0309] End-to-end risk detection refers to the comprehensive assessment of the existence of risk by combining multiple related risk behavior factors. Related risk behavior factors can refer to risk behavior factors that are related in their occurrence sequence. Therefore, detecting end-to-end triggered risk can be based on determining that the mobile terminal is at risk based on multiple related risk behavior factors.

[0310] For example, receiving an overseas call is a risk behavior factor, but it cannot be determined solely by this. However, if some risky behaviors are performed after receiving an overseas call, such as downloading unknown applications and sharing the screen, then it is very likely to be a risky call. Therefore, the end-to-end risk detection result can be determined to be risky, thus satisfying the end-to-end risk triggering condition. Consequently, the monitoring of suspicious video call events can be initiated.

[0311] Step S2802: Identify the face region of the other end of the call from the video frame.

[0312] It should be understood that in this embodiment of the application, the holder of the mobile phone is referred to as the user, and the other party in the video call with the user is referred to as the call counterpart (or the other party in the video call). Figure 29 As shown, the video call interface typically displays the faces of both parties; that is, the video frame usually shows the faces of both parties (man and woman). Figure 29 As shown in (a), assuming the woman is a local user image (such as a user using a mobile phone, also known as a mobile phone user), then, Figure 29 The man in (a) is the image of the other end of the call.

[0313] The main objective of the solution in this application embodiment is to identify whether the face of the other party in a video call is a fake face created using AI face-swapping technology to imitate the face of the user's real friend during the video call. Therefore, it is necessary to identify the face region of the other party in the video frame in step S2802, so that in step S2803 below, the other party can be identified as someone else impersonating the user's friend based on that face region.

[0314] In some examples, the phone can repeatedly capture video frames for face-swapping detection. For each current frame to be detected (i.e., the current video frame), the phone can perform face ROI region detection, obtaining at least two face regions: one on the phone itself and one on the other end of the call. The phone can then identify face regions that meet the constraints of the other end's face, which is the other end's face region (hereinafter referred to as the "other end face region"). It should be understood that the other end face region refers to the faces in the video call interface that need to be detected for face-swapping risk.

[0315] For example, the face constraints on the other end may include at least one of positional constraints or size constraints, which are described in detail below: (a) Positional constraints For example, the positional constraint may include: the other end's face region is located within a preset positional range. This preset positional range refers to the positional range of the other end's face region / face image within a video frame.

[0316] Specifically, the mobile phone can compare the facial regions identified from the video frames with preset location ranges. If a facial region is located within the preset location range, the location constraint condition is met, and the facial region can be determined to be the opposite facial region.

[0317] (ii) Size constraints Size constraints can also be called dimensional constraints. For example, a size constraint may include: the counterpart face region is a larger or smaller face region; that is, the size of the counterpart face region is greater than or less than the size of the local face region. In some embodiments, the size of the face region can be characterized by its resolution; therefore, a size constraint may include: the resolution of the counterpart face region is greater than or less than the resolution of the local face region.

[0318] It should be understood that after a video call is connected, the video call interface will be displayed in the default mode. The video call interface usually has at least two display windows. The display windows are used to display the images captured for both parties in the video call.

[0319] Taking two display windows as an example, such as Figure 29 As shown in (a), one display window displays an image (i.e., a woman) captured for the local user (i.e., the mobile phone user), and another display window displays an image (i.e., a man) captured for the other end of the call (i.e., the user of the other end's device). Each display window has a default position. Therefore, by using the above positional constraints, the face region located within the preset position range can be identified as the other end's face region.

[0320] Furthermore, in video call interfaces, the display window sizes for the local user and the remote user are typically different, resulting in different sizes of the face regions displayed within those windows. Therefore, the remote user's face region can also be identified based on these size constraints. For example, in the default display mode, the remote user's face region will be larger than the local user's face region. Therefore, the phone can identify the larger face region in the video frame as the remote user's face region. It should be noted that, assuming the remote user's face region is smaller than the local user's face region in the default display mode, the phone can also identify the smaller face region in the video frame as the remote user's face region.

[0321] Taking a video call between two people with two display windows as an example, this paper describes how to identify the face region of the other end from the perspective of data processing, using the following examples 1 and 2.

[0322] Example 1: The phone can compare the sizes of two facial regions and identify the smaller one as the opposite facial region.

[0323] Example 2: The display windows on different ends (this end and the other end) may have significant differences in display range. Therefore, during a video call, the display windows on different ends have different facial region size ranges (determined by the size of the display window). The facial region size range of this end is denoted as the first size range, and the facial region size range of the other end is denoted as the second size range. The mobile phone can compare the detected two facial regions with the second size range respectively, and identify the facial region that matches the second size range as the facial region of the other end.

[0324] It should be understood that during a video call, the user on this end may switch the display windows of both parties in the call, such as... Figure 29 As shown in (b), the image of the local user (female) is switched to a large window (i.e., a larger display window in the video call interface), while the image of the other end of the call (male) is switched to a small window (i.e., a smaller display window in the video call interface). If the position or size constraints of the other end's face are still used for recognition under the default display method, it is easy to mistakenly identify the local user's face as the other end's face, thus affecting the subsequent face swap detection of the other end's face region. Therefore, the mobile phone can adopt any one or more of the following two methods for detecting the other end's face region.

[0325] peer detection method 1: Based on the peer face image in the target video frame that conforms to the default display method, identify the peer face region in the current frame.

[0326] The "destination face image" refers to the face image of the person on the other end of the call. The "target video frame" refers to the video frame displayed in the default video call interface.

[0327] It should be understood that after a video call is connected, in the first N (N≥1) video frames, such as the first frame after the call is connected, the video call interface will generally be presented according to the default display method. That is, the faces of the user and the recipient will be displayed in their respective positions in the video call interface according to the default display method. Furthermore, based on the phone's own processing characteristics, it can determine how many frames after the video call is connected will be presented according to the default display method. In other words, the phone can know which first few video frames after the call is connected conform to the default display method. Therefore, the phone can identify the recipient's face region from any of the first N target video frames (i.e., the video frames presented according to the default display method) after the video call is connected. Then, the phone can use this recipient's face region as a reference, denoted as the reference face region. When identifying the recipient's face region from subsequent captured video frames, the phone can match all the face regions identified in the subsequent captured video frames with this reference face region, and identify the face regions that match the reference face region as the recipient's face region in the video frame.

[0328] For example, if the first frame after a video call is connected conforms to the default display mode, the other party's face region can be identified from the first frame after the video call is connected, and used as the reference face region. Then, the face regions identified in the subsequent second, third, and so on video frames can be matched with the reference face region identified in the first frame, thereby identifying the other party's face region in the second, third, and so on video frames.

[0329] It should be noted that the target video frame is not limited to the first frame alone. If the first N video frames are all displayed in the default mode, then any frame can be selected as the target video frame, and the identified face region of the other end in the target video frame can be used as the reference face region. For example, if the first 3 frames are all displayed in the default mode, then the face region of the other end can be identified from any frame, such as the first, second, or third frame, as the reference face region.

[0330] Method 2 for detecting the other end: The face constraint conditions of the peer corresponding to the current frame are determined based on the number of window switching.

[0331] The number of window switching times refers to the number of times the display windows of the two parties in the video call interface are switched.

[0332] Specifically, with user authorization or video call application authorization, the mobile phone can also listen to and obtain the number of window switching, and update the face constraint conditions of the other end corresponding to the current frame based on the number of window switching.

[0333] It should be understood that in a video call interface, when the display window switches, generally only the content displayed within the window is adjusted, without changing the size or position of the window. Taking two display windows as an example, suppose display window 1 is the main display window and display window 2 is the secondary display window (i.e., display window 1 is larger than display window 2), and by default, the other party's face is displayed in display window 1. Upon the first window switch, the display will be adjusted to show the other party's face in display window 2. Upon the second window switch, the display will revert to showing the other party's face in display window 1. Upon the third window switch, the display will again revert to showing the other party's face in display window 2.

[0334] In this way, the mobile phone can determine the target display window (i.e., the display window used to display the current frame) of the other party's face in the video call interface based on the number of window switching. Then, based on the position of the target display window, it can determine the constraints on the other party's face, such as the required positional range (i.e., positional constraints) and / or the required size (i.e., size constraints) of the other party's face region. Furthermore, based on the determined constraints on the other party's face corresponding to the current frame, the mobile phone can accurately identify the other party's face region in the current frame.

[0335] Step S2803: Perform face swapping recognition based on the face region to obtain the face forgery detection result corresponding to the video frame.

[0336] Specifically, the mobile phone can input the face region in each video frame into the forgery detection model for inference, and obtain the face forgery detection result corresponding to that video frame.

[0337] Step S2804: Based on the face forgery detection results corresponding to the video frames, determine the face-swapping risk identification results and corresponding handling for this video call.

[0338] When using active detection methods for face-swapping detection, please refer to the above. Figure 11 Before and during the detection process (steps S2801 to S2803), the user is unaware of the process. After the detection, if the face-swapping risk identification result obtained in S2804 indicates a face-swapping risk, the phone will display prompt 5, providing risk feedback. Only during this process is the user aware of the risk. For details regarding prompt 5, please refer to the above. Figure 12 A schematic description.

[0339] Next, we will combine Figure 30 Taking adding friends as an example, this section provides an overall illustration of the interface changes when using the active detection method. (See also...) Figure 30 After accepting David's friend request and adding him as a friend, if a video call request from David is received within a preset duration (e.g., 5 minutes), a face-swapping detection is performed on the video call. If a face-swapping risk is detected, a prompt is displayed, including a risk warning message (the specific content of the "Risk Reminder" on the interface) as well as controls for "Re-detect" and "Screen Recording to Keep Video". Upon receiving the user's trigger on the "Re-detect" control, the phone can re-perform the face-swapping detection process, i.e., re-intercept and re-identify the face-swapping risk. Upon receiving the user's trigger on the "Screen Recording to Keep Video" control, the phone can start screen recording to record the video call. If the number of re-detections reaches a preset threshold, the "Re-detect" control will no longer be displayed; instead, an "OK" button will be shown.

[0340] When using passive detection methods for face-swapping detection, please refer to the above. Figure 16 The system allows interaction with the user before, during, and after the detection process, enabling the user to experience the detection process firsthand. For specific interaction details, please refer to the above. Figures 14 to 27 Related descriptions.

[0341] Next, we will combine Figure 31 Taking adding friends as an example, this article provides an overall illustration of the interface changes when using a passive detection method. It should be noted that... Figure 31 In the video call interface, the man is the person on the other end of the call, and the woman is the user on the other end. This is equivalent to the user switching display windows, showing the image of the person on the other end in a large window and their own image in a small window. From Figure 31 It is known that when a video call is detected to pose a risk, the phone can display a security alert in the capsule. Once triggered by the user, this alert displays specific risk protection information, including a "Detect Now" or "Start and Detect" control. When the user triggers these controls, face-swapping detection is performed, and the capsule displays a minimized "Start Detection" message on the video call interface, showing "Detecting in progress" during the process. When the capsule is triggered by the user, it can expand to display a larger alert area for more complete information, such as "AI face-swapping detection in progress." Figure 31 It can be seen that in the detection process, the detection can target the opposite face region (i.e., Figure 31 Add detection animations to the male face area (within the image). After detection, it can display "No AI face-swapping risk detected". Further, after user triggering or within a certain time, the display area of ​​the capsule can be reduced, indicating "Detection complete".

[0342] In some examples, the phone may determine that the video call is at risk of face swapping only if the face forgery detection results of a single frame indicate that face forgery is at risk.

[0343] In other examples, the mobile phone can also combine the face spoofing detection results of multiple video frames to determine the face-swapping risk identification result and corresponding actions for the current video call. For example, the face spoofing detection result includes a face spoofing confidence score. The mobile phone can fuse the face spoofing confidence scores corresponding to multiple video frames to obtain a final confidence score. Based on this final confidence score, it can determine whether the current video call carries a face-swapping risk, obtaining a more accurate face-swapping risk identification result. The mobile phone can then take appropriate actions based on the face-swapping risk identification result.

[0344] Optionally, the mobile phone can fuse the face forgery confidence scores corresponding to multiple video frames using either direct weighted fusion or heuristic fusion.

[0345] (a) Direct weighted fusion refers to directly weighting and summing the confidence scores of face forgery corresponding to multiple captured video frames.

[0346] (ii) Heuristic fusion refers to selecting a portion of the face forgery confidence scores from multiple video frames and then weighting and fusing the selected face forgery confidence scores. The selected face forgery confidence scores are higher than those not selected; therefore, heuristic fusion based on the selected face forgery confidence scores can more accurately calculate the confidence scores.

[0347] In some embodiments, combined with Figure 32 Another software system architecture provided describes the processing flow of the video call method.

[0348] Please see Figure 32 In the application layer, the detection module includes a first detection module and a second detection module. The first detection module coordinates the identification and handling of face-swapping risks. This first detection module can be the system management service mentioned earlier. The first detection module includes a face-swapping detection module, a face-swapping risk identification and handling module, and a full-link identification module. The face-swapping detection module includes a detection scheduling module and a traffic interception module. The second detection module provides specific capabilities for face forgery detection, such as a face ROI detection module for providing face ROI detection capabilities, and a forgery detection model for providing face forgery detection capabilities, for use by the first detection module.

[0349] like Figure 32 As shown, the video call process specifically includes the following steps: Step 0: The sensing module detects the risk event at time point t.

[0350] It should be understood that Figure 32 This example only uses a preset risk event as an illustration and is not limited to initiating monitoring of suspicious video calls solely based on the detection of a risk event. It can also be triggered by detecting / initiating a friend-adding event or by detecting / initiating a risk triggered across the entire chain. Specifically, a risk triggered across the entire chain can be detected through... Figure 32 The end-to-end identification module in the system senses / detects.

[0351] Step 1: The sensing module detects the video call event within the time period [t, t+δ].

[0352] Where δ refers to the preset duration, for example, as mentioned above. Figure 13 The duration is 5, and there is no limit to this.

[0353] Step 2: The sensing module sends the video call event sensing results to the first detection module.

[0354] Step 3: The interception module calls the screenshot interface of the Window Manager to instruct the capture of video layer data from the Surface Flinger for video interception.

[0355] Step 4: The graphics synthesizer returns the cutoff buffer to the cutoff module.

[0356] Step 5: The detection scheduling module sends the captured video frames to the second detection module.

[0357] Specifically, the face ROI detection module in the second detection module can detect face regions in video frames, and the forgery detection model can perform face forgery detection inference based on the face regions detected by the face ROI detection module.

[0358] For example, the face ROI detection module can return the coordinates of the detected face region to the face-swapping detection module in the first detection module. Figure 32 (not shown in the image), so that the detection scheduling module in the face swap detection module sends the face region to the forgery detection model in the second detection module based on the face region location coordinates, so as to call the forgery detection model to perform face forgery detection inference.

[0359] In other embodiments, after the face ROI detection module detects the coordinates of the face region, it can communicate directly within the second detection module so that the forgery detection model can obtain the face region and perform face forgery detection inference, without being limited to being scheduled by the face swap detection module in the first detection model.

[0360] Step 6: The forgery detection model feeds back the confidence result of face forgery to the face swap risk identification and handling module to instruct on face swap risk identification and handling.

[0361] In some embodiments, to facilitate a better understanding of the processing flow of the video call method, the following is combined with... Figure 33 The timing diagram is described as follows: (1) The first detection module recognizes that the detection switch is turned on.

[0362] It should be understood that the detection switch (e.g., risk protection switch / face-swapping detection switch, etc.) has been turned on (i.e. the face-swapping detection function has been enabled), and the user does not need to trigger the start of detection (i.e. the user does not need to click a button or control to indicate the use of the face-swapping detection function to start detection). That is, there is no need to interact with the user. The face-swapping detection processing logic is automatically executed in the background without the user's awareness.

[0363] (2) After the first detection module determines that the preset conditions are met, it starts to listen for suspicious video calls within a preset time period.

[0364] (3) The perception module performs video call page recognition.

[0365] Specifically, within a preset time period, the sensing module can identify video call events through page recognition. For example, video call pages have specific interface features, and the sensing module can identify whether a video call event has been received based on the interface features displayed in the video call application.

[0366] (4) The sensing module sends the sensing fence result (i.e. video call event) to the first detection module.

[0367] (5) The first detection module starts to intercept the stream and acquire video frames.

[0368] (6) The first detection module reports the video frames to be inspected to the second detection module.

[0369] (7) The second detection module performs face ROI detection based on video frames.

[0370] (8) The second detection module returns the coordinates of the face region to the first detection module.

[0371] (9) The first detection module sends the face region to the second detection module.

[0372] (10) The second detection module performs forgery detection model reasoning.

[0373] Specifically, the second detection module uses a forgery detection model to perform face forgery inference based on the face region.

[0374] (11) The second detection module returns the face forgery reasoning result of the current frame to the first detection module.

[0375] (12) After the conditions for stopping the flow are met, the first detection module ends the flow interception.

[0376] (13) The second detection module ends detection and reasoning.

[0377] (14) The first detection module performs face-swapping risk identification and handling (such as high-risk pop-ups) based on the face forgery reasoning results of video frames.

[0378] In some examples, in addition to face-swapping detection, the video call method in this application embodiment also includes a full-link identification step. Specifically, the full-link identification steps can be found in [link to relevant documentation]. Figure 33 In steps (15) to (17), it should be noted that end-to-end identification is not a necessary step in the video call method of this application embodiment, and can be removed in some embodiments. The following will combine... Figure 33 Steps (15) to (17) in the diagram illustrate the full-link identification process.

[0379] (15) The first detection module reports the risk behavior sequence.

[0380] If a face-swapping risk is detected, the first detection module can report the video call event as a risky behavior event, and report other detected risky behaviors (risk events) to the decision-making module. For example, if a face-swapping risk is detected, and it is also detected that a mobile user opens a payment application and enters a transfer interface, then both should be reported.

[0381] (16) The decision module identifies based on full-link rules and models.

[0382] (17) The decision module feeds back the full-link identification results to the first detection module.

[0383] Specifically, the decision-making module can perform end-to-end risk identification based on end-to-end rules or a trained end-to-end risk identification model, and on the reported risk behavior sequence. After obtaining the end-to-end risk identification result, the decision-making module can feed back the end-to-end identification result to the first detection module. For example, based on the reported risk behavior sequence, if the decision-making module identifies that a user first had a video call with a counterparty with a risk of face swapping, and then opened a payment application to transfer money, it can determine that there is a transfer risk, and then return the identification result to prompt the user.

[0384] The above solution can detect the risk of face swapping during a video call and then track and identify the user's subsequent risky behavior in a comprehensive and end-to-end manner, thereby providing risk protection to users on a larger scale.

[0385] It should be understood that steps (5) to (11) belong to the single-frame (single video frame) processing steps of cyclic frame-by-frame detection. The steps of cyclic frame-by-frame detection are combined with Figure 33 Steps (12) to (14) are collectively referred to as the face-swapping detection and processing steps. It should be noted that the face-swapping detection and processing steps are not limited to... Figure 33 The description of steps (5) to (14) above is only based on... Figure 33 The illustration is intended to illustrate the point and should not be construed as limiting the detection and handling of face-swapping techniques.

[0386] Please see Figure 34 The steps for detecting and handling face swaps are explained in more detail, including the following: (a) The first detection module notifies the screenshot interface to start intercepting traffic.

[0387] (b) Screenshot interface for frame-by-frame screenshotting.

[0388] (c) The screenshot interface returns a screenshot of the video frame layer to the first detection module.

[0389] (d) The first detection module requests the face detection module to call the face detection module and inputs the original video frame.

[0390] (e) The face detection module loads and initializes the face detection model.

[0391] (f) The face detection module performs face ROI detection based on the face detection model.

[0392] (g) The face detection module returns the position of the face rectangle to the first detection module.

[0393] (h) The first detection module performs face screening and cutout based on the position of the face rectangle.

[0394] (i) The first detection module requests the forgery detection model from the forgery reasoning module and inputs the face cutout.

[0395] (j) The forgery detection model is used for face forgery detection inference.

[0396] (k) The forgery reasoning module returns the reasoning result (i.e., the confidence of face forgery in the current frame) to the first detection module.

[0397] (l) The first detection module performs multi-frame confidence heuristic fusion.

[0398] (m) The first detection module identifies and handles risks based on the confidence level of heuristic fusion.

[0399] The heuristic fusion processing can specifically include: a first detection module filtering out face forgery confidence scores higher than a confidence threshold, obtaining face forgery confidence scores for multiple selected video frames. Further, the first detection module can perform a weighted summation of the face forgery confidence scores from the multiple selected videos to obtain the final confidence score used for risk identification.

[0400] For example, the mobile phone can initialize an array of all zeros, with an array length equal to a preset number of throttled frames. The mobile phone can detect frames one by one. After detecting a valid frame (i.e., a valid video frame), it can refresh the array to store the corresponding confidence value. That is, for each video frame whose face spoofing confidence is calculated, it is updated and stored in the array. The face spoofing confidence values ​​stored in the array are then weighted and fused (e.g., heuristic fusion). Based on the confidence value calculated by the weighted fusion, the risk of this video call is identified to determine whether there is a risk of face swapping on the other party in this video call.

[0401] In some examples, the mobile phone can update and store the confidence scores of all video frames that meet the threshold for frame interception in an array. Then, it can perform weighted fusion (such as direct weighted fusion or heuristic fusion) based on multiple confidence scores in the array to obtain the final confidence score. Finally, it can identify risks based on the final confidence score.

[0402] In other examples, the phone can capture a video frame at a time, update the face spoofing confidence score of that frame to an array, and then perform weighted fusion (e.g., direct weighted fusion or heuristic fusion) on the currently stored face spoofing confidence scores in the array. Risk identification is then performed based on the current weighted fusion confidence score. The risk identification results can include high risk (referred to as the first risk level), moderate / medium risk (referred to as the second risk level), and low / no risk (referred to as the third risk level). If a low or high risk is identified, the screenshot interface is notified to stop further interception and detection. Furthermore, if a high risk is predicted for the current video call, appropriate action is required, such as a pop-up notification. This solution eliminates the need to detect all frames to obtain face-swapping risk identification results, saving system resources and improving detection efficiency.

[0403] If a general / medium risk is identified, the next frame is captured and sent to the second detection module for face swap detection. This process is repeated frame by frame until a low or high risk is detected, or the preset number of frames to be cut off is met, at which point the loop process ends.

[0404] In some embodiments, the specific process of risk identification based on confidence level may include any of the following methods: Method 1: Compare the confidence level with multiple preset risk confidence level intervals to determine the target risk confidence level interval. The risk level corresponding to the target risk confidence level interval is the risk level corresponding to this video call.

[0405] Method 2: Compare the confidence level with the first confidence threshold, the second confidence threshold, and the third confidence threshold respectively. If the confidence level is greater than or equal to the first confidence threshold, it is judged as high risk. If the confidence level is less than the first confidence threshold but greater than or equal to the second confidence threshold, it is judged as moderate risk. If the confidence level is less than the third confidence threshold, it is judged as low risk or no risk.

[0406] Please see Figure 35 Taking passive detection during a video call as an example, this illustrates the processing steps of a video call method.

[0407] It should be understood that Figure 33The illustrated processing steps (i.e., processing before the end of detection, including pre-detection and during detection) do not involve front-end interaction with the user, so active detection (i.e., actively performing face-swapping detection) is performed during the video call. Figure 33 and Figure 35 It is evident that one of the main differences between the two lies in whether or not the user is interacted with before face-swapping detection is performed.

[0408] like Figure 35 As shown, when the face-swapping detection process is triggered (i.e. Figure 35 Before face-swapping detection and processing (referred to as "pre-detection"), user interaction is required. For example, if the detection switch is not turned on, the user needs to actively turn on the detection switch through front-end interaction to enable the face-swapping detection function, thereby triggering the execution of subsequent face-swapping detection processing logic, as described above. Figure 15A As shown. For example, with all detection switches turned on, upon detecting a suspicious video call, the face-swapping detection process can be initiated without immediately starting. Instead, the user can be prompted to actively trigger the detection process via front-end interaction, as described above. Figure 14 As shown.

[0409] It should be understood that in passive detection scenarios, interaction with the user is not limited to "before detection," but can also include interaction during detection and interaction after detection.

[0410] Please see Figure 36 Taking passive detection as an example, it more clearly illustrates how to interact with the user on the front end before and after detection.

[0411] like Figure 36 As shown, before detection, referring to steps (4.1) to (4.2), the first detection module can instruct the notification manager to prompt the user to actively enable the detection switch, whether the detection switch is not turned on or is already turned on. For example, if the detection switch is not turned on, the module prompts the user to actively turn on the detection switch; if the detection switch is already turned on, the module prompts the user to actively enable face-swapping detection (i.e., actively trigger face-swapping detection). The prompt format can be seen in the pop-up prompt to the user in step (4.1). It should be understood that the user prompted by the mobile phone is the mobile phone user.

[0412] In some examples, if the first detection module receives a callback result (i.e., an event notification) from the perception module within a preset time after starting listening, it can determine whether the detection switch (such as the AI ​​face-swapping detection switch) is on through Settings. If the detection switch is not on, or if it is on, the RemoteView remote control software can be used to construct a dynamic capsule card, which is then used to notify the user through the NotificationManager. It should be understood that the dynamic capsule card is a notification tool that can switch between capsule and card formats. The specific processing is as follows: (a) Function not enabled: that is, the detection switch is not turned on.

[0413] The first detection module can determine whether the current video call was initiated by a newly added friend by using the newFriendInfo information. If so, it instructs the notification manager to prompt the user to turn on the detection switch via a pop-up window (such as a capsule card), for example, to turn on the face-swapping detection function.

[0414] (ii) Function enabled: that is, the detection switch is turned on.

[0415] The first detection module can also instruct the notification manager to pop up a prompt for the user to actively trigger the start of detection. For example, in the first passive detection, a capsule card can be used to prompt the user whether to perform face-swapping detection on this video call.

[0416] Furthermore, after the user performs step (4.2) to actively enable (e.g., actively turn on the detection switch or actively trigger the start of detection), the notification manager can trigger the first detection module to perform interception detection, i.e., see step (4.3).

[0417] like Figure 36 As shown, after the face-swapping detection (face forgery detection) is completed, the first detection module can instruct the notification manager to update the detection results to the user via a pop-up window, i.e., see [link to relevant documentation]. Figure 36 Step (14.1) in the prompt will update the detection results. For specific updates to the detection results, please refer to the above. Figure 18 or Figure 20 For example, it can display whether there is a risk or no risk. As mentioned earlier, updating test results can also be applied to active detection, and is not limited to active detection only.

[0418] It should be noted that, although Figure 36 The interaction during the passive detection process is not shown in the text, but in practice, when using passive detection to detect face swapping, the first detection module can also instruct the notification manager to provide corresponding prompts through pop-up windows.

[0419] In addition, during the passive detection process (i.e., passive detection), besides displaying prompts in the pop-up window, the first detection module can also instruct the addition of face recognition detection animations to the video call interface. See details in [link to relevant documentation]. Figure 17 It should be understood that when adding face recognition detection animations, from the perspective of underlying data processing, it is necessary to identify the other end's face region, and then add face recognition detection animations to the identified other end's face region. For details on how to identify the other end's face region, please refer to the implementation scheme described above, which will not be repeated here.

[0420] Please continue reading. Figure 36 In step (18), the perceptual subscription can be canceled. It should be noted that canceling the perceptual subscription is not limited to passive detection scenarios; it also applies to active detection scenarios. Figure 36 The example shown is for illustrative purposes only.

[0421] The triggers for canceling the awareness subscription include: Timing 1: If no video call event is detected or listened to within a preset time after starting suspicious video call monitoring, the subscription to the sensing fence will be canceled.

[0422] Timing 2: If a video call event is detected within a preset time period, and face swap detection is performed after the first video call occurs (i.e., after the first received video call event and the video call is established), then the perception fence subscription can be canceled after the face swap detection is completed.

[0423] This application also provides a mobile terminal, which may include a display screen, a memory, and one or more processors (such as a CPU, GPU, NPU, etc.). The display screen, memory, and processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the mobile terminal can perform various functions or steps performed by the device in the above method embodiments.

[0424] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of a mobile terminal). Or, for example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send those instructions to the processor. When the instructions are executed by the processor, the mobile terminal can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0425] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on a mobile terminal, the mobile terminal performs the aforementioned method steps to implement the image processing method described above.

[0426] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image processing method described above.

[0427] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the image processing methods in the above-described method embodiments.

[0428] In this embodiment, the mobile terminal, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0429] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0430] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0431] The unit described as a separate component may or may not be physically separate. The component shown as a unit can be one physical unit or multiple physical units, that is, it can be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0432] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0433] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0434] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A video call method, characterized in that, Applied to mobile terminals, including: A first video call is established, and a first video call interface is displayed. The first video call interface includes a first face and a second face, wherein the first face is the face corresponding to the user of the mobile terminal. In response to the first video call meeting the first condition, the system detects whether the second face is at risk of face swapping and obtains the first detection result. If the first detection result indicates that the second face has a risk of face swapping, a first prompt is displayed, indicating the first detection result. If the first detection result indicates that the second face does not have a risk of face swapping, the first prompt is not displayed. A second video call is established, and a second video call interface is displayed. The second video call interface includes a third face and a fourth face, wherein the third face is the face corresponding to the user of the mobile terminal. In response to the second video call satisfying the first and second conditions, a second prompt is displayed, which prompts the user to use the face-swapping detection function. In response to the first operation of the second prompt, detect whether the fourth face has a risk of face swapping and display a prompt indicating the detection process; After obtaining the second detection result, a third prompt is displayed, which indicates the second detection result, including whether the fourth face has a risk of face swapping or whether the fourth face does not have a risk of face swapping. The first condition and the second condition are different.

2. The method according to claim 1, characterized in that, The first condition or the second condition includes at least one of the following: The video call is established based on the video call event received by the mobile terminal; The time difference between the start time of the video call and the first moment is within a first duration. The first moment is the moment when the account corresponding to the other end of the video call is added as a friend. The account corresponding to the other end of the video call is the account logged in on the other end device that establishes the video call with the mobile terminal. The account corresponding to the other end of the video call is the first friend added after the video call application is installed on the mobile terminal; A video call is the first video call between the accounts corresponding to the other end of the video call; A video call is a video call initiated by the account corresponding to the other end of the video call; The account corresponding to the other end of the video call is in the detection list of the mobile terminal, and the detection list records the first number of friends recently added by the mobile terminal; The mobile terminal is experiencing a risk event; The time difference between the start time of the video call and the second time point is within the second duration, where the second time point is the moment when a risk event is detected on the mobile terminal; The time difference between the start time of the video call and the third moment is within the third duration. The third moment is the moment when the mobile terminal determines that the mobile terminal has a risk based on multiple related risk behavior factors.

3. The method according to claim 1, characterized in that, The first condition includes at least one of the following: the video call is established based on a video call event received by the mobile terminal; the video call is the first video call between the account corresponding to the other end of the video call; the video call is initiated by the account corresponding to the other end of the video call; the account corresponding to the other end of the video call is an account logged in on the device of the other end with which the video call is established; and the second condition is that the mobile terminal has a risk event, the risk event including at least one of the following: receiving risk information, receiving overseas calls, downloading unknown applications and performing actions such as entering bank account numbers, transferring money, or sharing screens.

4. The method according to claim 2, characterized in that, The first detection result indicates that the second face has a risk of face swapping, the second detection result indicates that the fourth face has a risk of face swapping, and the method further includes: Within a first preset time period after obtaining the first detection result or the second detection result, the transfer interface is displayed; After the transfer interface is displayed, a fourth prompt is displayed, indicating that the video call made before the transfer interface was displayed may have the risk of face swapping.

5. The method according to claim 1, characterized in that, The display of the first prompt includes: If the first video call has not ended when the risk of face swapping is detected in the second face, the first prompt is displayed on the first video call interface. The first prompt includes a re-detection control and / or a screen recording control. If the first video call has ended when the risk of face swapping is detected in the second face, the first prompt is displayed on the interface after the first video call ends. The first prompt does not include the re-detection control and the screen recording control. The re-detection control is used to trigger the mobile terminal to re-execute face-swapping detection; the screen recording control is used to record the video call process after being triggered.

6. The method according to claim 5, characterized in that, The second detection result indicates that the fourth face does not pose a risk of face swapping. The third prompt displayed after obtaining the second detection result includes: If the second video call has not ended when the second detection result is obtained, the third prompt is displayed on the second video call interface. The third prompt includes a re-detection control and / or a screen recording control. If the second video call has ended when the second detection result is obtained, the third prompt is displayed on the interface after the second video call ends. The third prompt does not include the re-detection control and the screen recording control.

7. The method according to claim 6, characterized in that, The method further includes: If the number of times the second face is detected for the risk of face swapping exceeds the number of times detected the first time, the first prompt does not include a re-detection control; If the number of times the fourth face detection checks for the risk of face swapping exceeds the number of times it was performed the first time, the third prompt does not include a re-detection control.

8. The method according to claim 7, characterized in that, The method further includes: The detection of whether the fourth face is at risk of face swapping includes: If the fourth face is not detected, a message indicating that the detection was incomplete is displayed; or, If the second video call ends before the mobile terminal acquires the video frame of the second video call, a message indicating that the detection was not completed will be displayed on the interface for ending the second video call. The detection of whether the second face poses a risk of face swapping includes: If the second face is not detected, or if the first video call ends before the mobile terminal acquires the video frame of the first video call, no message indicating that the detection was not completed will be displayed.

9. The method according to any one of claims 1-8, characterized in that, The prompts indicating the detection process include: Display at least one of the following messages: a prompt indicating that detection has started, a prompt indicating that detection is in progress, and a detection animation.

10. The method according to any one of claims 1-8, characterized in that, The display of the second prompt includes: When the second prompt is displayed from the 1st to the Nth time, the second prompt is displayed in the form of a card, which includes a description of the face-swapping detection function and / or a control to confirm the detection. When the second prompt is displayed for the N+1th time, the second prompt is displayed in capsule form, wherein the capsule-form second prompt does not include the function introduction of the face-swapping detection function and the control for confirming the detection; Where N is a natural number greater than 1.

11. The method according to any one of claims 1-8, characterized in that, The response to the second video call satisfying the first and second conditions, displaying a second prompt, includes: When the face-swapping detection switch on the mobile terminal is turned on, the second prompt includes a prompt to perform face-swapping risk detection; When the face-swapping detection switch on the mobile terminal is turned off, the second prompt includes a prompt to enable the face-swapping detection function and to perform face-swapping risk detection.

12. The method according to claim 1, characterized in that, The detection of whether the second face is at risk of face swapping includes: Capture video frames from the first video call interface; The captured video frames are subjected to face region detection, and the face regions in the detected face regions that meet the constraints of the other end face are identified as the second face; Face-swapping risk detection is performed based on the second face in the video frame.

13. The method according to claim 12, characterized in that, The face constraints at the other end include preset position constraints and / or preset size constraints.

14. The method according to claim 13, characterized in that, Before identifying the face region in the detected face region that meets the opposite face constraint condition as the second face, the method further includes: The number of window switching times when the video frame is captured is obtained. The number of window switching times refers to the number of times the display windows of the two parties in the first video call interface are exchanged when the video frame is captured. Based on the number of window switching, the face constraint conditions of the counterparty corresponding to the video frame are determined.

15. The method according to claim 14, characterized in that, The captured video frames are multiple; the step of performing face-swapping risk detection based on the second face in the video frames includes: For each captured video frame, face-swapping recognition is performed based on the second face in the video frame to obtain the face forgery confidence level corresponding to the video frame; The confidence scores of face forgery corresponding to the multiple video frames are fused to obtain the target confidence score; The presence of face-swapping risk is determined based on the target confidence level.

16. The method according to claim 15, characterized in that, The step of fusing the face forgery confidence scores corresponding to the multiple video frames to obtain the target confidence score includes: From the confidence scores of face forgery corresponding to the multiple video frames, a portion of the video frames are selected to have higher confidence scores of face forgery than those not selected. The selected confidence scores for face forgery are weighted and fused to obtain the target confidence score.

17. The method according to claim 16, characterized in that, The step of fusing the face forgery confidence scores corresponding to the multiple video frames to obtain the target confidence score includes: After capturing each video frame and calculating the confidence level of face spoofing for that video frame, the confidence level of face spoofing for that video frame is updated and stored in an array. Then, all the face spoofing confidence levels currently stored in the array are weighted and fused to obtain the target confidence level. The risk level of face-swapping risk is identified based on the target confidence level; If the risk level is medium risk, then continue to capture the next video frame; If the risk level is low or high, stop capturing video frames, and if the risk level is high, it is determined that there is a risk of face swapping.

18. The method according to any one of claims 15-17, characterized in that, The step of performing face-swapping recognition based on the second face in the video frame to obtain the face forgery confidence level corresponding to the video frame includes: The region corresponding to the second face in the video frame is input into the forgery detection model to obtain the face forgery confidence score corresponding to the video frame. The forgery detection model is deployed on the mobile terminal.

19. A mobile terminal, characterized in that, include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the mobile terminal to perform the method as described in any one of claims 1-18.

20. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed on the mobile terminal, the mobile terminal causes the mobile terminal to perform the method as described in any one of claims 1-18.

21. A computer program product comprising computer instructions, characterized in that, When the computer program product is run on a mobile terminal, the mobile terminal performs the method as described in any one of claims 1-18.

Images