Method, storage medium, program product and electronic device for portrait detection

Abstract

The application relates to the technical field of portrait detection, and discloses a method for portrait detection, a storage medium, a program product and an electronic device. In the method, M pieces of real portrait data of M users are used to determine M pieces of enhanced detection data (for example, a model, model parameters, real portrait features, etc.) corresponding to the M users respectively. Then, when an instruction of detecting a portrait of a first user is detected by a first electronic device, the first electronic device can detect whether the portrait of the first user is a fake portrait based on first enhanced detection data matched with the first user in the M pieces of enhanced detection data. Since the first enhanced detection data is determined based on the real portrait data of the first user, the detection accuracy of whether the portrait of the first user is a fake portrait is high. In this way, the risk that a user of the first electronic device mistakenly believes false information transmitted by a fake portrait of the first user can be reduced, and the privacy and property safety of the user of the first electronic device can be protected.

Description

Technical Field

[0001] This application relates to the field of human face detection technology, and in particular to a method, storage medium, program product and electronic device for human face detection. Background Technology

[0002] Artificial intelligence models have been widely used in the field of image processing. For example, generative models, large models, and other artificial intelligence models can generate images and videos, including portraits.

[0003] As AI models generate increasingly realistic portraits, users often cannot visually distinguish between real people and fake images or videos generated by AI. When scammers use fake images of people familiar to users (such as relatives, friends, colleagues, or public figures) to deliver false information (such as fake prize-winning messages, fake shopping websites, or fake investment opportunities), users may be misled, compromising their privacy and financial security. Summary of the Invention

[0004] In view of this, this application provides a method, storage medium, program product, and electronic device for human face detection.

[0005] In a first aspect, a method for human image detection is provided, applied to a first electronic device. The method includes: detecting an instruction to detect multimedia data, wherein the multimedia data includes a human image of a first user; if, among M enhanced detection data, there is first enhanced detection data matching the first user, detecting whether the human image of the first user in the multimedia data is a forged human image based on the first enhanced detection data, wherein the M enhanced detection data are obtained based on one or more real human image data of the corresponding user, and M is an integer greater than or equal to 1; if, among the M enhanced detection data, there is no enhanced detection data matching the first user, detecting whether the human image of the first user in the multimedia data is a forged human image based on general detection data.

[0006] Based on the above method, if a matching enhanced detection data point exists among M enhanced detection data points, the first electronic device can detect whether the image of the first user in the multimedia data is a forged image based on the first enhanced detection data. Since the first enhanced detection data is obtained based on the first user's real image data, the accuracy of detecting whether the first user's image is a forged image based on the first enhanced detection data is relatively high. This reduces the risk of users of the first electronic device mistakenly believing false content conveyed by a forged image of the first user, thus improving the privacy, personal safety, and property security of the users of the first electronic device.

[0007] In one possible implementation of the first aspect above, the enhanced detection data is a model, model parameters, or real human portrait features; when the M enhanced detection data are M models, the first enhanced detection data is the first model among the M models; when the M enhanced detection data are M sets of model parameters of the second model, the first enhanced detection data is the first model parameter among the M sets of model parameters; when the M enhanced detection data are M real human portrait features, the first enhanced detection data is the first real human portrait feature among the M real human portrait features.

[0008] In one possible implementation of the first aspect above, the detection of whether the image of the first user in the multimedia data is a forged image based on the first enhanced detection data includes: when the first enhanced detection data is a first model, determining whether the image of the first user in the multimedia data is a forged image by calling the first model; or, when the first enhanced detection data is parameters of the first model, determining whether the image of the first user in the multimedia data is a forged image based on a second model having the first model parameters; or, when the first enhanced detection data is first real image features, determining whether the image of the first user in the multimedia data is a forged image by calling a third model and based on the first real image features.

[0009] In one possible implementation of the first aspect above, the existence of first enhanced detection data matching the first user among the M enhanced detection data includes: the existence of a first user feature matching the first user identifier and / or the first facial feature of the first user's image among the M user features, wherein the M user features correspond one-to-one with the M enhanced detection data, the user features include the user's facial features and / or user identifier, and the first user feature is the user feature corresponding to the first enhanced detection data among the M user features; or, an operation is detected in which the user selects the first enhanced detection data from the M enhanced detection data.

[0010] In this implementation, the first electronic device can determine whether there is first enhanced detection data matching the first user among the M enhanced detection data by matching the first user identifier of the first user and / or the first facial feature of the first user's image in the multimedia data with M user features. If there is a first user feature among the M user features that matches the first user identifier and / or the first facial feature of the first user's image, the first electronic device can determine that the first enhanced data corresponding to the first user feature in the M enhanced detection data matches the first user.

[0011] Alternatively, the first electronic device may also determine, upon detecting that the user has selected first enhanced detection data for detecting the image of the first user in the multimedia data, that there is first enhanced detection data among the M enhanced detection data that matches the first user.

[0012] In one possible implementation of the first aspect described above, among the M user features, there exists a first user feature that matches the first user identifier and / or the first facial feature of the first user's image, including: a first similarity between the first user identifier and the user identifier in the first user features being greater than a first similarity threshold, and / or a second similarity between the first facial feature and the user facial feature in the first user features being greater than a second similarity threshold; wherein, a first similarity greater than the first similarity threshold indicates that the first user identifier and the user identifier in the first user features belong to the same user; and a second similarity greater than the second similarity threshold indicates that the first facial feature and the user facial feature in the first user features belong to the same user.

[0013] In this implementation, the first electronic device can determine whether each of the M user features matches the first user identifier and / or the first face feature based on a first similarity between the first user identifier and the user identifier among the M user features, and / or a second similarity between the first user face feature and the user face feature among the M user features. If the first similarity between the first user identifier and the user identifier among the first user features is greater than a first similarity threshold, and / or the second similarity between the first face feature and the user face feature among the first user features is greater than a second similarity threshold, the first electronic device can determine that the first enhanced detection data corresponding to the first user feature matches the first user.

[0014] In one possible implementation of the first aspect described above, the first user identifier is determined by: when the multimedia data is an image or video in a video call or video conference, determining the first user identifier based on one or more of the first user's title, nickname, remarks, phone number, and name in the video call or video conference; or, when the multimedia data is multimedia data in a chat session between the user of the first electronic device and the first user, determining the first user identifier based on one or more of the first user's nickname, remarks, phone number, and name in the chat session; or, obtaining the first user identifier selected or input by the user of the first electronic device; or, obtaining the first user's user identifier from an instruction to detect the multimedia data.

[0015] In one possible implementation of the first aspect described above, the first enhanced detection data is received from the second electronic device; or, the first enhanced detection data is generated by the first electronic device based on one or more real human portrait data of the first user.

[0016] In this implementation, the first enhanced detection data can be data already stored by the second electronic device and shared by the second electronic device to the first electronic device. Alternatively, the first enhanced detection data can be generated by the first electronic device sending one or more real-image data of the first user to the second electronic device, and then the second electronic device generating the first enhanced detection data based on the one or more real-image data of the first user and sending it back to the first electronic device. Alternatively, it can be data obtained by the first electronic device based on one or more real-image data of the first user.

[0017] In one possible implementation of the first aspect above, the real facial data of the first user includes: real facial data of the first user stored in the first electronic device; and / or, real facial data of the first user in the chat history of an instant messaging application; and / or, real facial data of the first user obtained from the second electronic device; and / or, real facial data of the first user captured by the first electronic device.

[0018] In this implementation, the source of the first user's real facial image data can be multiple.

[0019] In one possible implementation of the first aspect above, the aforementioned detection of the instruction to detect multimedia data includes: detecting that a user triggers an operation to detect a fake human image during a video call or video conference conducted by the first electronic device, wherein the multimedia data is an image or video in the video call or video conference; or detecting that the first electronic device is in the process of a video call or video conference, and the first electronic device is configured to automatically detect whether the human image in the video call or video conference is a fake human image, wherein the multimedia data is an image or video in the video call or video conference; or detecting that there is a security risk during the first electronic device's video call or video conference, wherein the multimedia data is an image or video in the video call or video conference; or detecting that a user triggers an operation to detect a fake human image in the multimedia data; or detecting that there is a security risk in a chat session of the user of the first electronic device, wherein the content of the chat session includes multimedia data; or receiving a detection request sent by a fourth electronic device, the detection request including multimedia data; or detecting that a first application is running, and the first electronic device is configured to automatically detect whether the human image in the multimedia data in the first application is a real human image during the running of the first application.

[0020] In this implementation, the first electronic device can detect instructions to detect multimedia data in the aforementioned various scenarios.

[0021] In one possible implementation of the first aspect described above, the method further includes: outputting a prompt message when the image of the first user in the multimedia data is detected to be a fake image; and / or, sending the detection result of whether the image of the first user in the multimedia data is a fake image to the fourth electronic device when a detection request is received.

[0022] In this implementation, the first electronic device may use a forged image of the first user in the multimedia data to alert the user of potential risks and remind the user to pay attention to their personal safety, privacy, and property security through text, images, audio, vibration, video, animation, etc., and / or send the detection results to a fourth electronic device.

[0023] In one possible implementation of the first aspect above, the multimedia data is received from a fourth electronic device; and the method further includes sending to the fourth electronic device a detection result of whether the image of the first user in the multimedia data is a forged image.

[0024] In this implementation, the first electronic device can receive multimedia data sent by the fourth electronic device and detect whether the image of the first user in the multimedia data is a fake. Then, the first electronic device can send the detection result to the fourth electronic device, so that if the detection result indicates that the image of the first user in the multimedia data is a fake, the fourth electronic device can output a prompt message to remind the user of their personal safety, privacy, and property security. Thus, even when the computing power of the fourth electronic device is limited, the detection of whether the image in the multimedia data is a fake can be achieved with the help of the first electronic device.

[0025] In one possible implementation of the first aspect above, the multimedia data further includes a portrait of the second user; the method further includes: if there is second enhanced detection data matching the second user among the M enhanced detection data, detecting whether the portrait of the second user in the multimedia data is a forged portrait based on the second enhanced detection data, wherein the second enhanced detection data is obtained based on one or more real portrait data of the second user; if there is no enhanced detection data matching the second user among the M enhanced detection data, detecting whether the portrait of the second user in the multimedia data is a forged portrait based on general detection data.

[0026] In this implementation, when the multimedia data includes images of multiple users, if there is enhanced detection data among the M enhanced detection data that matches one or more users, the first electronic device can detect whether the image of the corresponding user is a fake image based on the enhanced detection data corresponding to the one or more users respectively.

[0027] In one possible implementation of the first aspect above, the method further includes: detecting that the update conditions corresponding to the first user are met, and updating the first enhanced detection data; wherein the update conditions corresponding to the first user include any one of the following conditions: detecting that the first electronic device has captured at least one real human image data of the first user; detecting that the user updates the enhanced detection data of the first user; and reaching a preset update period.

[0028] In this implementation, the first electronic device can update the stored enhanced detection data to further improve the accuracy of the detection results obtained based on the enhanced detection data to determine whether the user's portrait is a forged portrait.

[0029] In one possible implementation of the first aspect above, the method further includes: in response to a user's sharing operation, sending one or more of the M enhanced detection data to a fifth electronic device or a first group.

[0030] In this implementation, the first electronic device can share M enhanced detection data points with the fifth electronic device or the first group. This allows the fifth electronic device or the electronic device corresponding to the first group (e.g., an electronic device within the first group or a user's electronic device within the first group) to detect whether the corresponding user's image is a forged image based on the one or more enhanced detection data points.

[0031] Secondly, embodiments of this application also provide a method for face detection, the method comprising:

[0032] The fourth electronic device sends a detection request to the first electronic device, requesting the detection of whether the image of the first user in the multimedia data is a fake image. In response to the detection request, if there is a first enhanced detection data matching the first user among M enhanced detection data, the first electronic device detects whether the image of the first user in the multimedia data is a fake image based on the first enhanced detection data, where the M enhanced detection data are obtained based on one or more real image data of the corresponding user, and M is an integer greater than or equal to 1. In response to the detection request, if there is no enhanced detection data matching the first user among the M enhanced detection data, the first electronic device detects whether the image of the first user in the multimedia data is a fake image based on general detection data. The first electronic device sends the detection result of whether the image of the first user in the multimedia data is a fake image to the fourth electronic device. The fourth electronic device outputs a prompt message based on the detection result.

[0033] Based on the above method, the first electronic device can collaborate with the fourth electronic device to detect whether the image of the first user in the multimedia data is a forged image. Since the first enhanced detection data is obtained based on the first user's real image data, the accuracy of detecting whether the first user's image is a forged image based on the first enhanced detection data is high. This reduces the risk of users of the fourth electronic device mistakenly believing false content conveyed by a forged image of the first user, thus improving the privacy, personal safety, and property security of users of the fourth electronic device.

[0034] Thirdly, a readable storage medium is provided, which includes one or more programs that, when executed on an electronic device, enable the electronic device to implement any of the methods for human face detection provided in the first and second aspects above.

[0035] Fourthly, an electronic device is provided, comprising: a memory for storing one or more programs; and a processor for executing any of the methods for human face detection provided in the first and second aspects above.

[0036] Fifthly, a program product is provided that, when executed on an electronic device, enables the electronic device to implement any one of the methods for human face detection provided in the first and second aspects above.

[0037] It should be noted that the beneficial effects of the second to fifth aspects mentioned above can be referred to the content of the first aspect, and will not be repeated here. Attached Figure Description

[0038] Figure 1 According to some embodiments of this application, a schematic diagram of a scenario for identifying forged human images is shown.

[0039] Figure 2 According to some embodiments of this application, a schematic diagram of human face detection is shown.

[0040] Figure 3A According to some embodiments of this application, a schematic diagram of an interface for triggering the determination of enhanced detection data for a user is shown.

[0041] Figure 3B According to some embodiments of this application, a schematic diagram of an interface for selecting real human portrait data of a user is shown.

[0042] Figure 3C According to some embodiments of this application, a schematic diagram of an interface is shown in the process of determining enhanced detection data for a user.

[0043] Figure 3D According to some embodiments of this application, a schematic diagram of a scenario is shown in the process of determining enhanced detection data for a user.

[0044] Figure 3E According to some embodiments of this application, a schematic diagram of an interface for capturing real human portrait data of a user is shown.

[0045] Figure 4A According to some embodiments of this application, another schematic diagram of an interface for triggering the determination of enhanced detection data for a user is shown.

[0046] Figure 4B According to some embodiments of this application, a schematic diagram of an interface for obtaining real human portrait data of a user is shown.

[0047] Figure 5 According to some embodiments of this application, a schematic diagram of a scenario for sharing enhanced detection data is shown.

[0048] Figure 6 According to some embodiments of this application, a schematic diagram is shown of an augmentation model that triggers training of users in a gallery application.

[0049] Figure 7A According to some embodiments of this application, a schematic diagram is shown of triggering the detection of whether a person's image is a fake image during a video call.

[0050] Figure 7B According to some embodiments of this application, another schematic diagram is shown for triggering the detection of whether a person's image is a fake image during a video call.

[0051] Figure 7C According to some embodiments of this application, a schematic diagram is shown of triggering the detection of whether a portrait is a fake portrait in a chat session.

[0052] Figure 7D According to some embodiments of this application, an interface diagram of a user corresponding to a person's image detected in a chat session is shown.

[0053] Figure 7E According to some embodiments of this application, another schematic diagram of the interface corresponding to a user's image detected in a chat session is shown.

[0054] Figure 7F According to some embodiments of this application, another schematic diagram is shown for triggering the detection of whether a portrait is a fake portrait in a chat session.

[0055] Figure 7G According to some embodiments of this application, a schematic diagram is shown of a method for triggering the detection of whether a human image is a forgery in response to a detection request from another electronic device.

[0056] Figure 8According to some embodiments of this application, an interactive flowchart of a method for determining enhanced detection data for a user is shown.

[0057] Figure 9A According to some embodiments of this application, a schematic diagram of a method for human face detection for a first electronic device is shown.

[0058] Figure 9B According to some embodiments of this application, an interactive flowchart of a method for human face detection is shown.

[0059] Figure 10 According to some embodiments of this application, a schematic diagram of a method for a second electronic device to determine enhanced detection data is shown.

[0060] Figure 11 According to some embodiments of this application, a schematic diagram is shown of an instruction that triggers the determination of first enhanced detection data of a first user.

[0061] Figure 12 A schematic diagram of a software architecture is shown according to some embodiments of this application.

[0062] Figure 13 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown.

[0063] Figure 14 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 200 is shown. Detailed Implementation

[0064] The illustrative embodiments of this application include, but are not limited to, methods for human face detection, readable storage media, computer program products, and electronic devices.

[0065] To prevent fake portraits generated by artificial intelligence models or other means from affecting users' privacy and financial security, in some embodiments, electronic devices can use a general portrait detection model to identify whether a portrait in an image or video is fake. If a fake portrait is identified, the electronic device can alert the user to avoid being misled by false information conveyed by the fake portrait. This protects users' privacy and financial security, for example, preventing users from sending personal information or transferring money to fraudsters due to being misled by false information.

[0066] For example, Figure 1 According to some embodiments of this application, a schematic diagram of a scenario for identifying forged human images is shown.

[0067] like Figure 1As shown, during a video call between user A's phone 10 and user B's phone 20, phone 10 can use a general facial recognition model to detect whether user B's image in the video call is a fake. If phone 10 detects that user B's image is a fake, it can display a prompt message U11 to remind user A, "A suspected fake image has been detected. Please pay attention to your privacy and property security." This prevents user A from mistakenly believing the false content in user B's fake image, thus avoiding privacy leaks or financial losses.

[0068] It should be noted that general-purpose facial recognition models are typically trained on training data that includes multiple real and multiple fake facial images. The accuracy of a general-purpose facial recognition model in detecting whether a user's facial image is fake usually depends on whether there are real or similar images of that user in the training data. Because the training data is often limited, general-purpose facial recognition models may not accurately detect whether some users' facial images are fake, and the privacy and property security of these users remain at risk.

[0069] For example, in the aforementioned Figure 1 In the scenario depicted, the training data for a general-purpose facial recognition model typically does not include user B's real facial image data. This model may fail to accurately detect highly realistic forged images of user B. Consequently, user A could still be misled by user B's forged image, potentially leading to the leakage of user A's privacy information or financial loss.

[0070] To improve the accuracy of detecting forged portraits, embodiments of this application provide a method for portrait detection. In this method, for any user, enhanced detection data (e.g., models, model parameters, real portrait features, etc.) can be obtained based on one or more real portrait data of that user (e.g., images, videos, etc., including the user's real portrait). Then, an electronic device can detect whether the user's portrait in an image is a forged portrait based on the enhanced detection data corresponding to that user, obtaining a detection result (e.g., real portrait or forged portrait).

[0071] For example, in some embodiments, the same or different portrait detection models can be trained using one or more real portrait data of a user to obtain an enhanced portrait detection model with high accuracy in detecting whether the portrait of that user is a fake portrait. That is, for different users, the corresponding enhanced detection data can be different enhanced portrait detection models (hereinafter referred to as enhanced models).

[0072] For example, an electronic device can use one or more real human images of a user as sample data and use the judgment result of real human image as label to train a human image detection model, so that the trained enhanced human image detection model can obtain the result of "real human image" for one or more real human images of the user as input.

[0073] For example, electronic devices can also use one or more real human image data of a user and multiple fake human image data that have been modified from real human image data as sample data, use real human image data as labels for real human image data, and use fake human image data as labels for fake human image data to train a human image detection model, so that the trained enhanced human image detection model can obtain the result of "real human image" for one or more real human image data of the user and the result of "fake human image" for the multiple fake human image data.

[0074] In some embodiments, the face detection model can be a model pre-trained based on real face data and fake face data of multiple users. This can improve the training convergence speed of electronic devices in the process of training the face detection model based on one or more real face data of a user to obtain the corresponding augmented model for that user.

[0075] For example, in some embodiments, the same portrait detection model (hereinafter referred to as the second model) can be trained using one or more real portrait data of a user to obtain the model parameters of the second model when the detection accuracy of the second model for the user's portrait is high. That is, for different users, the corresponding enhanced detection data can be different model parameters of the second model, wherein the model parameters can be some or all of the model parameters of the second model.

[0076] For example, an electronic device can use one or more real human portrait data of a user as sample data and use the judgment result of real human portrait as label to train a second model, so that the trained second model has a high accuracy in detecting whether the input human portrait of the user is a fake human portrait, and use some or all of the model parameters of the trained second model as the augmented detection data corresponding to the user.

[0077] For example, an electronic device can use one or more real human image data of a user, as well as multiple fake human image data modified from the real human image data, as sample data. Data that is judged as real human image data is used as the label for real human image data, and data that is judged as fake human image data is used as the label for fake human image data. This allows the trained enhanced human image detection model to obtain a "real human image" result for one or more real human image data of the user, and a "fake human image" result for the aforementioned multiple fake human image data. Then, the electronic device can use some or all of the model parameters of the trained second model as the enhanced detection data corresponding to that user.

[0078] In some embodiments, the second model has high accuracy in detecting whether the input image of a user is a fake image. This can be achieved by the second model obtaining a "real image" detection result for one or more real image data of the input user, and a "fake image" detection result for one or more fake images of the input user.

[0079] In some embodiments, the accuracy of the second model in detecting whether the input image of a user is a fake image is high, which can be: the accuracy of the second model in detecting whether one or more images of the input user are real images is greater than a preset value.

[0080] For example, in some embodiments, a partial model (hereinafter referred to as the third model) can determine whether the image of a user in the multimedia data to be detected is a fake image based on the facial features of a user's real facial data (hereinafter referred to as real facial features) and the multimedia data to be detected. In this case, for a user, the user's real facial features can be obtained based on one or more real facial data of that user. That is, for different users, the corresponding enhanced detection data can be different real facial features.

[0081] For example, an electronic device can pre-train a third model so that the third model can determine whether the input image is a fake image of the user corresponding to the input real image features based on the input image and real image features. Then, after acquiring one or more real image data of a user, the electronic device can train to obtain the user's real image features by repeating the following process: S1, extracting features from the one or more real image data to obtain the user's real image features; S2, inputting the extracted real image features and one or more images of the user (which may include real image data and / or fake image data) into the third model to obtain the detection results of the one or more images; S3, if the accuracy of the detection results is greater than the accuracy threshold, using the real image features obtained in S1 as the user's real image features; S4, if the accuracy of the detection results is less than or equal to the accuracy threshold, repeating S1 to S4.

[0082] It should be noted that the portrait detection model, the second model, and the third model can be any model used to detect whether a portrait in an input image or video is a forged portrait. Examples include data-efficient image transformer (DeiT) models, vision transformer (ViT) models, multi-attention deepfake detection (MADD) models, LipForensics (a model for detecting forged portraits based on lip features), diffusion models, MesoNet (a model for detecting forged portraits based on microscopic features in images), XceptionNet (a model for detecting forgery traces based on frequency domain features of images), FakeCatcher, generative adversarial networks fingerprint detection models, Face X-ray (a detection model for detecting boundary artifacts in forged images), and large models. This application does not limit the specific form of the portrait detection model (or enhancement model), the second model, and the third model.

[0083] It should be noted that the above method of determining the user's corresponding augmented detection data based on the user's real facial data is only an example. In other embodiments, the user's augmented detection data can also be determined by other methods, which are not limited here.

[0084] Based on the aforementioned enhanced detection data, the first electronic device can store M enhanced detection data points, each corresponding one-to-one with the user characteristics of M users. When the first electronic device detects a request to determine whether the image of the first user in the multimedia data is a forged image, it can detect whether the image of the first user is a forged image based on the first enhanced detection data if the first enhanced detection data corresponding to the first user exists among the M enhanced detection data points; otherwise, it can detect whether the image of the first user is a forged image using general detection data, where M is a positive integer greater than 1.

[0085] It should be noted that the general detection data is used to detect the faces of users other than the M users.

[0086] For example, when the augmented detection data is used as the model, the general detection data can be a general portrait detection model. This general portrait detection model can have the same structure but different model parameters as the M augmented detection models corresponding to the M users, or the general portrait detection model can have a different structure from the M augmented detection models; this is not limited here. It should be noted that the general portrait detection model can be trained based on real portrait data of multiple users. These multiple users can be completely different from the aforementioned M users, or they can include some or all of the aforementioned M users; this is not limited here.

[0087] For example, when the enhanced detection data serves as the model parameters for the second model, the general detection data can also serve as general model parameters. These general model parameters can be obtained by training the second model based on real facial data from multiple users. It should be noted that these multiple users can be completely different from the aforementioned M users, or they can include some or all of the aforementioned M users; this is not limited here.

[0088] For example, when the enhanced detection data is based on real human facial features, the general detection data can also be a general human facial feature, which can be obtained based on real human facial data from multiple users. It should be noted that these multiple users can be completely different from the aforementioned M users, or can include some or all of the aforementioned M users; this is not limited here.

[0089] It should be noted that when the enhanced detection data contains real human facial features, the general detection data can also be empty. In this case, the third model can determine whether the human image in the input multimedia data is a fake image without inputting real human facial features.

[0090] It should be noted that user features can be any feature that can identify different users. For example, user features can include user facial features and / or user identifiers. Among them, user facial features can be feature data extracted from the faces of one or more real human images of the user, or they can be real facial images / videos of the user; user identifiers can be information such as the user's name, phone number, email address, nickname, or account in an instant messaging application, or they can be feature data extracted from information such as the user's name, phone number, email address, nickname, or account in an instant messaging application.

[0091] In some embodiments, user facial features can be used to indicate features of a user’s real face, such as skin color, face shape, size / shape / position of eyes / mouth / nose / ears, eye color, outline / color of hair (hair, eyebrows, beard, etc.).

[0092] It should be noted that multimedia data may include one or more images, or one or more videos. Images may be either moving or still images; there is no limitation on this.

[0093] The above methods can improve the accuracy of detecting whether the images of the above M users are fake images, and effectively protect the privacy and property security of users associated with the above M users (such as users of the first electronic device).

[0094] For example, Figure 2 According to some embodiments of this application, a schematic diagram of human face detection is shown.

[0095] like Figure 2 As shown, the first electronic device can store the correspondence between the aforementioned M enhanced detection data and the corresponding user features (e.g., user facial features, user identifiers, etc.) of each enhanced detection data. For example, the M user features (e.g., user feature U1 to user feature U2) can be stored in the first electronic device. M ) and M enhanced detection data (e.g., enhanced detection data D1 to enhanced detection data D) M A one-to-one correspondence relationship (hereinafter referred to as the first correspondence relationship) is established. Based on the above first correspondence relationship, after detecting a detection request to detect a person's image (e.g., the image of the first user) in the multimedia data, the first electronic device can extract the first facial feature of the person's image and / or the first user identifier of the first user in the multimedia data. Then, the first electronic device can determine whether one of the M user features in the first correspondence relationship matches the first facial feature and / or the first user identifier. Next, the first electronic device can select user feature U from the M user features. PIn the case of matching the first facial feature and / or the first user identifier, based on the user feature U in the first correspondence relationship P (as the first user feature) corresponding to the enhanced detection data D P (As the first enhanced detection data) to detect whether the image of the first user in the multimedia data is a fake image, and obtain the detection result.

[0096] In some embodiments, user feature U P Matching with the first facial feature and / or the first user identifier may include at least one of the following scenarios A1 to A3:

[0097] A1, User Characteristics U P It only includes the user identifier, the first user identifier and the user feature U. P The user identifiers are the same (equivalent to a similarity of 100%), or the first user identifier matches the user feature U. P The similarity between user identifiers in the dataset is greater than a first similarity threshold. Specifically, the similarity between the first user identifier and user feature U... P If the similarity between user identifiers in the dataset is greater than a first similarity threshold, it indicates that the first user identifier and user feature U are similar. P The user identifier in the data is the same user identifier as the user identifier of the same user (i.e., the first user identifier and user feature U). P (The user identifiers in the data belong to the same user).

[0098] It should be noted that the first user identifier and user characteristic U P The greater the similarity between user identifiers in the dataset, the closer the first user identifier is to user feature U. P The higher the similarity threshold, the greater the probability that the user identifier in the first similarity threshold matches the user identifier of the same user. Therefore, the higher the value of the first similarity threshold, the greater the likelihood that the first user identifier and user feature U are. P The higher the accuracy of the match.

[0099] It should be noted that the first similarity threshold can be an empirical or experimental value, such as 90%, 95%, 99%, or any other arbitrary value. A2, User Feature U P It only includes the user's facial features, the first facial feature and the user feature U. P Users whose facial features are identical (equivalent to a 100% similarity) or whose first facial feature matches the user's feature U P The similarity between the user's facial features in the first facial feature and the user feature U is greater than the second similarity threshold. P If the similarity between the user's facial features is greater than the second similarity threshold, it indicates that the first facial feature is similar to the user feature U. PThe user facial features in the image represent the faces of the same user (i.e., the first facial feature and the user feature U). P (The facial features of the users in the data belong to the same user).

[0100] It should be noted that the first facial feature and the user feature U P The greater the similarity between the user's facial features, the better the similarity between the first facial feature and the user's feature U. P The higher the probability that the facial features of the users in the data belong to the same user, the greater the likelihood that they are the same user. Therefore, the higher the value of the second similarity threshold, the more likely the match between the first facial feature and the user feature U will be. P The higher the accuracy of the match.

[0101] It should be noted that the second similarity threshold can be an empirical or experimental value, such as 91%, 95%, 99%, or any other arbitrary value. The second similarity threshold can be the same as or different from the first similarity threshold; no limitation is made here.

[0102] A3, User Characteristics U P This includes user identifiers and user facial features. The first user identifier and user feature U P The user identifiers are the same (equivalent to a similarity of 100%), or the first user identifier matches the user feature U. P The similarity between user identifiers in the data is greater than a first similarity threshold, and / or, the first facial feature and user feature U... P Users whose facial features are identical (equivalent to a 100% similarity) or whose first facial feature matches the user's feature U P The similarity between the facial features of users in the dataset is greater than the second similarity threshold. Based on Figure 2 As shown, due to the enhanced detection data D P It is based on the real facial data of the first user, which can improve the accuracy of detecting whether the first user's facial image is a fake image.

[0103] The following section uses enhanced detection data as an example to illustrate how to acquire enhanced detection data. In some embodiments, the operating system of the first electronic device or an application within the first electronic device may be configured with a function for detecting whether a person's image in multimedia data is a real person's image (e.g., intelligent anti-fraud detection, face-swapping detection, or other functions; the following section uses intelligent anti-fraud detection as an example). The first electronic device can determine the enhanced detection data of the user or other users based on the user's actions in the configuration interface of the intelligent anti-fraud detection function.

[0104] For example, taking a mobile phone 10 as the first electronic device, Figures 3A to 3EAccording to some embodiments of this application, a schematic diagram is shown of enhanced detection data for determining the user of mobile phone 10.

[0105] like Figure 3A As shown, the security settings interface U12 of mobile phone 10 may include the intelligent anti-fraud detection option U13, the enhancement model management option U14 (including the My Enhancement Module option), and the automatic update enhancement model option U15.

[0106] Among them, the Smart Anti-Fraud Detection option U13 is used to turn the Smart Anti-Fraud Detection function on the phone 10 on or off.

[0107] The enhancement model management option U14 may include options for managing the enhancement model for users of mobile phone 10, as well as options for managing the enhancement model for users other than users of mobile phone 10.

[0108] For example, the enhanced model management option U14 may include a device owner enhanced model option U141, which is used to train or update the enhanced model of the user of the mobile phone 10 (e.g., the aforementioned user A). If the enhanced model of user A is not trained or stored in the mobile phone 10, the device owner enhanced model option U141 may include a training option U142 to trigger the training of the enhanced model of user A; if the enhanced model of user A is already stored in the mobile phone 10, the device owner enhanced model option U141 may include an update option to trigger the updating of the enhanced model of user A.

[0109] For example, the enhancement model management option U14 may also include options for adding, deleting, and updating enhancement models stored in mobile phone 10 for users other than the user of mobile phone 10. Exemplarily, the enhancement model management option U14 may include update and delete options for enhancement models already stored in mobile phone 10, such as update option U143 and delete option U144 for the enhancement model corresponding to user B. Exemplarily, the enhancement model management option U14 may include an add option U145 for adding enhancement models.

[0110] The automatic update enhancement model option U15 is used to configure whether to automatically update the enhancement model of the user of mobile phone 10 and / or the enhancement models of other users stored in mobile phone 10. When the automatic update enhancement model option U15 is enabled, mobile phone 10 can update the enhancement model of the corresponding user based on the real portrait data of user A (or other users whose corresponding enhancement models are already stored in mobile phone 10) captured by mobile phone 10.

[0111] For example, after capturing one or more images (or videos) including a person, mobile phone 10 can determine the facial features of the person in the images. Then, the determined facial features can be matched with the M user features in the first correspondence mentioned above. If one of the M user features matches the determined facial feature, mobile phone 10 can update the enhancement model corresponding to that user's feature based on the one or more images. These one or more images can be unmodified images captured by mobile phone 10. (Continue to refer to...) Figure 3A In response to a user's selection (e.g., clicking) of training option U142, mobile phone 10 can display the host augmentation model training interface U16, which may include a sample source selection option U161. The sample source selection option U161 may include an image selection option U162 and an image capture option U163.

[0112] refer to Figure 3B In response to the user's click on the image selection option U162, the mobile phone 10 can display the image selection interface U17 to prompt the user to "Please select your real portrait". The image selection interface U17 may include thumbnails of multiple images (and / or videos) and a confirmation option U171. These multiple images (and / or videos) may include, but are not limited to, images and / or videos stored on the mobile phone 10, and / or images and / or videos stored in the cloud, and / or videos in the user A's home storage (e.g., network attached storage, NAS), and / or chat logs / group files / group shares of the instant messaging application logged into the user A's account on the mobile phone 10.

[0113] It should be noted that, in some embodiments, the selection of the aforementioned plurality of images (and / or videos) in the image interface U17 may also include images or videos from other sources, which are not limited herein. Optionally, in some embodiments, reference... Figure 3B After detecting that the user selects one or more images in the image selection interface U17 and then clicks the "OK" option U171, the mobile phone 10 can display the image selection interface U18 if one of the selected images (or videos) contains multiple portraits, prompting the user that "Some images / videos contain multiple portraits, please select your real portrait." Then, the mobile phone 10 can obtain the real portrait of the user A selected from the images / videos containing multiple portraits.

[0114] refer to Figure 3CIn response to the user's action of selecting a real image of user A in the image selection interface U18 and clicking the confirmation option U181, the mobile phone 10 can train an augmented model corresponding to user A based on the real images (and / or videos) selected by the user, and display the model training interface U19 to prompt the user "Augmented model training in progress, please wait". After obtaining the augmented model of user A, the mobile phone 10 can display the training completion interface U20 to prompt the user "Augmented model training complete". The training completion interface U20 may include a share option U201 and a return option U202.

[0115] Optionally, the mobile phone 10 may respond to the user clicking the share option U201 to share the enhanced model of user A with other users; or, the mobile phone 10 may respond to the user clicking the return option U202 to display the security settings interface.

[0116] In some embodiments, user A of mobile phone 10 can also share enhanced models of user A or other users stored in mobile phone 10 within a first group. The first group can be any type of data-sharing group, such as a group in an instant messaging application, a group established based on a cloud service account, a group of related electronic devices (e.g., a group corresponding to devices of family members), or a group established through near-field communication (e.g., Bluetooth, Wi-Fi, etc.). Based on this approach, other users in the first group can also detect whether the corresponding user's image is a forged image based on user A's enhanced model or the enhanced models of other users stored in mobile phone 10.

[0117] In some embodiments, mobile phone 10 can train an augmentation model for the user of mobile phone 10 based on real human images in one or more images and / or videos selected by the user.

[0118] In some embodiments, mobile phone 10 may also send real human images (or features of such real human images) from one or more images and / or videos selected by the user to other electronic devices (e.g., a third electronic device with higher computing power than mobile phone 10), and receive an enhanced model of user A obtained by the third electronic device based on the real human images from the one or more images and / or videos. For example, refer to Figure 3D Mobile phone 10 can send a training request to a third electronic device, which may include real human images (or features of such real human images) selected by the user from one or more images and / or videos. After receiving the training request, the third electronic device can obtain an enhanced model of user A based on the real human images in the one or more images and / or videos, and send the enhanced model to mobile phone 10.

[0119] In some embodiments, reference Figure 3E In response to the user's selection of the aforementioned image capture option U163, the mobile phone 10 can also display one or more portrait acquisition interfaces to acquire multiple real portrait data (e.g., images and / or videos) of user A with different expressions (e.g., prompting the user to adjust their expression to a smile, frown, etc.), different postures (e.g., prompting the user to turn their head to the left, turn their head to the right, nod, raise their head, etc., open their mouth), and different makeup (e.g., prompting the user to remove / put on glasses, apply makeup, remove makeup, etc.). This increases the diversity of real portraits used to determine user A's augmented model, which is beneficial for improving the detection accuracy of the obtained augmented model of user A.

[0120] It should be noted that the above Figures 3A to 3E The process and interface shown are only an example. In other embodiments, the mobile phone 10 may obtain the real portrait data of the user (user A) of the mobile phone 10 in other ways, and / or obtain the enhanced model of user A based on the real portrait data of user A in other ways, which is not limited here.

[0121] It should be noted that the above Figures 3A to 3E The illustrated method for determining the augmented model of the user of mobile phone 10 is merely an example. When the augmented detection data serves as the model parameters of the second model, mobile phone 10 can also, after acquiring the real portrait data of user A, train the second model based on the real portrait data of user A to obtain the model parameters corresponding to user A, or send the real portrait data of user A to a third electronic device and receive the model parameters corresponding to user A obtained by the third electronic device through training the second model based on the real portrait data of user A. When the augmented detection data consists of real portrait features, mobile phone 10 can also, after acquiring the real portrait data of user A, perform feature extraction on the real portrait data of user A to obtain the real portrait features corresponding to user A, or send the real portrait data of user A to a third electronic device and receive the real portrait features corresponding to user A obtained by the third electronic device based on the real portrait data of user A.

[0122] In some embodiments, after acquiring the real facial image data of user A, mobile phone 10 can also determine user A's facial features and / or user identifier (equivalent to determining user A's user characteristics) based on the real facial image data of user A. Then, mobile phone 10 can store the correspondence between user A's user characteristics and user A's augmented model. The user A's facial features can be determined by mobile phone 10, or by other electronic devices other than mobile phone 10 (e.g., a third electronic device), and this is not limited thereto.

[0123] In some embodiments, the mobile phone 10 can also share the user features of user A with other devices while sharing the enhanced model of user A, so that other electronic devices can store the correspondence between the user features of user A and the enhanced model of user A.

[0124] Below, taking the first electronic device as mobile phone 10 and the enhanced detection data as the enhanced model as an example, we will introduce the scheme for the first electronic device to determine the enhanced detection data of other users besides the user of the first electronic device.

[0125] For example, Figure 4A According to some embodiments of this application, a schematic diagram of an interface for determining enhanced detection data of users other than the user of mobile phone 10 is shown.

[0126] like Figure 4A As shown, in response to the user's selection of the "Add" option U145 in the aforementioned security settings interface, the mobile phone 10 can display the "Select Contacts" interface U21. The "Select Contacts" interface may include options corresponding to some or all contacts in the mobile phone 10's address book and / or its instant messaging application, as well as the "Add" option U211. In response to the user's selection of a contact, the mobile phone 10 can trigger the training of an augmented model for the selected contact.

[0127] It should be noted that in some embodiments, the aforementioned security settings interface U12 may not include the owner enhancement model option U141. The mobile phone 10 can respond to the user's click on the add option U145, displaying the owner's corresponding option and / or other contact's options in the contact selection interface U21. The mobile phone 10 can trigger the aforementioned... Figures 3A to 3E The augmented model shown is for user A on mobile phone 10.

[0128] Continue to refer to Figure 4A Mobile phone 10 can respond to the user's action of clicking on user D's option, displaying user D's augmented model training interface U22. User D's augmented model training interface U22 may include the option to select a sample source U221. The option to select a sample source U221 may include the option to select an image U222, the option to take an image U223, the option to request D to share an image U224, the option to obtain from chat history U225, etc. Based on the user's action in the option to select a sample source U221, mobile phone 10 can obtain one or more real human image data of user D for training user D's augmented training model through different methods.

[0129] For example, in response to a user's tap to select image option U222, mobile phone 10 can obtain the real portrait data of user D selected by user A on mobile phone 10. For details, please refer to the section on mobile phone 10 obtaining real portrait data of user A in response to user operation; it will not be elaborated upon here.

[0130] For example, mobile phone 10 can respond to the user's selection of the image shooting option U223 to capture one or more real portrait data (such as images, videos, etc.) of user D. For details, please refer to the content of mobile phone 10 responding to user operation to capture real portrait data of user A, which will not be repeated here.

[0131] For example, refer to Figure 4A In response to the user's selection of the image sharing option U224, mobile phone 10 can send an image sharing request to user D's mobile phone 30 via an instant messaging application (or other means). Then, refer to... Figure 4B After receiving an image sharing request from mobile phone 10, mobile phone 30 can display message U231 in chat interface U23 to prompt user D of mobile phone 30 that "A requests to share an image for fraud prevention detection". Message U231 may include a denial option U232 and an agreement option U233. In response to user D clicking the agreement option U233, mobile phone 30 can display an image selection interface U24, which includes thumbnails of multiple images and / or videos. Mobile phone 30 can obtain one or more real-image data of user D selected by user D in image interface U24 and send one or more real-image data of user D to mobile phone 10. In some embodiments, the source of multiple images in image selection interface U24 may refer to the source of images / videos in image selection interface U17 displayed by mobile phone 10, which will not be elaborated here.

[0132] In some embodiments, mobile phone 30 may also respond to user D clicking the consent option U233, capture one or more real-life portrait data of user D, and send the captured one or more real-life portrait data to mobile phone 10.

[0133] For example, the phone 10 can respond to user selections. Figure 4A The operation shown in option U225 retrieves chat history, which retrieves chat history between user A and user D in an instant messaging application, or one or more real-life portrait data of user D in chat history between user D and other users.

[0134] It should be noted that in some other embodiments, the mobile phone 10 may also obtain one or more real human image data of user D through other means, which is not limited here.

[0135] After acquiring one or more real-life portrait data of user D, mobile phone 10 can obtain an augmented model of user D based on the acquired real-life portrait data. For example, mobile phone 10 can train an augmented model of user D based on one or more real-life portrait data of user D, or send one or more real-life portrait data of user D to a third electronic device and receive an augmented model of user D obtained by the third electronic device based on one or more real-life portrait data of user D.

[0136] It should be noted that the above Figures 4A to 4B The illustrated method for determining augmented models for users other than the user of mobile phone 10 is merely an example. When the augmented detection data serves as the model parameters of the second model, mobile phone 10 can also, after acquiring the real facial data of user D, train the second model based on the real facial data of user D to obtain the model parameters corresponding to user D, or send the real facial data of user D to a third electronic device and receive the model parameters corresponding to user D obtained by the third electronic device through training the second model based on the real facial data of user D. When the augmented detection data consists of real facial features, mobile phone 10 can also, after acquiring the real facial data of user D, perform feature extraction on the real facial data of user D to obtain the real facial features corresponding to user D, or send the real facial data of user D to a third electronic device and receive the real facial features corresponding to user D obtained by the third electronic device based on the real facial data of user D.

[0137] In some embodiments, after acquiring the real facial image data of user D, mobile phone 10 can also determine user D's facial features and / or user identifier (equivalent to determining user D's user characteristics) based on the real facial image data of user D. Then, mobile phone 10 can store and / or share the correspondence between user D's user characteristics and user D's enhanced model with other users' electronic devices. The user D's facial features can be determined by mobile phone 10 or by other electronic devices (e.g., a third electronic device).

[0138] In some embodiments, mobile phone 10 can send enhanced detection data of user A (and / or enhanced detection data of other users) to other users, and can also receive enhanced detection data of other users sent from other users' devices.

[0139] For example, mobile phone 10 can receive enhanced detection data sent by other electronic devices or send enhanced detection data to other electronic devices through instant messaging applications or other data-sharing applications, or through file sharing / shared power provided by the operating system of mobile phone 10.

[0140] For example, Figure 5According to some embodiments of this application, a schematic diagram of a scenario in which mobile phone 10 and mobile phone 20 share enhanced detection data is shown.

[0141] refer to Figure 5 Mobile phone 20 can respond to user B's action of clicking the add option U311 in the chat interface U31 with user A, displaying a selection box U312. Selection box U312 includes options such as the enhanced detection data option U313, and other data that can be sent to user A. Mobile phone 20 can respond to user B's action of clicking the enhanced detection data option U313, sending user B's enhanced detection data to user A's mobile phone 10.

[0142] Continue to refer to Figure 5 After receiving enhanced detection data of user B sent by mobile phone 20, mobile phone 10 can display message U321 in the chat interface U32 with user A to prompt user A that "user B has shared enhanced detection data". Message U321 may include a cancel option U322 and a save option U323. Mobile phone 10 can store user B's enhanced detection data in response to user A clicking the save option U323. After receiving enhanced detection data of user B, mobile phone 10 can store the correspondence between user B's enhanced detection data and user characteristics in the aforementioned first correspondence relationship. The user characteristics of user B can be determined by mobile phone 10 or sent to mobile phone 10 by mobile phone 20, and are not limited here.

[0143] It should be noted that the aforementioned Figures 3A to 5 The methods shown for training and sharing augmented models of users are merely examples. In other embodiments, augmented detection data of users can also be determined or shared in other ways, which are not limited here.

[0144] For example, an image management application in the first electronic device (such as a gallery application, photo album application, etc., hereinafter using a gallery application as an example) can automatically classify the portrait data stored in the first electronic device. For instance, it can classify portrait data belonging to the same user into one category and group portrait data of the same user into the same directory (hereinafter, the directory containing a user's portraits is referred to as that user's portrait directory). Then, in response to an operation by the user of the first electronic device in a user's portrait directory (or an operation by the user of the first electronic device on that user's portrait directory), the first electronic device can trigger the training of an augmented model for that user based on real portrait data in that user's portrait directory. For example, Figure 6 According to some embodiments of this application, a schematic diagram is shown of an augmentation model that triggers training of users in a gallery application.

[0145] Taking the first electronic device as a mobile phone 10 as an example, refer to Figure 6 The gallery application of mobile phone 10 can include a portrait directory U33 for user B and a portrait directory U34 for user C. In response to a user long-pressing the portrait directory U33, mobile phone 10 can display a menu U35, which includes an enhancement detection data training option U351. Then, in response to a user clicking the enhancement detection data training option U351, mobile phone 10 can display an image selection interface U36, prompting the user to "Please select a real portrait." The image selection interface U36 includes thumbnails of various images / videos in user B's portrait directory. Mobile phone 10 can acquire one or more real portrait data of user B selected by the user in the image interface U36, and train enhancement detection data for user B.

[0146] In some embodiments, the first electronic device may also automatically trigger the training or updating of the augmented model corresponding to the user. For example, the first electronic device may automatically trigger the training or updating of the augmented model for the user when the first electronic device captures a preset number of real images of the user (e.g., 5, 10, 20, or any other arbitrary value). As another example, the first electronic device may also automatically trigger the updating of the augmented detection model for the user when the time elapsed since the last training or updating of the user's augmented detection model reaches a preset duration (e.g., 3 days, 1 week, 1 month, or other duration), or a preset update cycle (e.g., 3 days, 1 week, 1 month, or other cycles).

[0147] The following describes a scenario for detecting whether a human image in multimedia data is a fake image.

[0148] Based on the first correspondence between the M user features and the M enhanced detection data, the first electronic device can detect whether the image in the multimedia data is a forged image by using the enhanced detection data corresponding to the first user after detecting a request to detect the image of the first user in the multimedia data. The following describes a scenario where the first electronic device detects whether the image in the multimedia data is a forged image based on the aforementioned first correspondence.

[0149] For example, taking a mobile phone 10 as the first electronic device, Figures 7A to 7G According to some embodiments of this application, several scenario diagrams are shown for detecting whether a human image in multimedia data is a forged human image.

[0150] In some embodiments, during a video call or video conference, the first electronic device may detect whether the image in the multimedia data of the video call or video conference is a fake image if it detects that the user has triggered an operation to detect fake images.

[0151] For example, refer to Figure 7ATaking user B of mobile phone 20 as an example, during a video call between mobile phone 10 and mobile phone 20, mobile phone 10 can display a detection control U41 on the video call interface. Upon detecting a user clicking the detection control U41, mobile phone 10 can determine the first user feature that matches the user identifier or first facial feature of user B among the aforementioned M user features, based on user B's user identifier (e.g., phone number, nickname, nickname, account, etc. in an instant messaging application) (and / or the first facial feature of the person in the video call). Then, mobile phone 10 can use the enhanced detection data corresponding to the first user feature (e.g., the aforementioned...) Figure 5 The enhanced detection data sent by user B (user B in the video call) is used to detect whether the image of user B in the video call is a fake image.

[0152] for Figure 7A As shown, in some embodiments, the mobile phone 10 may display the detection control U41 if it identifies a security risk in the video call between user A and user B; otherwise, it may not display the detection control U41.

[0153] In some embodiments, during a video call or video conference, the first electronic device may, if configured to automatically detect whether the image in the video call or video conference is a forged image, detect whether the image in the multimedia data of the video call or video conference is a forged image.

[0154] For example, when the aforementioned intelligent anti-fraud detection function is enabled (equivalent to the first electronic device being configured to automatically detect whether the image in a video call or video conference is a fake image), the mobile phone 10 can automatically detect whether the image in the video call or video conference is a fake image during the video call or video conference conducted by the mobile phone 10. In this case, the mobile phone 10 can extract the first facial feature of the image in the video call or video conference, and detect whether the corresponding image is a fake image based on the enhanced detection data corresponding to the user feature that matches the first facial feature among the aforementioned M user features.

[0155] In some embodiments, during a video call or video conference, the first electronic device may detect whether the image of a person in the multimedia data of the video call or video conference is a forged image, should a security risk be detected in the video call or video conference. The security risk may include, but is not limited to, the presence of preset keywords (such as transfer, money transfer, account number, password, privacy, etc.) in the voice or text of the video call (or video conference) that may pose a privacy or property security risk, or the detection of user operations involving privacy and property security (such as transfer operations, opening applications involving property (such as banking or payment applications, etc.).

[0156] For example, refer to Figure 7B During a video call between user A and an unknown number XX, mobile phone 10 can detect if the person in the video call is a fake user B, especially if the voice message includes the phrase "I am B, my phone is broken, please transfer 10000, I need it urgently." If a fake user B is detected, a prompt box U42 is displayed to warn the user, "A suspected fake person has been detected. Please be aware of your privacy and property security." In this scenario, since the phone number is unknown, mobile phone 10 can extract the user's self-address (e.g., "I am B") and / or the address used by user A to the unknown user from the video call voice message to determine a first user identifier, and / or extract the first facial features of the person in the video call. Then, mobile phone 10 can use enhanced detection data matching the first facial features and / or the first user identifier from among the aforementioned M user features to detect whether the corresponding person is a fake.

[0157] In some embodiments, the first electronic device may also detect whether the image in the multimedia data is a fake image when it detects that the user has triggered an operation to detect a fake image in the multimedia data. The multimedia data may include, but is not limited to, images and videos. The multimedia data may be multimedia data stored in the first electronic device, multimedia data sent by other users in an instant messaging application, multimedia data in the chat history of an instant messaging application, or multimedia data stored in the cloud by the user of the first electronic device.

[0158] For example, refer to Figure 7C Mobile phone 10 can display video U43 sent by user B to user A. After detecting a user's long press on video U43, mobile phone 10 can display detection control U431 (or display it automatically). Then, if mobile phone 10 detects a user's click on detection control U431 (equivalent to detecting a user triggering an operation to detect a fake image in multimedia data), mobile phone 10 can detect whether the image of user B in video U43 is a fake image.

[0159] for Figure 7C In some embodiments, as shown, mobile phone 10 can determine the first user identifier and / or the facial features of the person in video U43 by using the nickname, notes, phone number, name, etc. of user B in the chat session between user A and user B. Then, based on the enhanced detection data corresponding to the user features that match the first facial features and / or the first user identifier among the above M user features, it can detect whether the person in video U43 is a fake person.

[0160] for Figure 7C In some embodiments, as shown, mobile phone 10 can also use the enhanced detection model corresponding to the user selected by user A to detect whether the portrait in video U43 is a fake portrait.

[0161] For example, refer to Figure 7D After detecting that a user clicks on the detection control U431, the mobile phone 10 can display a user selection box U44. The user selection box U44 can include options corresponding to some or all of the users corresponding to the aforementioned M user features (e.g., options corresponding to users B and C). Then, the mobile phone 10 can obtain the user identifier of at least one user selected by the user in the selection box U44, and based on the enhanced detection model corresponding to each of these user identifiers, detect whether the corresponding human image is a fake image.

[0162] For example, refer to Figure 7E After detecting that a user clicks on the detection control U431, the mobile phone 10 can first identify the user corresponding to the image in the video U43 (e.g., user B and user C) and display the user selection box U45 to prompt the user, "The following user images have been detected. Please select a user for anti-fraud detection." The user selection box U45 includes options for the user corresponding to the image in the video U43 (e.g., options for user B and user C) and other user options U451. Then, the mobile phone 10 can obtain the user selected by the user from user B, user C (and / or other users added by the user through other options U451), and detect whether the image in the video U43 is a fake image based on the enhanced detection data corresponding to the user selected by user A.

[0163] In some embodiments, the first electronic device may also detect whether the portrait in the multimedia data of the chat session content is a fake portrait if it detects that the user of the first electronic device has a security risk in the chat session.

[0164] For example, refer to Figure 7FDuring a chat between user A and user B, if mobile phone 10 detects message U46 sent by user B, "Quickly transfer 10,000 to me, urgent," it can determine that user A may pose a security risk in the chat session. It then checks whether the image in the multimedia data of video U47 in the historical chat history between user A and user B is a fake. If a fake image is detected in video U47, mobile phone 10 can display a prompt box U48 to warn, "The video sent by user B is suspected of containing a fake image; please be aware of your financial security."

[0165] In some embodiments, the first electronic device may also detect whether the image in the multimedia data in the detection request is a fake image when it receives a detection request from another electronic device (e.g., a fourth electronic device).

[0166] For example, taking the fourth electronic device as a watch 40 as an example, refer to Figure 7G During a video call with user B, watch 40 can send a detection request to mobile phone 10. This detection request may include multimedia data to be detected (such as user B's video or image). Optionally, the detection request may also include user characteristics of user B in the multimedia data (such as user B's user identifier or user facial features), and / or enhanced detection data corresponding to user B.

[0167] Continue to refer to Figure 7G After receiving the detection request, mobile phone 10 can detect whether the image of user B in the multimedia data is a fake image based on the enhanced detection data corresponding to the user features that match the user features of user B among the aforementioned M user features, and send the detection result to watch 40. The detection result is used to indicate whether the image of user B in the multimedia data is a fake image.

[0168] for Figure 7G As shown, in some embodiments, when the detection request includes enhanced detection data, the mobile phone 10 can detect whether the image of user B in the multimedia data is a fake image based on the enhanced detection data in the detection request.

[0169] based on Figure 7G As shown, the first electronic device can provide a detection capability for electronic devices with poor computing power (such as smartwatches, smart bracelets, smart glasses and other wearable devices, smart door locks and other smart home devices) to detect whether the human image in the multimedia data is a real human image, so as to improve the security of the electronic devices with poor computing power to carry out related business (such as money transfer, transmission of private information, unlocking door locks, etc.) based on the detection results.

[0170] In some embodiments, the first electronic device may also be provided with a configuration entry for configuring whether to automatically detect whether the portraits in the multimedia data of the application are fake portraits. When the first electronic device is configured to automatically detect whether the portraits in the multimedia data of the first application are fake portraits after the first application is launched, the first electronic device may automatically detect whether the portraits in the multimedia data of the first application are fake portraits during the operation of the first application.

[0171] It should be noted that the above Figures 7A to 7G The scenario shown is merely an example. In other embodiments, the method for human face detection provided in this application is also applicable to any other scenario where it is necessary to detect whether a human face in multimedia data such as images and videos is a forged human face. This is not a limitation.

[0172] The following is combined Figures 4A to 7G The situation shown and Figures 8 to 11 This paper introduces the process of a method for human face detection.

[0173] First, we will introduce the process of determining the enhanced detection data for users.

[0174] For example, Figure 8 According to some embodiments of this application, an interactive flowchart of a method for determining enhanced detection data for a user is shown. Figure 8 As shown, the method includes:

[0175] S801, the first electronic device detects an instruction to determine enhanced detection data for a third user.

[0176] For example, after detecting an instruction to train on enhanced detection data for a third user, the first electronic device can trigger the method provided in this application embodiment. The third user can be a user of the first electronic device, or a user other than a user of the first electronic device; this is not limited here.

[0177] For example, for Figure 3A In the scenario shown, the first electronic device can be mobile phone 10, and the third user can be user A of mobile phone 10. When mobile phone 10 detects that the user clicks on the training option U142, it can detect the instruction to train the enhanced detection data for the third user (user A).

[0178] For example, regarding Figure 4A In the scenario shown, the first electronic device can be a mobile phone 10, and the third user can be user D. When the mobile phone 10 detects that the user clicks on the option of user D, it can detect the instruction to train the enhanced detection data for the third user (user D).

[0179] For example, the first electronic device may also detect the instruction to determine the augmented detection model of the third user when the number of real human images of the third user captured reaches a preset number, or when the time between the last training or update of the augmented detection model of the third user reaches a preset duration, or when a preset update cycle is reached.

[0180] It should be noted that the first electronic device may also detect the instruction to train on enhanced detection data for a third user in other situations, and this application embodiment does not limit this. For example, the first electronic device may detect the instruction to train on enhanced detection data for a third user when it receives a training request sent by another electronic device (e.g., a fourth electronic device).

[0181] S802, the first electronic device sends a training request to the third electronic device.

[0182] For example, when the first electronic device detects an instruction to train on enhanced detection data for a third user, it may acquire one or more real human image data of the third user and send a training request including the one or more real human image data to the third electronic device.

[0183] In some embodiments, the training request may not include the aforementioned one or more real facial images of the third user, but the real facial images of the third user may be acquired or determined by a third electronic device, which is not limited here.

[0184] In some embodiments, the training request may also include the user identifier of a third user.

[0185] S803, the third electronic device generates and sends enhanced detection data of the third user to the first electronic device.

[0186] For example, in response to a training request, a third electronic device can generate augmented detection data for a third user based on one or more real-life facial images of the third user. The third electronic device can then send the augmented detection data of the third user to a first electronic device.

[0187] In some embodiments, the third electronic device may also send the third user's facial features and / or the third user's user identifier to the first electronic device. For example, the third user's facial features may be obtained by the third electronic device based on one or more real-life portrait data of the third user.

[0188] It should be noted that S802 and S803 are optional operations. In some other embodiments, the enhanced detection data of the third user can also be obtained by the first electronic device based on one or more real human image data of the third user, which is not limited here.

[0189] S804, the first electronic device updates the enhanced detection data of the third user into the first correspondence.

[0190] For example, after receiving enhanced detection data from a third user, the first electronic device can update the correspondence between the enhanced detection data of the third user and the user characteristics of the third user into the first correspondence.

[0191] In some embodiments, the user characteristics of a third user may also be generated by a first electronic device.

[0192] In step S805, the first electronic device sends enhanced detection data of the third user to other electronic devices. Correspondingly, the other electronic devices receive the enhanced detection data of the third user.

[0193] For example, after receiving enhanced detection data from a third user, the first electronic device may also send the enhanced detection data from the third user to other electronic devices (e.g., other electronic devices besides the first and third electronic devices). For example, the other electronic devices may be one or more electronic devices selected by the user of the first electronic device, or they may be electronic devices in a first group.

[0194] In some embodiments, the first electronic device may also send user characteristics of a third user, such as the user identifier and / or the user facial features of the third user, to other electronic devices.

[0195] In some embodiments, other electronic devices may include, but are not limited to, electronic devices belonging to users who have a close relationship with the user of the first electronic device (e.g., family members, colleagues, relatives, friends, classmates, etc. of the first electronic device), or other electronic devices belonging to the user of the first electronic device.

[0196] In some embodiments, the first electronic device may send enhanced detection data of a third user (or enhanced detection data of other users besides the third user stored in the first electronic device) to another electronic device, or simultaneously send enhanced detection data of a third user (or enhanced detection data of other users besides the third user stored in the first electronic device) to multiple electronic devices. For example, the one electronic device may be any electronic device capable of establishing a communication connection with the first electronic device and receiving data. As another example, the multiple electronic devices may be electronic devices in the aforementioned first group or electronic devices of users in the first group.

[0197] In some embodiments, the first electronic device may also send enhanced detection data of other users stored in the first electronic device to other electronic devices, which is not limited thereto.

[0198] It should be noted that S805 is an optional operation. That is, the first electronic device may not send the enhanced detection data of the third user to other electronic devices.

[0199] S806, the first electronic device receives enhanced detection data sent by other electronic devices and updates the first correspondence based on the received enhanced detection data.

[0200] For example, other electronic devices may send additional enhanced detection data to the first electronic device. Accordingly, the first electronic device may receive the enhanced detection data sent by the other electronic devices and update the correspondence between the received enhanced detection data and the corresponding user features into the first correspondence.

[0201] For example, for Figure 5 In the scenario shown, the first electronic device can be mobile phone 10, and the other electronic devices can be mobile phone 20. After receiving the enhanced detection data of user B, mobile phone 10 can update the correspondence between the enhanced detection data of user B and the user characteristics of user B into the first correspondence.

[0202] It should be noted that the execution order between S806 and the aforementioned S801 to S805 can be arbitrarily adjusted. For example, S806 can be executed before S801, or during the execution of S801 to S805, etc., without any restrictions.

[0203] It should be noted that S806 is an optional operation. That is, the first electronic device may not receive enhanced detection data sent by other electronic devices.

[0204] Based on the above method, the first electronic device can acquire enhanced detection data of different users and update the correspondence between the enhanced detection data of users and the corresponding user features into the first correspondence.

[0205] The following describes a method for detecting a user's image in multimedia data based on a first correspondence and enhanced detection data of the user.

[0206] For example, Figure 9A According to some embodiments of this application, a schematic diagram of a method for human face detection is shown. The method is executed by a first electronic device, such as... Figure 9A As shown, the method includes:

[0207] S901, an instruction to inspect multimedia data was detected, wherein the multimedia data includes the image of the first user.

[0208] For example, when the first electronic device detects an instruction to detect multimedia data, it can trigger S902 or S903 to detect whether the image of the first user in the multimedia is a fake image.

[0209] In some embodiments, the first electronic device may detect an instruction to detect multimedia data under any of the following conditions C1 to C7:

[0210] C1, during a video call or video conference conducted by the first electronic device, an operation triggering a fake person detection function is detected, wherein the multimedia data is an image or video from the video call or video conference. For example, the first electronic device may display detection controls (such as those described above) on the video call or video conference interface. Figure 7A The first electronic device can detect the instruction to detect the image of the first user in the multimedia data when it detects that the user clicks the detection control (U41).

[0211] C2, the first electronic device is detected to be in the process of a video call or video conference, and the first electronic device is configured to automatically detect whether the image in the video call or video conference is a fake image (e.g., the aforementioned intelligent anti-fraud detection function is enabled), wherein the multimedia data is the image or video in the video call or video conference.

[0212] C3, A security risk is detected during a video call or video conference conducted by the first electronic device, where the multimedia data consists of images or videos from the video call or video conference. For example, if the first electronic device contains keywords related to privacy and property security in the content of the video call or video conference, or if user actions involving privacy and property security are detected, a security risk is determined to exist during the video call or video conference conducted by the first electronic device, thereby detecting an instruction to perform image detection on the first user's face in the multimedia data. For example, refer to... Figure 7B The first electronic device (mobile phone 10) can detect the instruction to detect the image of the first user (user B) in the multimedia data when it detects the voice message "I am B, my phone is broken, please transfer 10,000 to me, I need it urgently".

[0213] C4, a user-triggered operation to detect fake human figures in multimedia data was detected. For example, see reference. Figure 7C The first electronic device (mobile phone 10) can detect an instruction to detect multimedia data (video U43) when it detects that the user has long-pressed the video U43 and then clicked the detection control U431. For example, see reference... Figure 7DThe first electronic device can detect an instruction to detect the image of the first user (user B) in the multimedia data (video U43) when it detects that the user clicks on the option of user B.

[0214] C5, The system detects a security risk in a chat session involving a user on a first electronic device, where the chat session content includes multimedia data. For example, the first electronic device might determine a security risk by detecting keywords related to privacy and financial security (such as transfer, money transfer, account number, password, privacy, etc.) in the text, audio, or video of the chat session, or by detecting user actions involving privacy and financial security. This triggers an instruction to detect the user's image within the multimedia data. For example, for... Figure 7F In the scenario shown, the first electronic device (mobile phone 10) can detect the message "Quickly transfer 10,000 to me, I need it urgently" in message U46 and determine that there is a security risk in the user's chat session, thereby detecting the instruction to detect the image of the first user (user B) in the multimedia data.

[0215] C6 receives a detection request from a fourth electronic device, the detection request including the aforementioned multimedia data.

[0216] It should be noted that in other embodiments, the first electronic device may also detect the instruction to detect the image of the first user in the multimedia data in other situations, which is not limited here.

[0217] C7, the first application is detected to be running. The first electronic device is configured to automatically detect whether the user's image in the multimedia data of the first application is a real image when the first application is running.

[0218] It should be noted that the above situations C1 to C7 are only examples. In other embodiments, the first electronic device may also detect the instruction to detect multimedia data in other situations, which is not limited here.

[0219] In some embodiments, the multimedia data may be multimedia data in the first electronic device or multimedia data sent to the first electronic device by other electronic devices, without limitation.

[0220] S902, if there is a first enhanced detection data matching the first user among the M enhanced detection data, detect whether the portrait of the first user in the multimedia data is a fake portrait based on the first enhanced detection data, wherein the M enhanced detection data are obtained based on one or more real portrait data of the corresponding user, and M is an integer greater than or equal to 1.

[0221] For example, in some embodiments, the first electronic device may first determine a first user identifier of the first user and / or extract facial features of the first user's image from the multimedia data to obtain a first facial feature. Then, the first electronic device may determine whether any of the M user features corresponding one-to-one with the M enhanced detection data contain a user feature that matches the first user identifier and / or the first facial feature. If a first user feature matching the first user identifier and / or the first facial feature exists among the M user features (equivalent to first enhanced detection data matching the first user exists among the M enhanced detection data), the first electronic device may detect whether the first user's image is a forged image based on the first enhanced detection data corresponding to the first user feature, obtaining a detection result indicating whether the first user's image in the multimedia data is a forged image. The detection result may indicate whether the first user's image is a forged image, or indicate the probability that the first user's image is a forged image, or indicate the probability that the first user's image is a real image.

[0222] In some embodiments, the first electronic device may determine the first user's first user identifier based on any one of the following methods D1 to D4:

[0223] D1, when the multimedia data is images or videos from a video call or video conference, the first user identifier is determined based on one or more of the first user's title, nickname, remarks, phone number, and name in the video call or video conference. The first user's title can be how the first user refers to themselves in the video call or video conference, or how other users (e.g., users of the first electronic device) refer to the first user; this is not limited here.

[0224] D2, when the multimedia data is the multimedia data in a chat session between the user of the first electronic device and the first user, determine the first user identifier based on one or more of the first user's title, nickname, remarks, phone number, and name in the chat session. The first user's title can be the way the first user refers to themselves in the chat content, or the way other members in the chat session refer to the first user; this is not limited here.

[0225] D3. Obtain the first user identifier selected or input by the user of the first electronic device. For example, when the user of the first electronic device triggers the detection of multimedia data, they can select or input the user identifier (such as name, nickname, phone number, account, etc.) corresponding to the image in the multimedia data. The first electronic device can use the user identifier selected or input by the user of the first electronic device as the first user identifier.

[0226] D4. Obtain the user identifier of the first user from the instruction to detect multimedia data. For example, the instruction to detect multimedia data can be an instruction sent by another electronic device, which may include the user identifier of the first user corresponding to the multimedia data.

[0227] In some embodiments, since a user's facial features can typically uniquely identify a user, the first electronic device can determine that the first enhanced detection data corresponding to the first user feature matches the first user only if the first user's first facial feature matches the user's facial features in the first user features (e.g., if the similarity is greater than a threshold). This avoids inaccurate detection results caused by incorrectly matched enhanced detection data.

[0228] In some embodiments, the first electronic device may determine that the first user identifier and the user identifier in the first user feature match if the first similarity between the first user identifier and the user identifier in the first user feature is greater than a first similarity threshold, and / or, determine that the first user identifier and the user identifier in the first user feature match if the second similarity between the first face feature and the user face feature in the first user feature is greater than a second similarity threshold.

[0229] In some embodiments, the first electronic device may also determine, upon detecting that a user has selected the aforementioned first enhanced detection data from among the M enhanced detection data, that there is enhanced detection data matching the first user among the M enhanced detection data. For example, for Figure 7D In the scenario shown, mobile phone 10 (as an example of a first electronic device) can determine first enhanced detection data based on enhanced detection data (e.g., enhanced model) corresponding to user B when it detects that the user clicks on the option of user B (as a first user), and detect whether the image of user B in multimedia data (e.g., video U43) is a fake image based on the first enhanced detection data.

[0230] If it is determined that there is a first enhanced detection data that matches the first user among the M enhanced detection data, the first electronic device can detect whether the first user's portrait is a fake portrait based on the first enhanced detection data corresponding to the first user's features.

[0231] For example, the enhanced detection data can be an enhanced model, model parameters, or real human features. When the M enhanced detection data are M enhanced models, the first enhanced detection data can be the first model among the M enhanced models, and the first electronic device can determine whether the image of the first user in the multimedia data is a fake image by calling the first model. When the M enhanced detection data are M sets of model parameters of a second model, and the first enhanced detection data can be the first model parameter among the M sets of model parameters, the first electronic device can determine whether the image of the first user in the multimedia data is a fake image based on the second model having the first model parameters. When the M enhanced detection data are M real human features, and the first enhanced detection data can be the first real human feature among the M real human features, the first electronic device can determine whether the image of the first user in the multimedia data is a fake image by calling the third model and based on the first real human feature.

[0232] In some embodiments, the first enhanced detection data may be generated by the first electronic device based on one or more real facial images of the first user; it may also be received by the first electronic device from other electronic devices (e.g., a second electronic device or a third electronic device); or it may be determined by the first electronic device sending one or more real facial images of the first user to a third electronic device (the third electronic device may be the same as or different from the second electronic device) and being determined by the third electronic device based on one or more real facial images of the first user. For details, please refer to the foregoing. Figures 3A to 5 The specific implementation examples are not described in detail here.

[0233] It should be noted that the first user can be one of the aforementioned... Figure 8 The aforementioned third user could be the user corresponding to the enhanced detection data sent by other electronic devices in S806, or the user corresponding to the enhanced detection data previously stored by the first electronic device; no limitation is made here.

[0234] It should be noted that if there is no enhanced detection data matching the first user among the M enhanced detection data, the first electronic device can obtain the detection result of whether the image of the first user in the multimedia data is a fake image based on the general detection data.

[0235] S903, if there is no enhanced detection data matching the first user among the M enhanced detection data, detect whether the image of the first user in the multimedia data is a fake image based on the general detection data.

[0236] For example, if there is no user feature among the M user features of the first correspondence that matches the first user identifier and / or the first facial feature (equivalent to no enhanced detection data matching the first user among the M enhanced detection data), the first electronic device can detect whether the image of the first user in the multimedia data is a fake image based on general detection data.

[0237] In some embodiments, after obtaining the detection result of whether the image of the first user in the multimedia data is a forged image (e.g., after executing S902 or S903), the first electronic device can output the detection result through one or more of the following methods: text, image, audio, vibration, video, and animation. For example, if the image of the first user in the multimedia data is a forged image, the first electronic device can output a prompt message through one or more of the following methods: text, image, audio, vibration, video, and animation, to remind the user to pay attention to their privacy, personal safety, and property security.

[0238] In some embodiments, the first electronic device may also send the detection results to other electronic devices. For example, if the instruction to detect multimedia data comes from a detection request sent by a fourth electronic device, the first electronic device may send the detection results to the fourth electronic device.

[0239] In some embodiments, the first electronic device may also update the first enhanced detection data when the update conditions corresponding to the first user are met. Thus, through one or more updates, the accuracy of the first enhanced detection data in detecting whether the first user's image is a forged image can be further improved.

[0240] For example, the update conditions corresponding to the first user may include at least one of the following conditions: detecting that the first electronic device has captured at least one real human image data of the first user; detecting that the user (e.g., the user of the first electronic device or other users) updates the enhanced detection data of the first user; and reaching a preset update cycle.

[0241] For example, after detecting that a user has captured at least one real human image data of a first user through the first electronic device: if the first enhanced detection data is a first model, the first electronic device can retrain the first model based on the at least one real human image data to obtain an updated first model; if the first enhanced detection data is the first model parameter of a second model, the first electronic device can retrain the second model based on the at least one real human image data to obtain an updated first model parameter; if the first enhanced detection data is a first real human image feature, the first electronic device can update the first real human image feature based on the at least one real human image data.

[0242] For example, when the first electronic device detects that the user has updated the first enhanced detection data, or when a preset update cycle has been reached, the first electronic device can reacquire one or more real human image data of the first user and update the first enhanced detection data based on the acquired real human image data.

[0243] In some embodiments, the first electronic device may also, in response to a user's sharing operation, send one or more of the aforementioned M enhanced detection data to the fifth electronic device or the first group. Thus, the fifth electronic device or the electronic device corresponding to the first group can detect whether the corresponding user's image is a forged image based on the one or more enhanced detection data.

[0244] For example, for Figure 3C In the scenario shown, mobile phone 10 (an example of the first electronic device) can share user A's augmentation model to the fifth electronic device in response to the user clicking the share option U201 (equivalent to a share operation). Alternatively, the first electronic device may be in the same group as one or more other electronic devices (e.g., a first group). The first electronic device can share one or more of M augmentation detection data to the first group, allowing the electronic devices in the first group to receive and store the one or more augmentation detection data. Furthermore, the first group may include multiple users, including users within the first electronic device. The first electronic device can share one or more augmentation detection data to the first group, and the electronic devices of the multiple users can obtain the one or more augmentation detection data from the first group.

[0245] In some embodiments, the multimedia data may include portraits of multiple users. For any one user's portrait, the first electronic device can detect whether the portrait of that user is a real portrait by matching the enhanced detection data of that user among M enhanced detection data.

[0246] For example, the multimedia data in S901 may also include the image of a second user. If there is second enhanced detection data matching the second user among the M enhanced detection data, the first electronic device can detect whether the image of the second user in the multimedia data is a fake image based on the second enhanced detection data, wherein the second enhanced detection data is obtained based on one or more real image data of the second user; if there is no enhanced detection data matching the second user among the M enhanced detection data, the device can detect whether the image of the second user in the multimedia data is a fake image based on general detection data.

[0247] For example, when the multimedia data includes images of user A, user B, and user C, and among the M enhanced detection data, there exists a first enhanced detection data matching user A, a second enhanced detection data matching user B, and no enhanced detection data matching user C, the system detects whether user A's image is a forged image based on the first enhanced detection data, whether user B's image is a forged image based on the second enhanced detection data, and whether user C's image is a forged image based on the general detection data.

[0248] based on Figure 9A In the illustrated embodiment, the first electronic device can detect whether the image of the first user in the multimedia data is a fake image by using first enhanced detection data obtained based on one or more real image data of the first user. This can improve the accuracy of the detection results, reduce the possibility of users being misled by virtual content transmitted by fake images, and enhance the user's privacy and property security.

[0249] It should be noted that the first electronic device can be a terminal device or a server, and there is no limitation here.

[0250] The following is an example of a first electronic device working in conjunction with other electronic devices (a fourth electronic device) to detect whether a human image in multimedia data is a real human image.

[0251] For example, Figure 9B According to some embodiments of this application, a schematic diagram of the interactive flow of a method for human face detection is shown. For example... Figure 9B As shown, the method includes:

[0252] S901A, the fourth electronic device sends a detection request to the first electronic device, the detection request being used to request detection of whether the image of the first user in the multimedia data is a real image. Accordingly, the first electronic device receives the detection request from the fourth electronic device.

[0253] For example, a fourth electronic device (such as a terminal device with poor computing power, such as a smartwatch, smart bracelet, smart door lock, smart camera, etc.) can send a detection request to the first electronic device to detect whether the image of the first user in the multimedia data is a fake image.

[0254] In some embodiments, the detection request may include multimedia data, or the storage path of the multimedia data (e.g., storage path in the cloud or NAS, URL, etc.).

[0255] In some embodiments, the detection request may also include user characteristics of the first user, such as the first user identifier corresponding to the first user and / or the user facial features of the first user.

[0256] In some embodiments, the detection request may also include an enhanced detection model corresponding to the first user.

[0257] For example, for Figure 7G As shown, watch 40 (as an example of a fourth electronic device) can send a detection request to mobile phone 10 (as an example of a first electronic device) to detect the video (multimedia data) of user B during a video call with user B.

[0258] S902A, when there is a first enhanced detection data matching the first user among M enhanced detection data, the first electronic device obtains a detection result on the first enhanced detection data as to whether the image of the first user in the multimedia data is a fake image.

[0259] For example, after receiving a detection request, the first electronic device can determine the first facial feature of the portrait in the multimedia data or the first user identifier of the first user. Then, the first electronic device can determine whether there is a user feature among the M user features that matches the first facial feature and / or the first user identifier. Finally, if there is a first user feature among the M user features that matches the first facial feature and / or the first user identifier, the first electronic device can obtain a detection result of whether the portrait of the first user in the multimedia data is a forged portrait based on the first enhanced detection data corresponding to the first user feature. The method by which the first electronic device determines the first enhanced detection data and determines whether the portrait of the first user in the multimedia data is a forged portrait based on the first enhanced detection data can refer to the content of S902 above, and will not be repeated here.

[0260] S903A, when there is no enhanced detection data matching the first user among the M enhanced detection data, the first electronic device obtains the detection result of whether the image of the first user in the multimedia data is a fake image based on the general detection data.

[0261] S904, the first electronic device outputs the detection result.

[0262] For example, after obtaining the detection result of whether the image of the first user in the multimedia data is a forged image (e.g., after executing S902A or S903A), the first electronic device can output the detection result. For example, the first electronic device can output the detection result itself through audio, vibration, images, text, video, animation, etc. For example, if the detection result indicates that the image of the first user in the multimedia data is a forged image, the first electronic device can output a prompt message through audio, vibration, images, text, video, animation, etc., to remind the user to pay attention to their personal safety, privacy, and property security.

[0263] In some embodiments, if the probability of the first user's image in the multimedia data being a forged image is low, or if the first user's image is not a forged image, the first electronic device may not output the detection result, and this is not limited here.

[0264] S905, the first electronic device sends the detection result to the fourth electronic device. Accordingly, the fourth electronic device receives the detection result.

[0265] For example, after obtaining the detection result of whether the image of the first user in the multimedia data is a fake image (e.g., after executing S902A or S903A), the first electronic device can send the detection result to the fourth electronic device.

[0266] S906, the fourth electronic device outputs a prompt message.

[0267] After receiving the detection results, the fourth electronic device can output prompts through text, images, animations, videos, vibrations, audio, and other means. For example, if the detection results indicate that the first user's image in the multimedia data is a fake image, the fourth electronic device can output prompts through audio, vibration, images, text, videos, animations, and other means to remind the user to pay attention to their personal safety, privacy, and property security.

[0268] In some embodiments, if the detection result indicates that the probability of the first user's image in the multimedia data being a forged image is low or that the first user's image is not a forged image, the first electronic device may not output a prompt message, and this is not limited here.

[0269] based on Figure 9B The illustrated embodiment enables users of fourth electronic devices with lower computing power to obtain detection results on whether the portrait in multimedia data is a forged portrait, which helps protect the privacy and property security of users of fourth electronic devices.

[0270] This application also provides a method for human face detection applied to a second electronic device.

[0271] For example, Figure 10 The illustration shows a schematic diagram of a method for a second electronic device to determine enhanced detection data, according to some embodiments of this application. The method is executed by a second electronic device, such as... Figure 10 As shown, the method includes:

[0272] S1001, An instruction to determine the first enhanced detection data of the first user has been detected.

[0273] For example, when the second electronic device detects an operation that determines the first enhanced detection data of the first user, it can determine the first enhanced detection data of the first user based on the following S1002 and S1003.

[0274] For example, in the case where the second electronic device is the aforementioned user B's mobile phone 20, and the first user is user B: (Refer to...) Figure 11 In response to the user clicking the training option 51 or the update option 52, mobile phone 20 can detect and determine the augmented detection data of user B. In this case, the process by which mobile phone 20 determines the augmented detection data of user B can be referred to the process by which mobile phone 10 determines the augmented detection data of user A, and will not be elaborated here.

[0275] For example, when the second electronic device is the aforementioned mobile phone 10 and the first user is user A, the mobile phone 10 can detect the instruction to determine the enhanced detection data of user A when it detects that the user clicks the training option U142.

[0276] For example, when the second electronic device is the aforementioned mobile phone 10 and the first user is user D, the mobile phone 10 can detect and determine the enhanced detection data of user D when it detects that the user clicks on the option of user D in the aforementioned contact selection interface U21.

[0277] For example, the second electronic device may also detect an instruction to determine the first enhanced detection data of the first user in other situations, such as when the second electronic device detects that it has captured one or more real human image data of the first user, or when a preset update cycle is reached, and determine the first enhanced detection data of the first user based on the following S1002 and S1003.

[0278] It should be noted that in other embodiments, the second electronic device may also detect the instruction to determine the first enhanced detection data of the first user under other circumstances, and this application embodiment does not limit this.

[0279] In some embodiments, the first user may be a user of the second electronic device, or may not be a user of the second electronic device; this is not limited here.

[0280] S1002, Obtain one or more real human portrait data of the first user.

[0281] For example, after detecting an instruction to determine the first enhanced detection data of the first user, the second electronic device can acquire one or more real human image data of the first user.

[0282] For example, the second electronic device can obtain the real facial image data of the first user stored in the second electronic device, and / or obtain the real facial image data of the first user from chat logs of an instant messaging application, shared files, and / or obtain the real facial image data of the first user from other electronic devices (such as a fifth electronic device), and / or obtain the real facial image data of the first user captured by the second electronic device. Specific examples of the first electronic device obtaining the user's real facial image data can be found in the embodiments described above, which will not be elaborated upon here.

[0283] In some embodiments, the fifth electronic device may be a device with the same instant messaging application installed as the second electronic device, a server for storing images, a home storage device for storing images or videos, or any other electronic device that has established a communication connection or a trust relationship with the second electronic device, without limitation.

[0284] S1003, determine the first enhanced detection data based on one or more real human portrait data.

[0285] After acquiring one or more real-life portrait data of the first user, the second electronic device can determine the first enhanced detection data based on the one or more real-life portrait data.

[0286] For example, the second electronic device can train a portrait detection model based on the one or more real portrait data to obtain a first model (such as the aforementioned augmented model) corresponding to the first user. Alternatively, the second electronic device can train the aforementioned second model based on the one or more real portrait data to obtain the parameters of the first model corresponding to the first user. Yet another example is that the second electronic device can extract portrait features from the one or more real portrait data to obtain the first real portrait features corresponding to the first user.

[0287] It should be noted that the second electronic device can generate the first enhanced detection data itself based on one or more real human image data of the first user, or it can send one or more real human image data of the first user to other electronic devices, and let other electronic devices determine the first enhanced detection data. No limitation is made here.

[0288] S1004, First enhanced detection data is sent to the first electronic device, wherein the first enhanced detection data is used to determine whether the image of the first user in the multimedia data is a fake image.

[0289] After determining the first enhanced detection data, the second electronic device can send the first enhanced detection data to the first electronic device. The first enhanced detection data can be used to detect whether the image of the first user in the multimedia data is a fake image.

[0290] In some embodiments, the second electronic device may further extract the facial features of the first user from one or more real-life image data, obtain the user's facial features corresponding to the first enhanced detection data, and send the user's facial features to the first electronic device.

[0291] In some embodiments, the second electronic device may further determine the user identifier of the first user and send the user identifier of the first user to the first electronic device.

[0292] In some embodiments, the second electronic device may also send the first enhanced detection data to other electronic devices besides the first electronic device, or the first enhanced detection data and the user characteristics of the first user (e.g., the user's facial features and / or user identifier).

[0293] It should be noted that after receiving the first enhanced detection data, the first electronic device can update the correspondence between the first enhanced detection data and the first user features into the first correspondence, and detect whether the first user's portrait is a real portrait based on the first enhanced detection data.

[0294] based on Figure 10 In the illustrated embodiment, the second electronic device can determine and send the first enhanced detection data corresponding to the first electronic device to the first electronic device. This can improve the accuracy of the first electronic device in detecting whether the image of the first user is a real image, thereby reducing the possibility that the user of the first electronic device may mistakenly believe the virtual content transmitted by the fake image of the first user, and protecting the privacy and property security of the user of the first electronic device.

[0295] The following describes a software architecture for implementing the face detection method provided in the embodiments of this application.

[0296] For example, Figure 12 According to some embodiments of this application, a schematic diagram of a software architecture is shown. The aforementioned first electronic device may employ... Figure 12 The software architecture in it.

[0297] like Figure 12 As shown, this software architecture may include an application layer, a framework layer (and / or a service layer). The application layer may include applications in an electronic device, such as a primary application. The framework layer provides application programming interfaces (APIs) to the applications in the application layer.

[0298] The first application can send multimedia data (such as video files, image files, video streams during video calls / video conferences, etc.) from the first application to the anti-fraud detection service and obtain the detection results obtained by the anti-fraud detection service. For example, if any of the aforementioned C1 to C7 conditions are met, the first application can send multimedia data to the anti-fraud detection service to instruct the anti-fraud detection service to detect whether the image in the multimedia data is a fake image.

[0299] Fraud detection services can be set up in the framework layer or (or service layer).

[0300] The anti-fraud detection service can be used to detect whether the portrait in the multimedia data sent by the first application is a fake portrait based on the aforementioned first correspondence, obtain the detection result, and send the detection result to the first application.

[0301] Fraud detection services can also be used to obtain users' real facial image data and determine the corresponding enhanced detection data based on the user's real facial image data.

[0302] In some embodiments, the application layer application (e.g., the first application) may also integrate the functions of the aforementioned anti-fraud detection service. For example, the first application may acquire the user's real facial image data, determine the user's enhanced detection data based on the user's real facial image data, and detect whether the image in the multimedia data is a fake image based on the aforementioned first correspondence.

[0303] It should be noted that the first application, the method by which the anti-fraud detection service obtains the user's real facial image data, determines the user's enhanced detection data based on the user's real facial image data, and detects whether the user's image is a forged image based on the first correspondence relationship, can refer to the aforementioned method. Figures 2 to 11 The specific implementation examples are not described in detail here.

[0304] It should be noted that, Figure 12 The software architecture shown is only one example; in other embodiments, the first electronic device may adopt a similar architecture. Figure 12 The software architecture shown may be the same, similar, or different, and no limitation is made here.

[0305] This application also provides a computer program product, which may be a software or program product including instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, the at least one computing device implements the method for human face detection provided in this application.

[0306] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any storage medium (e.g., magnetic medium, optical medium, semiconductor medium, etc.) capable of storing and / or retrieving data by a computing device. The computer-readable storage medium stores instructions that direct the computing device to implement the face detection method provided in this application.

[0307] This application also provides a first electronic device, which includes at least one processor, memory, and non-volatile memory. The at least one processor is used to execute instructions to implement the method for human face detection performed by the first electronic device in the foregoing embodiments.

[0308] This application also provides a second electronic device, which includes at least one processor, memory, and non-volatile memory. The at least one processor is used to execute instructions to implement the method for human face detection performed by the second electronic device in the foregoing embodiments.

[0309] This application also provides a system, which includes the aforementioned first electronic device and second electronic device.

[0310] For example, Figure 13 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown. The aforementioned first electronic device, second electronic device, mobile phone 10, mobile phone 20, mobile phone 30, and watch 40 may adopt the same or similar structure as electronic device 100.

[0311] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0312] Processor 110 may include one or more processing units, such as a central processing unit (CPU), application processor, modem processor, graphics processing unit (GPU), image signal processor (ISP), microcontroller unit (MCU), video codec, digital signal processor (DSP), baseband processor, neural network processing unit (NPU), field-programmable gate array (FPGA), etc. In some embodiments, different processing units may be independent devices or integrated into one or more processors.

[0313] In some embodiments, the processor 110 may be used to execute instructions to implement the methods for human face detection provided in the foregoing embodiments.

[0314] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0315] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, display 194, camera 193, and wireless communication module 160, etc.

[0316] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0317] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0318] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G / 5G-A / 6G, for use on electronic devices 100.

[0319] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0320] Display screen 194 is used to display images.

[0321] Camera 193 is used to capture still images or videos, such as for taking real portrait data of the user.

[0322] The external memory interface 120 can be used to connect an external memory card. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, application data can be stored on the external memory card. In some embodiments, the memory connected to the external memory interface 120 can be a non-volatile memory, which can be used to store application buffers.

[0323] The internal memory 121 can be used to store one or more programs and corresponding data. The internal memory 121 may include a program storage area and a data storage area. In some embodiments, the program storage area may store an operating system, at least one application program required for a function, such as the program / instructions corresponding to the facial recognition method provided in the foregoing embodiments. The data storage area may store data created during the use of the electronic device 100 (e.g., the aforementioned first correspondence, the user's real facial data, multimedia data to be detected, etc.). Furthermore, the internal memory 121 may include high-speed random access memory (e.g., RAM); it may also include non-volatile memory, such as at least one disk storage device, flash memory device, general-purpose flash memory, etc. The processor 110 executes various functional applications of the electronic device 100 by running one or more programs stored in the internal memory 121 and / or one or more programs stored in memory disposed within the processor 110.

[0324] Electronic device 100 can implement audio functions through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor. For example, audio module 170 can be used to play prompts.

[0325] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 170 can also be used for audio signal encoding and decoding. Speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Receiver 170B, also called a "handset," is used to convert audio electrical signals into sound signals. Microphone 170C, also called a "microphone" or "microphone unit," is used to convert sound signals into electrical signals. Headphone jack 170D is used to connect wired headphones.

[0326] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0327] In some embodiments, the electronic device 100 may perform associated functions based on the touch event type determined by the application processor. For example, the electronic device 100 may detect operations such as user-triggered actions to determine enhanced detection data, user-sent enhanced detection data to or received enhanced detection data from other electronic devices, user-triggered actions to detect whether a person in multimedia data is a real person, and user-selected real person images, etc., based on the type of touch event.

[0328] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0329] Motor 191 can generate vibration as a notification. Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, notifications, etc.

[0330] In some embodiments, the electronic device 100 can prompt the first user that the image of the person is a fake by controlling the motor 191 to vibrate, the indicator 192 to flash or light up.

[0331] The SIM card interface 195 is used to connect a SIM card or eSIM.

[0332] It should be noted that the structure of the electronic device 100 shown in the embodiments of this application does not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than shown, or combine some components, or split some components, or have different component arrangements. The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0333] It should be noted that, Figure 13 The structure of the electronic device 100 shown is only an example. In other embodiments, the electronic device 100 may include more or fewer components, which is not limited here.

[0334] It should be noted that electronic device 100 can be any electronic device, including but not limited to mobile station (MS) and mobile terminal (MT). For example, electronic devices can be mobile phones, smart TVs, wearable devices, tablets, desktop computers, laptops, virtual reality (VR) devices, augmented reality (AR) devices, terminals in industrial control, self-driving, remote medical surgery, smart grids, transportation safety, smart cities, smart homes, and so on. This application does not limit the specific form of the electronic device.

[0335] This application also provides an electronic device 200, wherein the aforementioned first electronic device, second electronic device, third electronic device, fourth electronic device, and fifth electronic device may adopt the same or similar structure as electronic device 200.

[0336] For example, Figure 14 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 200 is shown.

[0337] like Figure 14 As shown, the electronic device 200 may include one or more processors 201, one or more memories 202, one or more communication interfaces 203, and a bus 204 for coupling the processors, memories, and communication units.

[0338] Processor 201 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0339] The processor 201 can be used to execute instructions to implement the methods for human face detection provided in the foregoing embodiments.

[0340] For example, when electronic device 200 is the first electronic device, processor 201 can determine the enhanced detection data of the user based on the user's real image, detect whether the user's image is a fake image based on the enhanced detection data corresponding to the user, receive or send enhanced detection data to other electronic devices, and send the user's real image to other electronic devices, etc.

[0341] For example, when electronic device 200 is a second electronic device, processor 201 can be used to determine the first enhanced detection data of the first user and send the first enhanced detection data to the first electronic device.

[0342] For example, when the electronic device is a third electronic device, the processor 201 can determine the enhanced detection data of the third user based on the training request sent by the first electronic device.

[0343] For example, when the electronic device is a fourth electronic device, the processor 201 can send a detection request to the first electronic device, receive the detection result from the first electronic device, and output the detection result.

[0344] For example, if the first electronic device is the fifth electronic device, the processor 201 can send the real human image data of the first user to the second electronic device.

[0345] Memory 202 may include volatile memory, such as random access memory (RAM). Memory 202 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0346] The memory 202 stores executable program code, which the processor 201 executes to implement the methods for human face detection provided in each embodiment.

[0347] The communication interface 203 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the electronic device 200 and other devices or communication networks.

[0348] Bus 204 serves as a pathway for information transmission between various components of electronic device 200 (e.g., processor 201, memory 202, communication interface 203). For example, bus 204 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 14 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0349] It should be noted that, Figure 14 The structure of the electronic device 200 shown is only an example. In other embodiments, the electronic device 200 may include more or fewer components, which is not limited here.

[0350] It should be noted that the electronic device 200 can be any electronic device, including but not limited to terminal devices, servers, server clusters, desktop computers, laptops, etc. This application does not limit the specific form of the electronic device 200.

[0351] It should be noted that in the embodiments of this application, "or" describes the relationship between related objects, indicating that there can be two relationships. For example, A or B can mean either A or B, where A or B can be singular or plural.

[0352] It should be noted that the term "connection" used in the embodiments of this application describes the connection relationship between two objects and can represent two types of connection relationships. For example, the connection between A and B can represent two situations: A is directly connected to B, and A is connected to B through C.

[0353] It should be noted that in the embodiments of this application, terms such as "for example," "in some embodiments," "in another embodiment," "in yet another embodiment," and "exemplarily" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as an "example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "example" is intended to present concepts in a concrete manner.

[0354] It should be noted that, in the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one item", "one or more of the following", or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c; a and b; a and c; b and c; or a and b and c. Here, a, b, and c can be single or multiple.

[0355] It should be noted that the terms "first" and "second" used in the embodiments of this application are only used for descriptive purposes and should not be construed as indicating or implying relative importance or order. The term "equal to" in the embodiments of this application can be used with "greater than" to apply to technical solutions used when "greater than," and can also be used with "less than" to apply to technical solutions used when "less than." It should be noted that when "equal to" is used with "greater than," it is not used with "less than," and vice versa.

[0356] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0357] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for human face detection, characterized in that, Applied to a first electronic device, the method includes: An instruction to inspect multimedia data was detected, wherein the multimedia data includes an image of a first user; If there is a first enhanced detection data matching the first user among the M enhanced detection data, the image of the first user in the multimedia data is detected as a fake image based on the first enhanced detection data. The M enhanced detection data are obtained based on one or more real image data of the corresponding user, and M is an integer greater than or equal to 1. If no enhanced detection data matching the first user is found among the M enhanced detection data, the image of the first user in the multimedia data is detected as a fake image based on general detection data.

2. The method according to claim 1, characterized in that, The enhanced detection data is a model, model parameters, or real human facial features; When the M enhanced detection data are M models, the first enhanced detection data is the first model among the M models; When the M enhanced detection data are M sets of model parameters of the second model, the first enhanced detection data is the first model parameter in the M sets of model parameters; When the M enhanced detection data are M real portrait features, the first enhanced detection data is the first real portrait feature among the M real portrait features.

3. The method according to claim 2, characterized in that, The step of detecting whether the image of the first user in the multimedia data is a fake image based on the first enhanced detection data includes: When the first enhanced detection data is the first model, the first model is invoked to determine whether the image of the first user in the multimedia data is a fake image; Alternatively, if the first enhanced detection data is the first model parameter, based on the second model having the first model parameter, determine whether the image of the first user in the multimedia data is a fake image; Alternatively, if the first enhanced detection data is the first real human portrait feature, a third model can be invoked, and based on the first real human portrait feature, it can be determined whether the portrait of the first user in the multimedia data is a fake portrait.

4. The method according to any one of claims 1 to 3, characterized in that, Among the M enhanced detection data, there is a first enhanced detection data that matches the first user, including: Among the M user features, there exists a first user feature that matches the first user identifier and / or the first facial feature of the first user's image. The M user features correspond one-to-one with the M enhanced detection data. The user features include user facial features and / or user identifiers. The first user feature is the user feature among the M user features that corresponds to the first enhanced detection data. Alternatively, an operation by the user to select the first enhanced detection data from the M enhanced detection data is detected.

5. The method according to claim 4, characterized in that, Among the M user features, there exists a first user feature that matches the first user identifier and / or the first facial feature of the first user's image, including: The first similarity between the first user identifier and the user identifier in the first user feature is greater than a first similarity threshold, and / or the second similarity between the first face feature and the user face feature in the first user feature is greater than a second similarity threshold; Wherein, the first similarity greater than the first similarity threshold indicates that the first user identifier and the user identifier in the first user feature belong to the same user; the second similarity greater than the second similarity threshold indicates that the first facial feature and the user facial feature in the first user feature belong to the same user.

6. The method according to any one of claims 1 to 5, characterized in that, The first enhanced detection data is received from the second electronic device; or, the first enhanced detection data is generated by the first electronic device based on one or more real human portrait data of the first user.

7. The method according to any one of claims 1 to 6, characterized in that, The detected instruction to detect multimedia data includes: During a video call or video conference conducted by the first electronic device, an operation triggering a fake human image detection by the user is detected, wherein the multimedia data is an image or video from the video call or video conference; Alternatively, the first electronic device is detected to be in the process of a video call or video conference, and the first electronic device is configured to automatically detect whether the image in the video call or video conference is a fake image, wherein the multimedia data is an image or video in the video call or video conference; Alternatively, a security risk is detected during the first electronic device's video call or video conference, wherein the multimedia data is an image or video from the video call or video conference; Alternatively, it may detect that a user has triggered an operation to detect fake human images in the multimedia data; Alternatively, a security risk is detected in a chat session involving a user of the first electronic device, wherein the chat session content includes the multimedia data; Alternatively, a detection request is received from a fourth electronic device, the detection request including the multimedia data; Alternatively, the first application is detected to be running, and the first electronic device is configured to automatically detect whether the portrait in the multimedia data of the first application is a real portrait during the running of the first application.

8. The method according to claim 7, characterized in that, The method further includes: If the image of the first user in the multimedia data is detected to be a fake image, a prompt message is output. And / or, upon receiving the detection request sent by the fourth electronic device, send to the fourth electronic device the detection result of whether the image of the first user in the multimedia data is a fake image.

9. The method according to any one of claims 1 to 8, characterized in that, The multimedia data is received from a fourth electronic device; and the method further includes: The detection result of whether the image of the first user in the multimedia data is a fake image is sent to the fourth electronic device.

10. The method according to any one of claims 1 to 8, characterized in that, The multimedia data also includes a second user's image; the method further includes: If there is a second enhanced detection data that matches the second user among the M enhanced detection data, the second enhanced detection data is used to detect whether the image of the second user in the multimedia data is a fake image. The second enhanced detection data is obtained based on one or more real image data of the second user. If no enhanced detection data matching the second user is found among the M enhanced detection data, the image of the second user in the multimedia data is detected as a fake image based on the general detection data.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: If the update condition for the corresponding first user is detected, update the first enhanced detection data; The update conditions corresponding to the first user include any one of the following conditions: The first electronic device was detected to have captured at least one real human image data of the first user. An operation was detected in which the user updated the enhanced detection data of the first user. The preset update cycle has been reached.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: In response to a user's sharing action, one or more of the M enhanced detection data are sent to a fifth electronic device or a first group.

13. A readable storage medium, characterized in that, The readable storage medium stores one or more programs, which, when executed on an electronic device, cause the electronic device to implement the method for human face detection as described in any one of claims 1 to 12.

14. An electronic device, characterized in that, include: At least one main memory and at least one non-volatile memory; A processor for executing one or more programs to cause the electronic device to implement the method for human face detection as described in any one of claims 1 to 12.

15. A program product, characterized in that, When the program product is executed on an electronic device, it causes the electronic device to implement the method for human face detection as described in any one of claims 1 to 12.

Images