Image processing method and electronic device

Abstract

The application discloses an image processing method and an electronic device, relates to the technical field of video anti-counterfeiting detection, and is used for reducing the memory occupied by an image frame for video anti-counterfeiting detection and without losing face information in the image frame. The method comprises the following steps: after a first image frame is acquired, a first face region is adjusted in size based on the fact that the first face region exists in the first image frame, so that a second face region corresponding to the first face region is obtained. Finally, the first image frame is cropped along the boundary of the second face region, so that a second image frame corresponding to the first image frame is obtained. The first image frame is an image frame corresponding to a target layer constituting a video call interface, and the target layer is used for displaying a caller user image. The boundary of the second face region does not exceed the boundary of the first image frame. The second image frame is used for identifying the authenticity of a caller user corresponding to the caller user image.

Description

Technical Field

[0001] This application relates to the field of video anti-counterfeiting detection technology, and in particular to an image processing method and an electronic device. Background Technology

[0002] Video calling is a communication method that allows real-time transmission of voice and images between the parties in a call (referred to as the caller and receiver). However, the receiver cannot verify whether the person they are video calling is actually the caller.

[0003] Therefore, during a video call, image frames including the caller's image can be captured from the video call interface. These image frames are then analyzed using a video anti-spoofing detection model to determine if the person in the video call is indeed the real caller. Typically, a sufficient number of image frames are required for video anti-spoofing detection. Therefore, electronic devices need to acquire a sufficient number of image frames for this purpose. However, a large number of image frames consumes a significant amount of the electronic device's RAM, leading to lag, application crashes, and other malfunctions. Summary of the Invention

[0004] This application provides an image processing method and an electronic device for reducing the memory occupied by image frames used for video anti-counterfeiting detection without losing facial information in the image frames.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, an image processing method is provided, comprising: after an electronic device acquires a first image frame, adjusting the size of the first face region based on the existence of a first face region in the first image frame to obtain a second face region corresponding to the first face region. The boundary of the adjusted second face region does not exceed the boundary of the first image frame. Finally, the electronic device crops the first image frame along the boundary of the second face region to obtain a second image frame corresponding to the first image frame.

[0007] The first image frame is the image frame corresponding to the target layer that constitutes the video call interface, and the target layer is used to display the caller's user image. The second image frame is used to verify the authenticity of the caller's user image in the video call.

[0008] The second image frame obtained using the image processing method provided in this application includes the facial information from the first image frame, ensuring the accuracy of video anti-counterfeiting detection. Furthermore, the size of the second image frame is smaller than that of the first image frame, resulting in a smaller memory footprint for the second image frame compared to the first. This allows for direct anti-counterfeiting detection of the video call data composed of the second image frame within the electronic device during a video call, without needing to upload the video call data to the cloud for detection, thus maximizing user privacy protection.

[0009] In one possible implementation of the first aspect, the above-mentioned adjustment of the size of the first face region to obtain the second face region corresponding to the first face region may include: first, the electronic device expands the first face region into the background region to obtain the third face region corresponding to the first face region; then, the electronic device adjusts the size of the third face region based on the size relationship between the third size and the target size to obtain the second face region corresponding to the third face region.

[0010] The boundary of the third face region does not exceed the boundary of the first image frame, the background region is the region in the first image frame other than the first face region, and the size of the third face region is the third size.

[0011] Specifically, adjusting the size of the third face region based on the size relationship between the third size and the target size to obtain the second face region corresponding to the third face region can include: the electronic device reducing the size of the third face region to the target size by downsampling based on the third size being larger than the target size, thereby obtaining the second face region; or the electronic device increasing the size of the third face region to the target size by upsampling based on the third size being smaller than the target size, thereby obtaining the second face region.

[0012] Based on this, the obtained second face region retains the facial information of the first face region and some background information around the first face region, while also ensuring that all second face regions have the same size. Ultimately, the second image frame obtained by cropping the first image frame along the boundary of the second face region satisfies the conditions of the video anti-counterfeiting detection model.

[0013] In another possible implementation of the first aspect, the extension of the first face region to the background region may include at least one of the following:

[0014] The electronic device extends the boundary of the first face region toward the corresponding boundary of the first image frame by a first preset pixel, based on the fact that the distance between the boundary of the first face region and the corresponding boundary in the first image frame is greater than a first preset pixel.

[0015] The electronic device extends the boundary of the first face region towards the corresponding boundary of the first image frame by a second preset pixel, based on the distance between the boundary of the first face region and the corresponding boundary in the first image frame. The second preset pixel is greater than 0 and less than the first preset pixel.

[0016] The electronic device determines that the boundary of the first face region is not extended to the corresponding boundary of the first image frame because the distance between the boundary of the first face region and the corresponding boundary in the first image frame is 0.

[0017] In another possible implementation of the first aspect, the above-mentioned adjustment of the size of the third face region based on the size relationship between the third size and the target size to obtain the second face region corresponding to the third face region includes: when the third size is greater than the second size threshold, adjusting the size of the third face region based on the size relationship between the third size and the target size to obtain the second face region corresponding to the third face region.

[0018] Typically, when the face region is small, it contains less facial information, which cannot guarantee the accuracy of video anti-spoofing detection. Therefore, further processing can be applied only to the third face region whose size is larger than the second size threshold.

[0019] In another possible implementation of the first aspect, the above-mentioned adjustment of the size of the first face region to obtain the second face region corresponding to the first face region includes: adjusting the size of the first face region based on the fact that the size of the first face region is greater than a first size threshold to obtain the second face region corresponding to the first face region.

[0020] Typically, when the face region is small, it contains less facial information, which cannot guarantee the accuracy of video anti-spoofing detection. Therefore, only the first face region whose size is larger than the first size threshold can be further processed.

[0021] In another possible implementation of the first aspect, before adjusting the size of the first face region based on the existence of a first face region in the first image frame to obtain a second face region corresponding to the first face region, the image processing method further includes: filtering out the transparency data of the first image frame to obtain a first image frame with RGB format.

[0022] Typically, the first image frame obtained directly from the multiple layers that make up the video call interface is in RGBA format. Compared to a first image frame in RGB format, an RGBA format first image frame also includes data from the alpha channel (i.e., transparency data). This transparency data does not include facial information. Therefore, the transparency data of the first image frame can be filtered out. This reduces the memory footprint of the first image frame.

[0023] In another possible implementation of the first aspect, the existence of a first face region in the first image frame means that there is one or two first face regions in the first image frame.

[0024] In another possible implementation of the first aspect, the resolution of the first image frame is the video call resolution.

[0025] Video call resolution is typically lower than the screen resolution of electronic devices. The lower the image resolution, the less memory the image occupies. Therefore, using the video call resolution as the resolution of the first image frame can reduce the memory occupied by the first image frame.

[0026] In another possible implementation of the first aspect, the acquisition of the first image frame includes: acquiring the first image frame at a sampling frequency of the video call frame rate of the electronic device.

[0027] Typically, the video call frame rate is lower than the screen refresh rate. By acquiring the first image frame at the video call frame rate of the electronic device, not only is facial information not lost, but the first image frame is also not repeatedly acquired.

[0028] In another possible implementation of the first aspect, the image processing method further includes: losslessly compressing the second image frame to obtain a third image frame to be detected. The third image frame is used as input to the video anti-counterfeiting detection model.

[0029] Lossless compression does not lose facial information in each image frame and does not need to consider the correlation between image frames, thus resulting in high compression efficiency.

[0030] In another possible implementation of the first aspect, the image processing method further includes: counting the cumulative number of third image frames acquired this time; if the cumulative number reaches a threshold, inputting all third image frames into the video anti-counterfeiting detection model for video anti-counterfeiting detection; if the cumulative number does not reach the threshold, continuing to acquire third image frames.

[0031] Typically, video anti-counterfeiting detection models require a sufficient number of third image frames as input before performing anti-counterfeiting detection. Therefore, electronic devices need to acquire a sufficient number of third image frames.

[0032] Secondly, an image processing method is provided, applied to an electronic device, which includes an acquisition module and an algorithm module. The method includes:

[0033] The acquisition module obtains the first image frame and sends it to the algorithm module. The first image frame is the image frame corresponding to the target layer that constitutes the video call interface, and the target layer is used to display the caller's user image.

[0034] The algorithm module adjusts the size of a first face region based on the existence of a first face region in the first image frame to obtain a second face region corresponding to the first face region. The boundary of the adjusted second face region does not exceed the boundary of the first image frame.

[0035] The algorithm module crops the first image frame along the boundary of the second face region to obtain the second image frame corresponding to the first image frame.

[0036] Thirdly, an electronic device is provided, comprising one or more processors, and a display screen, a camera, a speaker, a microphone, and a memory coupled to the processors. The display screen is used to display a video call interface and a video call interface. The camera is used to capture an image of the call subject. The speaker is used to play sound. The microphone is used to capture sound. The memory stores computer program code, which includes instructions. When the instructions are executed by the processor, the electronic device performs the method as described in any one of the first aspects above.

[0037] Fourthly, a computer-readable storage medium is provided that stores instructions which, when executed on an electronic device, cause the electronic device to perform the method described in any one of the first aspects.

[0038] Fifthly, a computer program product containing instructions is provided, which, when run on an electronic device, enables the electronic device to perform the method described in any one of the first aspects above.

[0039] In a sixth aspect, embodiments of this application provide a chip, the chip including a processor, the processor being configured to invoke a computer program in memory to perform the method as described in the first aspect.

[0040] Understandably, the beneficial effects achievable by the image processing method described in the second aspect, the electronic device described in the third aspect, the computer-readable storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip described in the sixth aspect can be referred to the beneficial effects in the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0041] Figure 1 A schematic diagram of a video call scenario provided in an embodiment of this application;

[0042] Figure 2 One of the schematic diagrams of a video call interface provided in this application embodiment;

[0043] Figure 3A second schematic diagram of a video call interface provided for an embodiment of this application;

[0044] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0046] Figure 6 This is one of the schematic flowcharts of an image processing method provided in an embodiment of this application;

[0047] Figure 7 The third schematic diagram of a video call interface provided in this application embodiment;

[0048] Figure 8 This is one of the schematic diagrams of an image frame captured from a video call interface using conventional technology;

[0049] Figure 9 This is the second illustration of an image frame captured from a video call interface using conventional technology.

[0050] Figure 10 This is the third illustration of an image frame captured from a video call interface using conventional technology.

[0051] Figure 11 A schematic diagram of an image frame captured from a video call interface, provided as an embodiment of this application;

[0052] Figure 12 The fourth schematic diagram of a video call interface provided in this application embodiment;

[0053] Figure 13 This is a second schematic flowchart of an image processing method provided in an embodiment of this application;

[0054] Figure 14 One of the schematic diagrams of a first image frame provided in an embodiment of this application;

[0055] Figure 15 A second schematic diagram of a first image frame provided in an embodiment of this application;

[0056] Figure 16 The third schematic flowchart of an image processing method provided in this application embodiment;

[0057] Figure 17 A schematic diagram illustrating the principle of acquiring a first image frame, provided in an embodiment of this application;

[0058] Figure 18This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0059] The terms "first" and "second" used in the embodiments of this application are only used to distinguish features of the same type and should not be construed as indicating relative importance, quantity, order, etc.

[0060] The terms "exemplary" or "for example" used in the embodiments of this application are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0061] The terms "coupling" and "connection" used in the embodiments of this application should be interpreted broadly. For example, they can refer to a physical direct connection or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.

[0062] This application provides an image processing method that can be applied to... Figure 1 The video call scenario shown is as follows. Figure 1 As shown, the video call scenario may include a first electronic device 110 and a second electronic device 120. Both the first electronic device 110 and the second electronic device 120 are equipped with a video calling application. During a video call, the first electronic device 110 may be the calling device and the second electronic device 120 may be the receiving device; alternatively, the first electronic device 110 may be the receiving device and the second electronic device 120 may be the calling device. The first electronic device 110 and the second electronic device 120 may be the same type of electronic device or different types of electronic devices.

[0063] Typically, video call scenarios can include one-to-one video call scenarios and one-to-many video call scenarios. In some examples, one-to-many video call scenarios may also be referred to as group chat video call scenarios, multi-person video call scenarios, multi-person call scenarios, etc.

[0064] In one embodiment, a one-to-one video call scenario refers to an electronic device triggering a one-to-one video call function in response to the activation of relevant controls in a one-to-one chat interface with a contact within a video call application. In a one-to-one video call scenario, other contacts within the video call application cannot join the video call using their own electronic devices. The call participants in a one-to-one video call scenario can include two individuals: a caller and a receiver.

[0065] In one embodiment, a one-to-many video call scenario refers to an electronic device triggering a one-to-many video call function in response to the activation of relevant controls of at least one contact in the group chat interface of a video call application. In a one-to-many video call scenario, other contacts in the group can join the video call using their own electronic devices. The call participants in a one-to-many video call scenario can include at least two individuals: one caller and at least one receiver.

[0066] Therefore, the difference between one-to-one video call scenarios and one-to-many video call scenarios lies in whether other participants can join the video call through their own electronic devices, rather than being limited by the number of participants.

[0067] The following text uses the example of the first electronic device being the receiving device, the second electronic device being the calling device, and both the first and second electronic devices being mobile phones, to illustrate this point. Figure 2 and Figure 3 Detailed introduction to one-on-one video call scenarios:

[0068] In one embodiment, for the caller, such as Figure 2 As shown in Figure A, mobile phone A (i.e., the calling device) displays a one-to-one chat interface 210 with contact A in the video call application. The one-to-one chat interface 210 includes a plus sign control 211. In response to a triggering operation on the plus sign control 211, such as... Figure 2 As shown in Figure B, mobile phone A displays a menu window 220 in the one-to-one chat interface 210. The menu window 220 includes a video call control 221. In response to a trigger operation on the video call control 221, such as... Figure 2 As shown in C, mobile phone A switches from a one-to-one chat interface 210 to a one-to-one video call interface 230 with contact A. The one-to-one video call interface 230 can include the recipient's user information. For example, the recipient's user information can include the recipient's avatar 231 and the recipient's name 232. When the recipient's device accepts the video call request, as... Figure 2 As shown in D, mobile phone A displays a first video call interface 240. The first video call interface 240 includes an image 241 of the receiving user (such as contact A) and an image 242 of the calling user. The first video call interface 240 is a one-to-one video call interface.

[0069] In one embodiment, for the receiving party, such as Figure 3As shown in Figure A, after receiving a video call request, mobile phone B (i.e., the receiving device) displays a one-to-one video call interface 310. The one-to-one video call interface 310 includes an answer control 311 and the caller's user information. For example, the caller's user information may include the caller's avatar 312 and the caller's name 313. In response to a trigger operation on the answer control 311, such as... Figure 3 As shown in Figure B, mobile phone B switches from a one-to-one video call interface 310 to a second video call interface 320. The second video call interface 320 includes an image 241 of the receiving user (such as contact A) and an image 242 of the calling user. The second video call interface 320 is a one-to-one video call interface.

[0070] Of course, in a one-to-many video call scenario, the one-to-many video call interface displayed on mobile phone B will also include the images of both the caller and the receiver. This will not be elaborated upon further in this article.

[0071] In summary, in one-to-one or one-to-many video call scenarios, after the receiving device 110 and the calling device 120 establish a communication connection, the receiving device 110 can display the calling user's image 242. However, based on the calling user's image 242 displayed on the receiving device 120, the receiving user cannot determine whether the person in the video call is the actual calling user, which may lead to potentially dangerous actions that could cause losses to the receiving user.

[0072] To avoid the aforementioned security risks and protect user privacy, this application provides an electronic device with video anti-counterfeiting detection functionality. Specifically, firstly, during a video call, when the electronic device receives a video call request and its camera is on and its audio is in call mode, the electronic device can acquire a first image frame from multiple layers constituting the video call interface. This first image frame, corresponding to the target layer, is used for video anti-counterfeiting detection to determine the authenticity of the caller's image. The target layer displays the caller's image.

[0073] In this embodiment, the first image frame is the layer containing the caller's user image. Compared to directly obtaining an image frame containing the caller's user image from a screenshot of the video call interface, the caller's user image in the first image frame is unobstructed, which can improve the accuracy of subsequent video anti-counterfeiting detection. Simultaneously, the first image frame does not include layers other than the target layer, resulting in a smaller data volume. This significantly reduces the computational load on the electronic device in subsequent processes, saves the device's operating memory, and helps improve the efficiency of the subsequent video anti-counterfeiting detection process.

[0074] Furthermore, without losing the facial information in the first image frame, the electronic device resizes the first image frame to obtain a second image frame corresponding to the first image frame. The second image frame includes the facial information from the first image frame, and the memory occupied by the second image frame is less than that occupied by the first image frame. The second image frame is used as input to the video anti-counterfeiting detection model for video anti-counterfeiting detection.

[0075] Therefore, during video calls, anti-spoofing detection can be performed directly on the electronic device on the video call data composed of the second image frame, without needing to upload the video call data to the cloud for detection. This maximizes the protection of user privacy. Furthermore, since the second image frame includes facial information and occupies less memory, it does not affect subsequent video anti-spoofing detection and further conserves system resources such as the electronic device's RAM.

[0076] Furthermore, if it is determined that the person in the ongoing video call is not the actual caller, the electronic device can display a corresponding prompt to remind the receiving user, thereby preventing the aforementioned potentially dangerous behavior from occurring.

[0077] The electronic devices involved in the embodiments of this application can also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, etc. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0078] Figure 4 A schematic diagram of the hardware structure of an electronic device according to an embodiment of this application is shown. The following description, in conjunction with... Figure 4 The hardware structure of electronic devices will be introduced.

[0079] Take a mobile phone as an example. Figure 4 As shown, the electronic device 400 may include: a processor 410, a memory 420, a universal serial bus (USB) interface 430, a power management module 440, an antenna, a communication module 450, a display screen 460, an audio module 470, a camera 480, a sensor module 490, etc.

[0080] Processor 410 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processor. Network processing units (NPUs), etc. Different processing units can be independent devices or integrated into one or more processors. The controller can be the nerve center and command center of the electronic device 400. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0081] Memory 420 can be used to store executable program code, including instructions. Processor 410 executes various functional applications and data processing of the electronic device by running the instructions stored in memory 420. Memory 420 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, interface display, etc.). The data storage area may store data created during the use of the electronic device (such as notification messages). Furthermore, memory 420 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0082] The power management module 440 is used to connect the battery to the processor 410. The power management module 440 receives battery and / or power input to power the processor 410, memory 420, communication module 450, display 460, audio module 470, camera 480, and sensor module 490, etc. The power management module 440 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 440 may also be located within the processor 410.

[0083] Communication module 450 can provide applications on electronic device 400 including wireless local area networks (WLAN) (such as wireless fidelity, Wi-Fi) Solutions for wireless communications include Fibre and Cable (Fi) networks, Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). The communication module 450 can be one or more devices integrating at least one communication processing module. The communication module 450 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to the processor 410. The communication module 450 can also receive signals to be transmitted from the processor 410, perform frequency modulation and amplification, and then convert them into electromagnetic waves for radiation via the antenna.

[0084] In some embodiments, the antenna of the electronic device 400 is coupled to the communication module 450, enabling the electronic device 400 to communicate with networks and other devices via wireless communication technologies. The wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, Global Navigation Satellite System (GNSS), WLAN, NFC, FM, and / or IR technologies. The GNSS may include Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), GLONASS, and / or Galileo.

[0085] Electronic device 400 implements display functions through a GPU, a display screen 460, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 460 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0086] Display screen 460 is used to display images, videos (such as video calls), etc. Display screen 460 includes a display panel. The display panel can be a liquid crystal display (LCD) or an organic light-emitting diode (OLED). Organic light-emitting diodes (OLEDs), also known as active matrix organic light-emitting diodes or active matrix organic light-emitting diodes (OLEDs). Matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (Flexible Light Emitting Diode). emitting diode, FLED), Mini LED, Micro OLED, Micro OLED, quantum dot light emitting diodes (QLED), etc.

[0087] Electronic device 400 can realize video call function through ISP, camera 480, video codec, GPU, display 460 and application processor.

[0088] The audio module 470 may include a speaker and a microphone, etc. The speaker is used to convert digital audio information into analog audio signals for output. The microphone is used to convert analog audio input into digital audio signals. The audio module 470 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 470 may be located in the processor 410, or some functional modules of the audio module 470 may be located in the processor 410.

[0089] The camera 480 is used to capture still images or videos. An optical image of an object is generated through the lens and projected onto a photosensitive element. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) sensor. oxide Semiconductor (CMOS) phototransistors. The photosensitive element converts light signals into electrical signals, which are then passed to the photoelectric image sensor (ISP) for conversion into digital image signals. The ISP outputs the digital image signals to the digital signal processing DSP. The DSP converts the digital image signals into standard RGB, YUV, and other image formats.

[0090] The sensor module 490 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors, etc.

[0091] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 400. In other embodiments, the electronic device 400 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0092] Generally, electronic devices require both hardware support and software cooperation to implement video anti-counterfeiting detection functions. The software system of electronic devices can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes the layered architecture of the Android operating system® as an example, combined with... Figure 5 The software architecture of the electronic device involved in the embodiments of this application will be described.

[0093] Figure 5 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application is shown.

[0094] like Figure 5 As shown, the layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android operating system® is divided into three layers, from top to bottom: the application (APP) layer, the application framework (FWK) layer, and the hardware layer.

[0095] The application layer can include a series of application packages, such as video calling applications, sensing fences, early warning modules, algorithm modules, and data acquisition modules. When these application packages are run, they can access the application framework layer and hardware layer through application programming interfaces (APIs) and execute corresponding intelligent services (such as video anti-counterfeiting detection services).

[0096] The video calling application is used to provide video calling services. The video calling application can be a system application built into the electronic device or a third-party application installed on the electronic device; this embodiment of the application does not limit this.

[0097] The sensing fence can be an event-aware fence (or behavior-aware fence). When a video call application receives a video call request, the sensing fence can detect the video call event, and if the video call event is a video call event subscribed to by the alert module, it will notify the alert module so that the alert module can perform the next step.

[0098] The early warning module stores a whitelist. The whitelist can include application information for at least one video calling application. For example, the application information for a video calling application may include the application name (or package name) and the name of the video call interface in the application, but is not limited to these. During a video call, the video calling applications in the whitelist are those that can be detected for video fraud prevention. The video call events subscribed to by the early warning module are events where a video call interface appears in a video calling application in the whitelist. The early warning module can send subscription messages to the sensing fence to subscribe to the video call events corresponding to the whitelist. The subscription message can include the application information corresponding to the whitelist.

[0099] In one embodiment, the video call interface involved in the whitelist is a one-to-one video call interface. In this embodiment, the video call event subscribed to by the alert module refers to the event where a video call application on the whitelist displays a one-to-one video call interface.

[0100] In another embodiment, the video call interface involved in the whitelist is a one-to-many video call interface. In this embodiment, the video call event subscribed to by the alert module refers to the event where the whitelisted video call application displays a one-to-many video call interface.

[0101] In one embodiment, the warning module can obtain a whitelist from the server. For example, the warning module can proactively obtain the whitelist from the server, or the server can proactively send the whitelist to the warning module.

[0102] The alert module also retrieves the camera status from the camera manager in the application framework layer and the voice status from the audio manager in the application layer. The camera status can include whether it's on or off. The voice status can include whether it's in a call or not.

[0103] The early warning module is also used to notify the acquisition module to start acquiring the target layer from the multiple layers constituting the video call interface when it receives a notification from the sensing fence that a video call event has occurred, and the camera is on and the voice is in a call state, thereby obtaining the first image frame. The target layer is the layer containing the caller's image.

[0104] For example, the early warning module can send a notification message to the acquisition module to acquire the target layer, so as to notify the acquisition module to start acquiring the first image frame.

[0105] The early warning module also receives the third image frame output by the algorithm module and counts the cumulative number of third image frames. When the cumulative number of third image frames reaches a threshold, the early warning module inputs all third image frames into the video anti-counterfeiting detection model for video anti-counterfeiting detection to obtain the video anti-counterfeiting detection result.

[0106] For example, the video anti-counterfeiting detection result can be a confidence level indicating that the object being video-called by the receiving user is not the real calling user.

[0107] For example, if the confidence level is greater than the first confidence threshold, it indicates that the person being video-called by the receiving user is not the actual caller.

[0108] For example, if the confidence level is less than the first confidence level threshold and greater than the second confidence level, it indicates that the person being video-called by the receiving user may not be the actual caller.

[0109] For example, if the confidence level is less than the second confidence level, it indicates that the person being video-called by the receiving user is the actual caller.

[0110] The early warning module is also used to send a reminder message to the display module in the hardware layer through relevant interfaces in the application framework layer when it is determined that the person being video-called by the receiving user is not the real caller. This is to alert the receiving user that the person being called is not the real caller. Alternatively, the early warning module may not take any action if it is determined that the person being video-called by the receiving user is the real caller. Alternatively, if it is determined that the person being video-called by the receiving user may not be the real caller, the early warning module will store the video anti-spoofing detection result and, in conjunction with other relevant information (such as the video anti-spoofing detection result during the next video call), comprehensively determine in subsequent processes whether the same person video-called by the receiving user's device is the real caller.

[0111] The acquisition module, upon receiving a notification message from the warning module, begins acquiring the first image frame and sends it to the algorithm module. Typically, the first image frame is in RGBA format, meaning it includes data from the red, green, blue, and alpha channels.

[0112] For example, the acquisition module can obtain the target layer based on the layer identifier of the target layer through the window manager service (WMS) of the application framework layer, thereby obtaining the first image frame.

[0113] For example, the acquisition module can obtain the target layer through the layer rendering compositing service of the application framework layer and the virtual screen created by the virtual machine control monitoring service (DMS), thereby obtaining the first image frame.

[0114] The algorithm module filters the alpha data in the first image frame to obtain the first image frame in RGB format. Alternatively, the algorithm module can choose not to filter the alpha data in the first image frame.

[0115] The algorithm module is also used to perform face recognition on each first image frame to determine whether a face region (such as a first face region) exists in the first image frame. If the first image frame contains a face region, the algorithm module will continue to process the first image frame. Otherwise, the algorithm module will no longer process the first image frame.

[0116] The algorithm module is also used to expand the first face region in the first image frame into the background region according to a preset ratio to obtain the third face region; then, the algorithm module normalizes the size of the third face region in the first image frame to obtain the second face region.

[0117] The algorithm module is also used to crop the first image frame along the boundary of the second face region to obtain the second image frame corresponding to the first image frame.

[0118] In one embodiment, the algorithm module is further configured to compare the size of a face region (such as at least one of a first face region, a second face region, and a third face region) in the first image frame with a corresponding size threshold. If the size of a face region is greater than the corresponding size threshold, the algorithm module will continue to process that face region. Otherwise, the algorithm module will no longer process that face region. This part can be referred to in the detailed description of S607 below, and will not be repeated here.

[0119] The application framework layer can include a camera manager, an audio manager, a layer rendering and compositing service, a window management service, and a virtual machine monitoring and control service. The camera manager is used to acquire camera status. The audio manager is used to acquire voice status. The layer rendering and compositing service is used to draw individual layers, composite them, and then display them. The window management service is used for window management, window animation management, and layer management. The virtual machine monitoring and control service is used to create virtual screens.

[0120] The hardware layer may include a camera, microphone, speaker, and display module. The camera is used to capture images during a video call. For example, the camera on the calling device captures an image of the calling user, and the camera on the receiving device captures an image of the receiving user. The microphone is used to capture sound during a video call. The speaker is used to play sound during a video call. The display module is used to display the call participants during a video call. The display module may include a screen.

[0121] It should be noted that, Figure 5 The layers in the illustrated software architecture and the components contained within each layer do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more layers than illustrated, such as a system library layer and a kernel layer. Each layer may include more or fewer components than illustrated. Furthermore, the various functional modules described above may be combined into a single functional module, and the various layers may be combined into a single layer.

[0122] It is understood that, in order to implement the methods in the embodiments of this application, electronic devices include hardware and / or software modules that perform various functions. Based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0123] Figure 6 This illustration shows one of the flowcharts of an image processing method provided in an embodiment of this application. For example... Figure 6 As shown, the method may include:

[0124] S601, In response to the operation of making a video call to contact A in the first video call application, the calling device sends a video call request to the receiving device.

[0125] All electronic devices involved in the embodiments of this application have a video call application installed. For ease of distinction, the video call application installed on the calling device is referred to as the first video call application, and the video call application installed on the receiving device is referred to as the second video call application.

[0126] The receiving device is the electronic device held by contact A in the first video call application.

[0127] A video call request can be either a one-to-one video call request or a one-to-many video call request. The type of video call request depends on the type of operation (or video call scenario) in which the video call is made. In a one-to-one video call scenario, the video call request is a one-to-one video call request. In a one-to-many video call scenario, the video call request is a one-to-many video call request.

[0128] In one embodiment, such as Figure 2 China A and Figure 2 As shown in Figure B, the operation of making a video call to contact A in the first video call application can sequentially include: triggering the plus sign control 211 in the one-to-one chat interface 210 with contact A on mobile phone A (i.e., the calling device), and triggering the video call control 221 in the menu window 220 displayed in the one-to-one chat interface 210 on mobile phone A. At this time, the video call request is a one-to-one video call request.

[0129] It should be noted that when the calling device sends a video call request to the receiving device, the calling device will display the video call interface of the first video call application.

[0130] For example, such as Figure 2 As shown in C, when mobile phone A sends a one-to-one video call request to mobile phone B, mobile phone A displays the one-to-one video call interface 230 of the first video call application.

[0131] S602, The receiving device displays a video incoming call interface.

[0132] Upon receiving a video call request, the receiving device will display a video call interface. The type of video call scenario corresponds one-to-one with the type of video call request, and the type of video call request corresponds one-to-one with the type of video call interface. Examples are given below:

[0133] In one embodiment, the video call scenario is a one-to-one video call scenario. In this case, the video call request is a one-to-one video call request, and the video call interface is a one-to-one video call interface.

[0134] For example, such as Figure 3As shown in A, after receiving a one-to-one video call request, mobile phone B (i.e., the receiving device) will display the one-to-one video call interface 310 of the video call application.

[0135] In another embodiment, the video call scenario is a one-to-many video call scenario. In this case, the video call request is a one-to-many video call request, and the video call interface is a one-to-many video call interface.

[0136] For example, such as Figure 7 As shown, after receiving a one-to-many video call request, mobile phone B will display a one-to-many video call interface 710. The multi-person video call interface 710 may include caller user information and at least one receiver user information. In response to an answering operation on any receiver device, the corresponding receiver user joins the video call through their receiver device.

[0137] In one embodiment, each video call interface has a unique identifier (such as an interface name). Therefore, when a video call interface is displayed on an electronic device, the electronic device can determine whether the video call interface is a one-to-one video call interface based on whether the identifier of the video call interface is the identifier of a one-to-one video call interface; or, the electronic device can determine whether the video call interface is a one-to-many video call interface based on whether the identifier of the video call interface is the identifier of a one-to-many video call interface.

[0138] In one embodiment, such as Figure 3 As shown in Figure A, when an electronic device displays a video call interface, the camera of the electronic device is turned on.

[0139] S603, In response to the answering operation on the video call interface, the receiving device switches from the video call interface to the video call interface.

[0140] The types of video call interfaces correspond one-to-one with the types of video call interfaces. When the video call interface is a one-to-one video call interface, the corresponding video call interface is a one-to-one video call interface.

[0141] For example, such as Figure 3 As shown in A, the answering operation on the video call interface refers to triggering the answering control 311 on the one-to-one video call interface 310.

[0142] In one embodiment, in response to an answering operation on the video call interface, the speaker of the receiving device switches from a closed state to a closed state, and the microphone switches from a closed state to a closed state.

[0143] For example, such as Figure 3As shown in B, the second video call interface 320 (i.e., one-to-one video call interface) of mobile phone B displays: the camera is on, the microphone is on, and the speaker is on.

[0144] Typically, in response to an answering action on the video call interface, the receiving device switches from the video answering interface to a full-screen video call interface.

[0145] S604. The video call interface is a video call interface of a video call application in the whitelist. The voice state switches from non-call state to call state, and the camera is turned on. The receiving device obtains the first image frame from the multiple layers that constitute the video call interface.

[0146] The electronic devices involved in this application embodiment have a pre-defined whitelist. The whitelist includes application information of video calling applications that need to be detected for video anti-counterfeiting. The application information may include: the application name (or package name) of the video calling application, and the interface name of the video call interface in the video calling application, etc.

[0147] The video call interface involved in the whitelist can be a one-to-many video call interface or a one-to-one video call interface.

[0148] In one embodiment, the electronic device can obtain an updated whitelist from the server. Specifically, when the whitelist on the server is updated, the server will proactively send the updated whitelist to the electronic device. Alternatively, the electronic device can send query messages to the server at a fixed frequency to check whether the whitelist on the server has been updated. During a query, if the whitelist on the server has been updated, the server will send the updated whitelist to the electronic device. Otherwise, the server will not send the whitelist to the electronic device.

[0149] It should be noted that in S604, the receiving device can determine whether the current video call interface is the video call interface of the latest whitelisted video call application.

[0150] In one embodiment, the receiving device begins acquiring the first image frame when: the receiving device determines that the video call interface is from a whitelisted video call application, the receiving device's camera is on, and the voice status is in a call state. If the receiving device determines that the video call interface is not from a whitelisted video call application, the camera is off, or the voice status is not in a call state, then the receiving device will not be triggered to acquire the first image frame from the multiple layers constituting the video call interface.

[0151] For example, the receiving device can obtain the camera status from the camera manager. The receiving device can call the AudioManager.getMode() function to obtain the voice status from the audio manager. When the voice status is AudioManager.MODE_IN_COMMUNICATION, it indicates that the voice status is in call state.

[0152] In conventional technologies, during a video call, the electronic device needs to call the video call application's interface to obtain the image frames corresponding to the video call interface. Different video call applications have different software architectures, therefore requiring different interfaces to be called. Typically, to obtain the image frames corresponding to the video call interface, separate development is needed for each different video call application. This excessive coupling significantly increases cost and workload.

[0153] In this embodiment, the receiving device can acquire the first image frame as soon as the condition for starting to acquire the first image frame is triggered, without calling the video call application's interface. Therefore, the image processing method provided in this embodiment is applicable to any video call application, exhibiting strong versatility.

[0154] Typically, an interface displayed on an electronic device is composed of multiple layers superimposed. The following example illustrates the one-to-one video call interface displayed on the receiving device, assuming the receiving device is a non-foldable screen device:

[0155] In one embodiment, such as Figure 8 As shown in Figure A, the receiving device is a non-foldable screen phone, meaning the receiving device includes a display screen. The second video call interface 320 displayed on the receiving device may include: a layer containing the status bar 810 at the top of the screen, a layer containing the navigation bar at the bottom (or side) of the screen, and a layer containing the navigation bar. Figure 8 (Not shown in the image), and the layer where the application interface 820 is located. The application interface 820 can be used to display the caller's user image 242. Alternatively, the application interface 820 can also be used to display the receiver's user image 241.

[0156] Next, as Figure 8 As shown in B, if the receiving device directly calls the system's screen recording and screenshot interfaces to obtain image frames, the obtained image frames are composed of multiple superimposed layers. Although the image frame fully displays the images of both the receiving and calling users, it includes too much other data besides these images. This increases the computational load on the receiving device during subsequent processing and also consumes more of its operating memory.

[0157] In another embodiment, combined Figure 8 In the case of A, such as Figure 9 As shown in A, when the receiving device displays the second video call interface 320, it may also display other layers. For example, the receiving device may display the layer containing the message pop-up 910, the layer containing the floating ball (not shown in the figure), and the layer containing the floating window (not shown in the figure) above the layer containing the application interface 820. In this way, the message pop-up 910 will obscure part of the content of the application interface 820, especially the face area of ​​the caller's user image 242 in the application interface 820.

[0158] Next, as Figure 9 As shown in B, if the receiving device directly calls the system's screen recording and screenshot interfaces to obtain image frames, the obtained image frames are composed of multiple superimposed layers. Due to the obstruction of the layer containing message pop-up 910, the caller's image cannot be fully displayed in this image frame. This hinders subsequent video anti-spoofing detection of this image frame, thus affecting the receiving device's ability to determine whether the person in the video call is the genuine caller based on the obtained image frames.

[0159] The following section uses a foldable screen device as an example to introduce the one-to-one video call interface displayed on the receiving device:

[0160] In one embodiment, such as Figure 10 As shown in Figure A, the receiving device is a foldable screen phone, and the receiving device includes two displays. One of the displays, 1010, can display the second video call interface 320, and the other display, 1020, can display an interface of another application (such as a gallery application) (such as the gallery preview interface 1030). The interface displayed on each display is formed by superimposing multiple layers. In another embodiment, the receiving device may display other layers on the second video call interface 320, as detailed in the relevant description above.

[0161] In this embodiment, as Figure 10 As shown in B, if the receiving device directly calls the system's screen recording and screenshot interfaces to capture image frames, the acquired image frames will simultaneously include the content displayed on both screens. These image frames contain a significant amount of invalid data (such as data from the gallery preview interface displayed on screen 1020, and data from layers other than the application interface layer among the multiple layers constituting the second video call interface 320). This not only increases the computational load on the receiving device in subsequent processes but also reduces the accuracy of the video anti-counterfeiting detection results.

[0162] Therefore, in this embodiment, the receiving device only acquires the layer containing the application interface 820 of the video call application within the video call interface. In this document, the layer containing the application interface of the video call application is referred to as the target layer. That is, in this document, the layer used to display the caller's user image among the multiple layers constituting the video call interface is called the target layer. Based on this, not only can the computational load on the receiving device be reduced in subsequent processing, but it also helps to improve the accuracy of subsequent video anti-counterfeiting detection.

[0163] For example, such as Figure 11 As shown, even if the layer containing the message pop-up 910 is displayed above the video call interface, the first image frame acquired using the method provided in this application embodiment only includes the layer containing the caller's user image (i.e., the target layer).

[0164] The following describes the specific method for obtaining the first image frame in the embodiments of this application:

[0165] In one embodiment, the receiving device can invoke a virtual machine control and monitoring service to create a virtual display solely for displaying the target layer. In this case, the receiving device includes two displays: a virtual display and a primary display. The primary display corresponds to the receiving device's main display, and its content can be displayed on the receiving device's main display for user viewing. Content displayed on the virtual display is not shown on the receiving device's main display.

[0166] During the display process, the layer rendering compositing service can composite the layer containing the application interface separately and send it to the virtual screen. Simultaneously, the layer rendering compositing service can composite all layers, including the layer containing the application interface, and send them to the main screen. In this way, the virtual screen only displays the layer containing the application interface. Then, the receiving device can directly obtain the layer containing the application interface from the virtual screen, thus obtaining the first image frame.

[0167] In this embodiment, the number of application interface layers acquired by the receiving device from the virtual screen per unit time (i.e., the acquisition frequency) is related to the screen refresh rate of the receiving device. In this embodiment, the layer rendering and compositing service needs to synthesize two sets of display data simultaneously, one sent to the main screen and one sent to the virtual screen. This reduces the screen refresh rate of the receiving device, for example, from 120Hz to 60Hz. This can cause frame drops and stuttering on the receiving device. To solve this problem, this embodiment also proposes the following method to acquire the first image frame.

[0168] For example, when the screen refresh rate of the receiving device is 60Hz, the receiving device can obtain the layers of 60 application interfaces through the virtual screen per unit time, and thus the receiving device can obtain 60 first image frames per unit time.

[0169] In another embodiment, each layer that makes up an interface has a unique layer identifier (taskid). The capturelayers interface of the window management service can obtain layers with the identifier of the target layer (such as the layer where the application interface is located). Therefore, the receiving device can obtain the layer where the application interface that constitutes the video call interface is located through the capturelayers interface in the window management service, thereby obtaining the first image frame.

[0170] In this embodiment, the number of first image frames acquired by the receiving device per unit time is configurable. That is, the acquisition frequency can be set. Specifically, the capture layer interface acquires one first image frame each time it receives a target layer identifier. Therefore, the number of first image frames per unit time can be controlled by setting the frequency of occurrence of the target layer identifier. Furthermore, in this embodiment, the layer rendering and compositing service only needs to synthesize one set of display data and send it to the main screen, thus not reducing the screen refresh rate.

[0171] In one embodiment, the video anti-counterfeiting detection model can be a frame-by-frame detection-based anti-counterfeiting detection model. For a frame-by-frame detection video anti-counterfeiting detection model, the receiving device only needs to provide a sufficient number of image frames to be detected (such as the third image frame mentioned herein, which is obtained by processing the first image frame), and the various image frames to be detected do not need to be correlated. Therefore, the acquisition frequency of the first image frame by the receiving device can be arbitrarily set, as detailed below:

[0172] In one example, the receiving device acquires the first image frame at a capture frequency of the call video frame rate.

[0173] Typically, the screen refresh rate of electronic devices is higher than the video call frame rate. For example, the screen refresh rate is 120Hz, 60Hz, etc., while the video call frame rate is 15 frames per second (FPS), 30FPS, etc. Assuming the video call frame rate is 30FPS and the screen refresh rate is 120Hz, the content on the application interface will only change every four screen refreshes on the receiving device. To avoid losing the information to be detected (i.e., facial information) and to prevent duplicate acquisition of the first image frame, electronic devices can acquire the first image frame at the same frequency as the video call frame rate.

[0174] In another example, building upon the above example, to further reduce the number of first image frames, the acquisition frequency of the receiving device can be further reduced. For example, when the video call frame rate is 30 FPS, the acquisition frequency can be reduced from 30 FPS to 15 FPS.

[0175] In one embodiment, the screen display resolution of the electronic device is higher than the video call resolution. Here, the screen display resolution refers to the pixel density of the display screen. For example, the screen display resolution could be 2K, 1080P, etc. The video call resolution could be 720P, 540P, etc. The higher the image resolution, the more memory is required to store that image. Therefore, when the receiving device acquires the first image frame, it can normalize the resolution of the acquired original image frame to the video call resolution, thereby obtaining the first image frame. In other words, in this embodiment, the resolution of the first image frame is the video call resolution.

[0176] In one embodiment, the video anti-counterfeiting detection model requires a sufficient number of image frames to be detected in order to perform video anti-counterfeiting detection and ensure the accuracy of the detection results. Therefore, the number of first image frames acquired by the receiving device needs to be greater than or equal to the number of image frames to be detected. Throughout the acquisition process, the receiving device can determine the start time of acquiring the first image frame.

[0177] Furthermore, the method provided in this application also involves conditions for the receiving device to terminate the acquisition of the first image frame. The conditions for terminating the acquisition of the first image frame include normal termination conditions and abnormal termination conditions. Examples are given below:

[0178] The normal termination condition refers to the following: from the start of acquisition to the current time, the receiving device continuously displays the video call interface in full screen, and the first acquisition time is equal to the preset time.

[0179] Typically, the number of image frames acquired by an electronic device per unit time is fixed. Therefore, the number of first image frames acquired by the receiving device within a preset duration (which can be referred to as the first preset number) is fixed. In this embodiment, the first preset number can be much larger than the number of image frames to be detected. When the acquisition duration equals the preset duration, it indicates that the receiving device has acquired a sufficient number of first image frames. Therefore, the receiving device can stop the current acquisition process.

[0180] It should be noted that if the conditions for starting to acquire the first image frame are still met after the current acquisition process is completed, the receiving device can start the next acquisition process.

[0181] The abnormal termination conditions may include at least one of the following: the receiving device exits the video call interface at the current moment (or switches from the video call interface to another interface, thus no longer displaying the video call interface), the receiving device switches from full-screen display of the video call interface to small window display of the video call interface at the current moment, or the video call ends at the current moment, but are not limited to these. It should be noted that, in the abnormal termination conditions, the second acquisition duration from the start time of acquisition to the current time is less than the preset duration.

[0182] In one embodiment, such as Figure 12 As shown in Figure A, phone B displays the second video call interface 320 (i.e., the one-to-one video call interface) of the video call application. In response to switching operations (such as swiping up), as... Figure 12 As shown in B, mobile phone B switches from displaying the second video call interface 320 to displaying the main interface 1210.

[0183] In one embodiment, continue as follows Figure 12 As shown in Figure A, the second video call interface 320 includes a zoom control 1220. In response to a triggering operation on the zoom control 1220, such as... Figure 12 As shown in C, mobile phone B switches the second video call interface 320 from full-screen display to small window display.

[0184] In one embodiment, continue as follows Figure 12 As shown in Figure A, the second video call interface 320 includes a hang-up control 1230. In response to a trigger operation on the hang-up control 1230, such as... Figure 12 As shown in D, mobile phone B switches from the second video call interface to the chat interface 1240 (i.e., the one-to-one chat interface) with the caller.

[0185] In one embodiment, when the calling device ends the video call in response to the calling user triggering the hang-up control on the video call interface (such as the first video call interface) of the calling device, the receiving device will also end the current video call and switch from the video call interface to the chat interface with the calling user.

[0186] It should be noted that after the acquisition process ends, if the conditions for starting to acquire the first image frame are met again, the receiving device can start the next acquisition process.

[0187] Figure 13 This is a second schematic flowchart of an image processing method provided in an embodiment of this application. For example... Figure 13 As shown, the method may further include the following after S604:

[0188] S1301, The receiving device filters out the transparency (alpha) data of the first image frame to obtain the first image frame in RGB format.

[0189] Typically, the first image frame obtained directly from the video call interface (such as a one-on-one video call interface) by the receiving device is in RGBA format. That is, the first image frame obtained directly may include: data from the alpha (A) channel, the red (R) channel, the green (G) channel, and the blue (B) channel.

[0190] In this system, the transparency data of a first image frame (i.e., transparency data) indicates the transparency of the first image frame, while the RGB data indicates the color information of the first image frame. The color information may include facial information (i.e., information to be detected), background information, etc. Therefore, filtering out the transparency data of a first image frame by the receiving device will not affect the color information of the first image frame, and can also reduce the memory size of the first image frame.

[0191] For example, for the same image frame, an image frame in RGBA format occupies 7.91MB of memory, while an image frame in RGB format occupies 5.93MB of memory. Therefore, the RGBA format of the same image frame occupies nearly 2MB less memory than the RGB format.

[0192] S605. The receiving device performs face recognition on the first image frame to determine whether a first face region exists in the first image frame.

[0193] In this step, the first image frame can be a first image frame with RGBA format or a first image frame with RGB format. This application embodiment does not limit this.

[0194] Typically, whether a first image frame is in RGBA format or RGB format, it may or may not include a face. The electronic device involved in this application has a pre-installed face recognition algorithm to identify faces in image frames (such as the first image frame), thereby determining whether a face region (such as the first face region) exists in the image frame. This facilitates subsequent determination, through a video anti-spoofing detection model, whether the person in the video call with the receiving user is the actual caller.

[0195] In one embodiment, such as Figure 14As shown, the first image frame 1400 may include a first face region 1410 and a background region 1420. The first face region, also known as the region of interest (ROI), is the smallest region of the face. The first face region 1410 can be a rectangular region or other shapes, such as a circle or an ellipse. This paper uses a rectangular first face region 1410 as an example. The background region 1420, as discussed in this paper, refers to all regions in the first image frame 1400 other than the first face region 1410.

[0196] S606. If the first face region is not present in the first image frame, the receiving device will no longer process the first image frame.

[0197] If the first image frame does not contain a first face region, it indicates that the first image frame does not include the face of the receiving user, nor the face of the calling user. In this case, the first image frame does not meet the video anti-spoofing detection conditions. Therefore, the receiving device will no longer process this first image frame.

[0198] In one embodiment, in order not to occupy the receiving device's storage space (such as RAM), if a first image frame does not contain a first face region, the receiving device will delete the first image frame.

[0199] In one embodiment, if there are two or more (e.g., three) first face regions in the first image frame, the receiving device will no longer process the first image frame.

[0200] Specifically, if there are two or more first face regions in the first image frame, it indicates that the first image frame includes the face image of one or more receiving users and the face image of one calling user; or, the first image frame includes the face image of one receiving user and the face image of one or more calling users.

[0201] Typically, during a video call, the probability of the aforementioned security risky behavior occurring is low if the first image frame includes the face image of one receiving user and more than one calling user, or if the first image frame includes the face image of more than one receiving user and one calling user. In this case, a first image frame containing two or more first face regions does not meet the video anti-spoofing detection conditions, and therefore the receiving device will discard the first image frame.

[0202] S607. If a first face region exists in the first image frame, the receiving device adjusts the size of the first face region to obtain a second face region corresponding to the first face region.

[0203] Typically, the image frames input into the video anti-counterfeiting detection model need to meet the following requirements: the image frame includes a face region, the face region is surrounded by a suitable amount of background area, and the image frame size is the target size. Therefore, the receiving device adjusts the size of the first face region, which may include:

[0204] First, the receiving device needs to expand the first face region into the background region by a preset number of pixels to obtain the third face region corresponding to the first face region. The boundary of the third face region does not exceed the boundary of the first image frame. The following section details how to expand the first face region into the background region:

[0205] In one example, the receiving device can select appropriate preset pixels to expand the first face region into the background region according to the position of the first face region in the first image frame, thereby obtaining the third face region corresponding to the first face region.

[0206] The preset pixels may include a first preset pixel P1, a second preset pixel P2, and a third preset pixel P3. The second preset pixel P2 is smaller than the first preset pixel P1 and larger than the third preset pixel P3. The first preset pixel P1 and the third preset pixel P3 are fixed values. The size of the first preset pixel P1 is related to a preset ratio (e.g., 12.5%) and the size of the first face region.

[0207] For example, the first preset pixel P1 is the product of a preset ratio and the size of the first face region. The third preset pixel P3 can be 0 pixels. The second preset pixel P2 is related to the distance between a boundary of the first face region and a corresponding boundary of the first image frame. Examples are given below:

[0208] For example, such as Figure 15 As shown in Figure A, the distance D1 between the boundary A1 (such as the left boundary) in the first face region 1410 and the corresponding boundary B1 (such as the left boundary) in the first image frame is greater than the first preset pixel P1. Figure 15 As shown in B, the receiving device expands the boundary A1 of the first face region 1410 towards the boundary B1 of the first image frame by a first preset pixel P1.

[0209] For example, continuing as Figure 15 As shown in Figure A, the distance D2 between the boundary A2 in the first face region 1410 and the boundary B2 of the first image frame is the second preset pixel P2. Continuing as... Figure 15 As shown in B, the receiving device expands the boundary A2 of the first face region 1410 towards the boundary B2 of the first image frame by a second preset pixel P2.

[0210] For example, continuing as Figure 15As shown in Figure A, the distance D3 between the boundary A3 in the first face region 1410 and the boundary B3 of the first image frame is 0 pixels. Continuing as... Figure 15 As shown in B, the receiving device will not expand the boundary A3 of the first face region 1410 toward the boundary B3 of the first image frame; or, the receiving device will expand the boundary A3 of the first face region 1410 toward the boundary B3 of the first image frame by a third preset pixel P3 (i.e., 0 pixels).

[0211] Next, the receiving device needs to normalize the size of the third face region to the target size to obtain the second face region. In other words, the size of the second face region is the target size. The process of normalizing the size of the third face region is related to the size of the third face region (which can be simply referred to as the third size) and the size of the second face region. The following section details how to normalize the size of the third face region:

[0212] In one example, if the target size is smaller than the third size, the size normalization process refers to reducing the size of the third face region to the target size. For instance, the receiving device reduces the size of the third face region to the target size by downsampling.

[0213] In another example, if the target size is larger than the third size, the size normalization process refers to enlarging the size of the third face region to the target size. For example, the receiving device enlarges the size of the third face region to the target size by upsampling.

[0214] In another example, if the target size is equal to the third size, the receiving device will not normalize the size of the third face region. In this case, the third face region is the second face region.

[0215] In one embodiment, the receiving device will adjust the size of the first face region to obtain the second face region only when it determines that there is one or two first face regions in the first image frame.

[0216] Typically, during a video call, if the first image frame includes the face image of both the receiving user and the calling user, or if the first image frame only includes the face image of the calling user, the probability of the aforementioned security-risk behavior occurring is relatively high. In this case, the first image frame containing one or two first face regions meets the video anti-spoofing detection conditions. Therefore, the receiving device will retain the first image frame and resize the first face region in the first image frame to obtain the second face region.

[0217] Furthermore, if the caller is far from the camera of the calling device, the image of the caller captured by the camera of the calling device will be smaller. Similarly, if the receiver is far from the camera of the receiver device, the image of the receiver will also be smaller. Therefore, when at least one user is far from the camera of the corresponding electronic device, the image of the caller displayed on the receiver device will be smaller, and / or, the smaller image of the receiver will ultimately result in a smaller size of the face region (such as the first face region, the second face region, or the third face region) in the first image frame. Typically, the probability of video anti-spoofing detection in this scenario is extremely low. Even if video anti-spoofing detection is performed, the small face region results in less effective information, thus the accuracy of the detection results is low and has no reference value. Based on this, if the size of the face region in the first image frame is smaller than a size threshold (the first size threshold or the second size threshold), the receiver device will discard the first image frame. This will be explained in detail below:

[0218] In one embodiment, before resizing the first face region, the receiving device can determine whether the size of the first face region is greater than a first size threshold. If the size of a first face region is greater than the first size threshold, the receiving device will resize the first face region to obtain a second face region corresponding to the first face region. If the size of a first face region is less than the first size threshold, the receiving device will directly discard the first face region. In this way, the receiving device can avoid performing subsequent steps on the first face region, reducing the computational load on the receiving device and saving system resources.

[0219] In another embodiment, after obtaining the third face region by background augmentation of the first face region, the receiving device can determine whether the size of the third face region is greater than a second size threshold. If the size of a third face region is greater than the second size threshold, the receiving device continues to process that third face region. If the size of a third face region is less than the second size threshold, the receiving device directly discards that third face region. In this way, the receiving device can avoid performing subsequent steps on that third face region, reducing the computational load on the receiving device and saving system resources.

[0220] The second size threshold can be greater than the first size threshold. For example, the first size threshold can be 96 pixels × 96 pixels, and the second size threshold can be 120 pixels × 120 pixels.

[0221] S608, The receiving device crops the first image frame along the boundary of the second face region to obtain the second image frame.

[0222] Since the second image frame is obtained by cropping the first image frame along the boundary of the second face region, the size of the second image frame is the same as the size of the second face region, and the content of the second image frame is the same as the content of the second face region. Thus, the second image frame not only retains facial information and some background information, but also has a smaller size. Generally, the smaller the image frame size, the less storage space it occupies. Therefore, compared to the first image frame, the second image frame occupies less storage space on the receiving device.

[0223] Assuming one pixel corresponds to 4 bytes, and video anti-counterfeiting detection requires 120 image frames, with each image frame being 1920 pixels × 1080 pixels, the following section compares the total memory occupied by the 120 image frames obtained through conventional methods with the total memory occupied by the 120 first image frames (320 pixels × 320 pixels) obtained through the method provided in this application embodiment, in conjunction with Table 1.

[0224] Table 1

[0225]

[0226] The comparison shows that, for the same number of image frames, the total memory occupied by the first image frame obtained by the method provided in this application embodiment is 4.94% or even 3.70% of the total memory occupied by the image frame obtained by the conventional method.

[0227] Currently, most electronic devices have relatively small amounts of RAM, such as 6GB. Therefore, in conventional techniques, the total memory occupied by all image frames acquired would account for 15.45% of the electronic device's RAM. However, in this embodiment, the total memory occupied by all the first image frames acquired accounts for only 0.76% or 0.57% of the electronic device's RAM.

[0228] In summary, the method provided in this application provides a method for obtaining a certain number of first image frames with a relatively small total memory footprint. This ensures the smooth operation of the video anti-counterfeiting detection process without consuming excessive memory resources of the receiving device, thus not affecting the video call function or other application functions of the receiving device.

[0229] For example, the method described in this application embodiment will not cause malfunctions such as buffering or abnormal exit of the video call application on the receiving device during the video call. As another example, when the receiving device is a foldable screen device, with one display screen used for video calls and the other for watching live video, the method described in this application embodiment will not cause buffering or other malfunctions in the live video stream.

[0230] S609. The receiving device performs lossless compression on the second image frame to obtain the third image frame to be detected.

[0231] In this step, the receiving device can use a lossless compression algorithm to losslessly compress the second image frame frame by frame, thereby obtaining the third image frame to be detected. The third image frame is the image frame to be input into the video anti-counterfeiting detection model. The video anti-counterfeiting detection model is used for video anti-counterfeiting detection.

[0232] For example, lossless compression algorithms can be derived from the LZ77 data compression algorithm. For instance, portable network graphics (PNG) compression algorithms can achieve a compression rate of up to 30%. Based on this, taking 120 third image frames in RGB format as an example, the total memory occupied by all third image frames is 10.55 MB.

[0233] Compared with conventional compression algorithms, the lossless compression algorithm provided in this application has the advantages of not losing the detection information in the image frame and having higher compression efficiency. The compression method provided in this application will be compared with conventional compression methods below:

[0234] Conventional technologies using compression algorithms like H.264 / H.265 take into account the correlation between adjacent frames during the encoding (or compression) process, resulting in a longer encoding time. This leads to longer processing times for video anti-counterfeiting detection, preventing the receiving device from outputting detection results in real time and resulting in a poor user experience. Furthermore, considering the correlation between adjacent frames during encoding causes the loss of some detection information from the captured image frames, thus reducing the accuracy of the video anti-counterfeiting detection results.

[0235] The embodiment of this application employs a lossless compression algorithm, requiring only consideration of one second image frame. This results in high compression efficiency, enabling the receiving device to output the video anti-counterfeiting detection results in real time. Furthermore, due to the inherent characteristics of the lossless compression algorithm, no information to be detected in each second image frame is lost. This ensures the accuracy of the video anti-counterfeiting detection results.

[0236] S610, The receiving device counts the cumulative number of third image frames during this acquisition process.

[0237] In this step, the receiving device caches each third image frame it acquires for later input into the video anti-counterfeiting detection model. During this process, the receiving device updates the cumulative number of third image frames in a timely manner.

[0238] Typically, video anti-counterfeiting detection models require a sufficient number (i.e., reaching a threshold number) of third image frames to perform video anti-counterfeiting detection. Therefore, in order to perform video anti-counterfeiting detection, the receiving device needs to acquire a sufficient number of third image frames.

[0239] S611. If the cumulative number of third image frames reaches the number threshold, the receiving device will perform video anti-counterfeiting detection on all third image frames through the video anti-counterfeiting detection model.

[0240] After performing anti-spoofing detection on all third-party image frames, the video anti-spoofing detection model can output a confidence score. This confidence score can be used to indicate the probability that the person in the video call with the recipient is not the real caller.

[0241] For example, if the confidence level is greater than a first confidence level threshold, the receiving device can prompt the receiving user that the person being video-called by the receiving user is not the actual caller.

[0242] For example, if the confidence level is less than the first confidence level threshold and greater than the second confidence level, the receiving device will combine other relevant information (such as the confidence level in the next video call) to make a comprehensive judgment on whether the person being video-called by the receiving user is the real caller.

[0243] For example, if the confidence level is less than the second confidence level, the receiving device determines that the person in the video call with the receiving user is the actual caller. In this case, the receiving device may not need to take any action.

[0244] In one embodiment, even after the cumulative number of third image frames reaches a threshold, the acquisition process is not yet complete, meaning there are still other third image frames. At this point, the receiving device will not cache subsequent third image frames. This saves the receiving device's operating memory.

[0245] S612. If the cumulative number of third image frames does not reach the number threshold, the receiving device continues to acquire third image frames.

[0246] If the cumulative number of third image frames does not reach the threshold, the receiving device will continue to acquire third image frames until the cumulative number of third image frames reaches the threshold. Then, all third image frames will be input into the video anti-counterfeiting detection model for video anti-counterfeiting detection.

[0247] In one embodiment, if the cumulative number of third image frames does not reach the number threshold, but still meets the condition for the receiving device to acquire the first image frame, then the receiving device enters the next acquisition process to continue acquiring the first image frame and processes the first image frame to obtain the third image frame.

[0248] In one embodiment, the cumulative number of third image frames does not reach the threshold, but the condition for the receiving device to acquire the first image frame is not met. Therefore, the receiving device will not acquire the first image frame, and thus cannot acquire the third image frames. In this case, since the cumulative number of third image frames is insufficient, the receiving device will not perform video anti-counterfeiting detection this time. Additionally, the receiving device will delete all third image frames, thereby avoiding the use of the receiving device's operating memory.

[0249] The preceding text described the image processing method provided in this application's embodiments, using an electronic device (such as the receiving device) as the execution subject. In practice, this method requires the support of various hardware and software modules within the electronic device. The following text, in conjunction with... Figure 16 The image processing method provided in this application embodiment is introduced through the interaction of various software and hardware modules in the receiving device.

[0250] Figure 16 This is illustrated as a third schematic flowchart of an image processing method provided in an embodiment of this application. Figure 16 As shown, the method may include:

[0251] S01, The application layer's early warning module receives the whitelist sent by the server.

[0252] It should be noted that S01 does not necessarily mean it must be the first step executed. The server can send a whitelist to the electronic device. Upon receiving the whitelist, the electronic device's underlying communication module can transmit it to the application layer. The application layer's alerting module can receive the whitelist transmitted from the underlying communication module.

[0253] S02, The application layer's early warning module sends a subscription message to the application layer's perception fence.

[0254] The subscription message may include application information of video calling applications in the whitelist, which is used by the warning module to subscribe to events from the perception fence: the appearance of a video call interface of a video calling application in the whitelist.

[0255] S03. When the video call application in the application layer displays the video call interface, the video call application in the application layer sends the video call message to the perception fence in the application layer.

[0256] The video call message may include the name of the currently displayed video call interface and the name of the corresponding video call application.

[0257] Specifically, S03 corresponds to S602 above. Therefore, S03 can be referred to the relevant description in S602 above.

[0258] S04. In response to the answering operation on the video call interface, the video call application at the application layer switches from the video call interface to the video call interface.

[0259] Specifically, S04 corresponds to S603 above. Therefore, S04 can be referred to in the relevant description in S603 above.

[0260] S05. Based on the application layer's sensing fence, the interface name of the currently displayed video call interface and the application name of the corresponding video call application are retrieved from the subscription message. The application layer's sensing fence then sends a feedback message to the application layer's early warning module.

[0261] The feedback message indicates that the sensing fence has detected the subscribed event.

[0262] Specifically, when the sensing fence receives a video call message, it can query the subscription messages to see if the interface name of the currently appearing video call interface exists, as well as the application name of the corresponding video call application.

[0263] If the sensing fence finds the name of the currently appearing video call interface and the corresponding video call application name in the subscription messages, it indicates that the video call application is on the whitelist. Therefore, during a video call through this application, the receiving device can perform video anti-spoofing detection. At this time, the sensing fence can send a feedback message to the warning module.

[0264] Conversely, if the video call application is not on the whitelist, it indicates that the video call application is not included. Therefore, during a video call using this application, the receiving device does not need to perform video anti-spoofing detection. In this case, the sensing fence will not send a feedback message to the warning module.

[0265] It should be noted that the execution order of S04 and S05 is not important.

[0266] S06. The application layer's early warning module receives the camera status sent by the camera manager in the application framework layer.

[0267] The camera status can be either on or off. When the receiver's device camera is on, it can capture an image of the user. When the receiver's device camera is off, it cannot capture an image of the user.

[0268] In one embodiment, after the camera manager detects a change in the camera status, the camera manager will proactively send the camera status information to the warning module.

[0269] In another embodiment, after receiving the feedback message from the sensing fence, the early warning module can send a camera status query message to the camera manager. The camera status query message is used to query the camera status. Upon receiving the camera query message, the camera manager obtains the camera status and sends the camera status information to the early warning module.

[0270] S07. The application layer's early warning module receives the voice status sent by the application layer's audio manager.

[0271] The voice status can include a call status and a non-call status. When the voice status is in a call status, the receiving device has activated the voice call function. That is, the receiving device's microphone can capture the receiving user's voice, and the receiving device's speaker can play the calling user's voice.

[0272] In one embodiment, after the audio manager detects a change in the voice status, the audio manager will proactively send the voice status to the warning module.

[0273] In another embodiment, after receiving the feedback message from the sensing fence, the early warning module can send a voice status query message to the audio manager. The voice status query message is used to query the voice status. Upon receiving the voice status query message, the audio manager can obtain the voice status and send it to the early warning module.

[0274] S08. When a video call application in the whitelist displays a video call interface, the camera is on, and the voice is in call status, the application layer's early warning module sends a notification message to the application layer's acquisition module to obtain the target layer.

[0275] Typically, when a whitelisted video call application displays a video call interface with the camera on and the voice status indicating a call, the application will switch from the video call interface to the video call interface.

[0276] S09. The application layer acquisition module obtains the first image frame from the multiple layers that constitute the video call interface.

[0277] Specifically, S08 and S09 can correspond to S604 mentioned above.

[0278] In one embodiment, such as Figure 17 As shown in Figure A, the acquisition module 1710 sends a creation instruction message to the virtual machine control and monitoring service 1720. The creation instruction message instructs the virtual machine control and monitoring service 1720 to create a virtual screen. The creation instruction message may include the name (or identifier) ​​of the virtual screen. There is a mapping relationship between the name of the virtual screen and the layer identifier of the target layer. Therefore, the virtual screen can be used only to display the target layer.

[0279] In this embodiment, the receiving device includes two displays: a virtual screen 1730 and a main screen 1740. The layer rendering and compositing service 1750 composites all layers and sends the composite to the main screen 1740. The layer rendering and compositing service 1750 also composites the target layer and sends it to the virtual screen 1730. Next, the acquisition module 1710 acquires the target layer from the virtual screen 1730.

[0280] In this embodiment, the acquisition frequency of the target layer by the acquisition module is related to the screen refresh rate of the electronic device.

[0281] In another embodiment, such as Figure 17 As shown in B, the acquisition module 1710 sends the layer identifier of the target layer to the window management service 1760. The window management service 1760 obtains the target layer from all layers through the capture layer interface of the surface control. Then, the acquisition module 1710 obtains the target layer from the capture layer interface of the window management service 1760.

[0282] In this embodiment, the acquisition module can send the layer identifier of the target layer to the window management service at a preset acquisition frequency, thereby controlling the acquisition frequency of the target layer.

[0283] S10. The acquisition module of the application layer sends the first image frame to the algorithm module of the application layer.

[0284] The acquisition module obtains the first image frame and sends it to the algorithm module for processing.

[0285] S11. The application layer algorithm module performs face recognition on the first image frame to determine whether a first face region exists in the first image frame.

[0286] Specifically, S11 corresponds to S605 above. Therefore, S11 can be referred to in the relevant description in S605 above, and will not be repeated here.

[0287] S12. If the first face region does not exist in the first image frame, the application layer algorithm module will no longer process the first image frame.

[0288] Specifically, S10 corresponds to S606 above. Therefore, S12 can be referred to the relevant description in S606 above, and will not be repeated here.

[0289] S13. If a first face region exists in the first image frame, the algorithm module of the application layer adjusts the size of the first face region to obtain the second face region corresponding to the first face region.

[0290] Specifically, S13 corresponds to S607 above. Therefore, S13 can be referred to in the relevant description in S607 above, and will not be repeated here.

[0291] S14. The application layer algorithm module crops the first image frame along the boundary of the second face region to obtain the second image frame.

[0292] Specifically, S14 corresponds to S608 above. Therefore, S14 can be referred to in the relevant description in S608 above, and will not be repeated here.

[0293] It should be noted that before the condition for the receiving device to stop acquiring the first image frame is met, the acquisition module will continuously acquire the first image frame from the multiple layers constituting the video call interface, and send each acquired first image frame to the algorithm module for the aforementioned processing. In other words, S07-S14 will be executed in a loop until the condition for the receiving device to stop acquiring the first image frame is met.

[0294] S15. The application layer algorithm module performs lossless compression on the second image frame to obtain the third image frame to be detected.

[0295] Specifically, S15 corresponds to S609 mentioned above. Therefore, S15 can be referred to in the relevant description in S609 above, and will not be repeated here.

[0296] S16. The application layer algorithm module sends the third image frame to the application layer early warning module.

[0297] The early warning module includes a video anti-counterfeiting detection model, which is used to perform video anti-counterfeiting detection on a sufficient number of third image frames to determine whether the person having a video call with the receiving user is the real caller.

[0298] S17. The application layer's early warning module counts the cumulative number of third image frames.

[0299] Specifically, S17 corresponds to S610 mentioned above. Therefore, S17 can be referred to in the relevant description in S610 above, and will not be repeated here.

[0300] S18. If the cumulative number of third image frames reaches the quantity threshold, the application layer's early warning module will input all third image frames into the video anti-counterfeiting detection model for video anti-counterfeiting detection.

[0301] Specifically, S18 corresponds to S611 above. Therefore, S18 can be referred to in the relevant description in S611 above, and will not be repeated here.

[0302] S19. If the cumulative number of the third image frames does not reach the number threshold, the application layer's early warning module continues to acquire the third image frames.

[0303] Specifically, S19 corresponds to S612 mentioned above. Therefore, S19 can be referred to in the relevant description in S612 above, and will not be repeated here.

[0304] like Figure 18 As shown in the illustration, this application also provides a chip system. The chip system 1800 includes at least one processor 1801 and at least one interface circuit 1802. The at least one processor 1801 and the at least one interface circuit 1802 are interconnected via lines. The processor 1801 is used to support an electronic device in implementing the various steps in the above method embodiments, and the at least one interface circuit 1802 can be used to receive signals from other devices (e.g., memory) or to send signals to other devices (e.g., a communication interface). The chip system may include a chip and may also include other discrete devices.

[0305] This application also provides a computer storage medium including instructions that, when executed on the electronic device, cause the electronic device to perform the steps in the above method embodiments.

[0306] This application also provides a computer program product including instructions that, when executed on the electronic device, cause the electronic device to perform the steps in the method embodiments described above.

[0307] The technical effects of the chip system, computer storage medium, and computer program product are similar to those in the preceding method embodiments.

[0308] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0309] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0310] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0311] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0312] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located on one device or distributed across multiple devices. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0313] In addition, the functional modules in the various embodiments of this application can be integrated into one device, or each module can exist physically separately, or two or more modules can be integrated into one device.

[0314] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted from one computer storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage medium can be any available medium accessible to a computer or a data storage device including one or more servers, data centers, etc., that can be integrated with the medium. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0315] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: Acquire a first image frame; wherein the first image frame is an image frame corresponding to a target layer that constitutes the video call interface, and the target layer is used to display the image of the caller user. Based on the existence of a first face region in the first image frame, the size of the first face region is adjusted to obtain a second face region corresponding to the first face region; the size of the second face region is the target size; The first image frame is cropped along the boundary of the second face region to obtain a second image frame corresponding to the first image frame; wherein, the second image frame is used to identify the authenticity of the caller user corresponding to the caller user image in the video call; The step of adjusting the size of the first face region to obtain the second face region corresponding to the first face region includes: The first face region is expanded into the background region to obtain a third face region corresponding to the first face region; based on the size relationship between the size of the third face region and the target size, the size of the third face region is adjusted to obtain a second face region corresponding to the third face region; wherein, the boundary of the third face region does not exceed the boundary of the first image frame, and the background region is the region in the first image frame other than the first face region; The step of expanding the first face region into the background region includes at least one of the following: Based on the fact that the distance between the boundary of the first face region and the corresponding boundary in the first image frame is greater than a first preset pixel, the boundary of the first face region is extended to the boundary of the first image frame by the first preset pixel. Based on the distance between the boundary of the first face region and the corresponding boundary in the first image frame as a second preset pixel, the boundary of the first face region is extended to the boundary of the first image frame by the second preset pixel; wherein, the second preset pixel is greater than 0 and the second preset pixel is less than the first preset pixel; Since the distance between the boundary of the first face region and the corresponding boundary in the first image frame is 0, the boundary of the first face region does not extend to the boundary of the first image frame.

2. The method according to claim 1, characterized in that, The size of the third face region is a third size; adjusting the size of the third face region based on the size relationship between the third size and the target size to obtain the second face region corresponding to the third face region includes: Based on the fact that the third size is larger than the target size, the size of the third face region is reduced to the target size by downsampling to obtain the second face region; or, Based on the fact that the third size is smaller than the target size, the size of the third face region is expanded to the target size by upsampling to obtain the second face region.

3. The method according to claim 1 or 2, characterized in that, The step of adjusting the size of the third face region based on the size relationship between the size of the third face region and the target size to obtain the second face region corresponding to the third face region includes: If the size of the third face region is greater than the second size threshold, the size of the third face region is adjusted based on the size relationship between the size of the third face region and the target size to obtain the second face region corresponding to the third face region.

4. The method according to claim 1 or 2, characterized in that, The step of adjusting the size of the first face region to obtain the second face region corresponding to the first face region includes: Based on the fact that the size of the first face region is greater than the first size threshold, the size of the first face region is adjusted to obtain the second face region corresponding to the first face region.

5. The method according to claim 1 or 2, characterized in that, Before adjusting the size of the first face region based on the existence of a first face region in the first image frame to obtain a second face region corresponding to the first face region, the method further includes: The transparency data of the first image frame is filtered out to obtain the first image frame in RGB format.

6. The method according to claim 1 or 2, characterized in that, The presence of a first face region in the first image frame means that there is one or two first face regions in the first image frame.

7. The method according to claim 1 or 2, characterized in that, The resolution of the first image frame is the same as that of a video call.

8. The method according to claim 1 or 2, characterized in that, The acquisition of the first image frame includes: The first image frame is acquired at the video call frame rate of the electronic device as the acquisition frequency.

9. The method according to claim 1 or 2, characterized in that, The method further includes: The second image frame is losslessly compressed to obtain the third image frame to be detected; wherein, the third image frame is used as input to the video anti-counterfeiting detection model.

10. The method according to claim 9, characterized in that, The method further includes: Count the cumulative number of the third image frames acquired this time; If the cumulative number reaches the number threshold, then all third image frames are input into the video anti-counterfeiting detection model for video anti-counterfeiting detection. If the cumulative number does not reach the number threshold, the third image frame is then acquired.

11. An electronic device, characterized in that, It includes one or more processors, and a display screen, camera, speaker, microphone, and memory coupled to the processors; the display screen is used to display a video call interface and a video call interface; the camera is used to capture an image of the call subject; the speaker is used to play sound; The microphone is used to collect sound; The memory stores computer program code, which includes instructions; when the instructions are executed by the processor, the electronic device performs the method of any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-10.

13. A computer program product, characterized in that, Includes instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-10.

Images