Data transmission method and electronic device

Abstract

The present application discloses a data transmission method and an electronic device, the method comprising: in the case of a video call, at a first time point, sending the first video data of a first user to a second user; the first video data includes multiple user identification information, and the second user is a user other than the first user in the video call; at a second time point, in response to the operation of turning on the privacy protection function, based on the first video data including multiple user identification information, stop sending the first video data to the second user; in the case of turning on the privacy protection function, at a third time point, based on the second video data including one user identification information, and the one user identification information matches the pre-saved first user identification information, send the second video data to the second user; the second time point is later than the first time point, and the third time point is later than the second time point. Through the present application, privacy leakage can be avoided and power consumption can be saved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular to a data transmission method and an electronic device. Background Art

[0002] When a user makes a video call (such as a multi-party video conference) via a mobile phone, the following situations may occur:

[0003] (1) When a user is talking to someone on the spot and does not turn off the microphone, the content of the conversation will be transmitted to other users online;

[0004] (2) If the user leaves the room but does not turn off the microphone, and someone else is talking to him / her, the user's voice will be transmitted to other users online.

[0005] (3) When the user leaves but the camera is not turned off, a person beside him / her appears on the screen. At this time, the image content of the person beside him / her will be transmitted to other users online.

[0006] When these situations occur, privacy leakage and power consumption will be wasted. Summary of the invention

[0007] The present application provides a data transmission method and an electronic device, which can avoid privacy leakage and save power consumption.

[0008] In a first aspect, the present application provides a data transmission method, the method comprising: in the case of a video call, at a first time point, sending first video data of a first user to a second user; the first video data includes multiple user identification information, and the second user is a user other than the first user in the video call; at a second time point, in response to an operation of turning on a privacy protection function, based on the fact that the first video data includes multiple user identification information, stopping sending the first video data to the second user; when the privacy protection function is turned on, at a third time point, based on the fact that the second video data includes one user identification information, and the one user identification information matches the pre-saved first user identification information, sending the second video data to the second user; the second time point is later than the first time point, and the third time point is later than the second time point.

[0009] Based on the method described in the first aspect, after the privacy protection function is turned on, the video data will only be sent to other users in the video call when there is only one user identification information in the video data and the one user identification information matches the pre-saved first user identification information, thereby preventing the conversation audio and facial images of surrounding users at the call site (such as surrounding users at the call site of the first user) from being exposed to other users in the video call, thereby avoiding privacy leakage and saving power consumption.

[0010] In a possible implementation, the user identification information includes voiceprint information and / or facial image information.

[0011] In a possible implementation, the user identification information includes human voiceprint information, and based on the first video data including multiple user identification information, stopping sending the first video data to the second user includes: based on the loudness of the first audio data in the first video data being greater than or equal to a first preset threshold, and the first audio data including multiple human voiceprint information, stopping sending the first audio data to the second user. Based on this approach, privacy leakage can be effectively avoided.

[0012] In a possible implementation, based on the second video data including user identification information, and the user identification information matches the pre-stored first user identification information, sending the second video data to the second user includes: based on the loudness of the second audio data in the second video data being greater than or equal to the first preset threshold, the second audio data including a human voiceprint information, and the human voiceprint information matches the pre-stored first human voiceprint information, sending the second audio data to the second user. Based on this approach, privacy leakage can be effectively avoided.

[0013] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being less than a first preset threshold, not retaining the third audio data; and the fourth time point is later than the third time point. Based on this method, power consumption can be effectively saved.

[0014] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being greater than or equal to the first preset threshold and the third audio data not having human voiceprint information, not retaining the third audio data; and the fourth time point is later than the third time point. Based on this method, power consumption can be effectively saved.

[0015] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being greater than or equal to the first preset threshold, the third audio data includes a human voiceprint information, and the human voiceprint information does not match the pre-saved first human voiceprint information, the third audio data is not retained; and the fourth time point is later than the third time point. Based on this method, power consumption can be effectively saved.

[0016] In a possible implementation, the user identification information includes facial image information, and based on the first video data including multiple user identification information, stopping sending the first video data to the second user includes: based on the first image data in the first video data including multiple facial image information, stopping sending the first image data to the second user. Based on this approach, privacy leakage can be effectively avoided.

[0017] In a possible implementation, based on the second video data including user identification information, and the user identification information matches the pre-stored first user identification information, sending the second video data to the second user includes: based on the second image data in the second video data including a face image information, and the face image information matches the pre-stored first face image information, sending the second image data to the second user. Based on this approach, privacy leakage can be effectively avoided.

[0018] In a possible implementation, the method further includes: when the privacy protection function is enabled, at a fourth time point, based on the third image data in the third video data not having any facial image information, not retaining the third image data. Based on this approach, power consumption can be effectively saved.

[0019] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, based on the fact that the third image data in the third video data includes a face image information, and the face image information does not match the pre-stored first face image information, the third image data is not retained. Based on this method, power consumption can be effectively saved.

[0020] In a possible implementation, the method further includes: when the number of repetitions of the first user's vocal content reaches a second preset threshold within a preset time period, turning off the privacy protection function; or when a sound event that turns off the privacy protection function is detected, turning off the privacy protection function. Based on this method, a rollback mechanism for the privacy protection function can be implemented, that is, when the privacy protection function fails, the privacy protection function can be automatically turned off to ensure the normal conduct of the video call.

[0021] In a possible implementation, the method further includes: displaying a video call interface; in response to pulling down the video call interface, displaying a notification bar; the notification bar including an activation button for the privacy protection function; and in response to clicking the activation button, activating the privacy protection function. Based on this method, the flexibility of activating the privacy protection function can be achieved.

[0022] In a second aspect, the present application provides a data transmission device, which may be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device; wherein the data transmission device may also be a chip system, and the data transmission device may execute the method executed by the electronic device in the first aspect. The functions of the data transmission device may be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units corresponding to the above functions. The unit may be software and / or hardware. The operations and beneficial effects performed by the data transmission device may refer to the methods and beneficial effects described in the first aspect above, and the repetitive parts will not be repeated.

[0023] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the data transmission method in any possible implementation of the first aspect.

[0024] In a fourth aspect, the present application provides a data transmission device, which includes a function or unit for executing any method as described in the first aspect.

[0025] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the data transmission method in any possible implementation of the first aspect.

[0026] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the data transmission method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A It is a schematic diagram of a video call (multi-party video conference) provided in an embodiment of the present application;

[0028] Figure 1B is a schematic diagram of another video call (multi-party video conference) provided in an embodiment of the present application;

[0029] Figure 1C is a schematic diagram of another video call (multi-party video conference) provided in an embodiment of the present application;

[0030] Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0031] Figure 3 It is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0032] Figure 4 It is a flowchart of a data transmission method provided by an embodiment of the present application;

[0033] Figure 5A is a schematic diagram of another video call (multi-party video conference) provided in an embodiment of the present application;

[0034] Figure 5B is a schematic diagram of another video call (multi-party video conference) provided in an embodiment of the present application;

[0035] Fig. 6A is a schematic diagram of a pull-down video call interface provided in an embodiment of the present application;

[0036] Figure 6B It is a schematic diagram of a notification bar and privacy protection function provided in an embodiment of the present application;

[0037] Figure 6C is a schematic diagram of another video call (multi-party video conference) provided in an embodiment of the present application;

[0038] Figure 7 It is a flowchart of another data transmission method provided in an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of a flow chart of detecting audio data in the case of a video call or a voice call provided by an embodiment of the present application;

[0040] Fig. 9 It is a flowchart of another data transmission method provided in an embodiment of the present application;

[0041] Fig.10 It is a schematic diagram of a process of detecting image data in the case of a video call or a voice call provided by an embodiment of the present application;

[0042] Fig.11 It is a flowchart of another data transmission method provided in an embodiment of the present application;

[0043] Fig.12 It is a schematic diagram of a flow chart of detecting audio data and image data in the case of a video call or a voice call provided by an embodiment of the present application;

[0044] Fig.13 It is a structural schematic diagram of a data transmission device provided in an embodiment of the present application;

[0045] Fig.14It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0047] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0048] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of a terminal device.

[0049] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0050] 1. Voice calls and multi-party voice conferences

[0051] Voice call is a communication method through voice and transmission media. It is a technology that connects two or more people through a communication network for real-time communication. It allows both parties to hear each other's voice during the call, thus achieving long-distance communication. With the development of network technology, multi-party voice calls can now be realized, that is, multi-party, remote, real-time online voice communication can be achieved. It is mainly used in business voice talks, and the typical implementation method is multi-party voice conferencing.

[0052] 2. Video calls and multi-party video conferences

[0053] Video calling is a communication method based on the Internet and mobile Internet terminals, which transmits people's voice and images (user's facial images, photos, objects, etc.) in real time between mobile phones. With the development of network technology, multi-party video calling can now be realized, that is, multi-party, remote and real-time online video communication can be realized. It is mainly used in business video conferences, and the typical implementation method is multi-party video conferencing. Specifically, in a video call, the electronic device collects the user's image data through the camera and then sends it to other users in the video call; collects the user's audio data through the microphone and then sends it to other users in the video call. Of course, if the camera is turned off during a video call, the image data cannot be collected at this time; if the microphone is turned off during a video call, the audio data cannot be collected at this time.

[0054] When the first user makes a video call (such as a multi-party video conference) via a mobile phone, the following situations may occur:

[0055] (1) Figure 1A As shown, the first user is talking to surrounding users at the call site without turning off the microphone, and the content of the conversation will be transmitted to other users on the line (ie, second user 1, second user 2, and second user 3).

[0056] (2) Figure 1B As shown, the first user leaves without turning off the microphone, and the surrounding users at the first user's call site are having a conversation. At this time, the voices of the surrounding users will be transmitted to other users on the line (i.e., second user 1, second user 2, and second user 3).

[0057] (3) Figure 1C As shown in (a), the first user leaves, but the camera is not turned off, and the surrounding users at the first user's call site appear on the screen. At this time, the image content of the surrounding users will be transmitted to other users on the line (i.e., second user 1, second user 2, and second user 3). Or, Figure 1CAs shown in (b), the camera is not turned off, and the first user and surrounding users at the first user's call site appear on the screen together. At this time, the image content of the surrounding users will be transmitted to other users on the line (i.e., second user 1, second user 2, and second user 3); if the first user is still conversing with surrounding users at the first user's call site at this time, the conversation content will also be transmitted to other users on the line.

[0058] When these situations occur, privacy leakage and power consumption will be wasted.

[0059] Therefore, in order to avoid privacy leakage and save power consumption during video calls or voice calls, the present application provides a data transmission method and an electronic device. In a specific implementation, the above-mentioned data processing method can be performed by an electronic device 100. Among them, the electronic device 100 can be a mobile phone, a tablet computer, a laptop computer, or a wearable electronic device with wireless communication function (such as a smart watch), etc., but is not limited to this. The electronic device 100 is configured with a display screen and can be installed with a preset application (application, APP), such as a call APP. Users can make multi-user video calls or voice calls through the call APP.

[0060] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0061] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0062] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0063] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0064] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0065] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The processor 110 calls the instructions or data stored in the memory, so that the electronic device 100 executes the data transmission method executed by the electronic device in the following method embodiment.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0067] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0068] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0069] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0070] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0071] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0072] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0073] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0074] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0075] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0076] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0077] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0078] NPU is a neural-network (NN) computing processor. It can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0079] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0080] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as audio data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0081] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0082] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0083] Speaker 170A, also known as "speaker", is used to convert audio electrical signals into sound signals. Receiver 170B, also known as "earpiece", is used to convert audio electrical signals into sound signals. Microphone 170C, also known as "microphone" and "microphone", is used to convert sound signals into electrical signals. Headphone interface 170D is used to connect wired headphones. Pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, pressure sensor 180A can be set on display screen 194. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Air pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes Hall sensor. Acceleration sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Proximity light sensor 180G can include, for example, light emitting diode (LED) and light detector. Ambient light sensor 180L is used to sense ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0084] In addition, an operating system is running on the above components. For example, operating systems such as iOS and Android. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0085] Figure 3 It is a software structure block diagram of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiment of the present application, the operating system (taking the Android system, the Android system running on the AP as an example) can be divided into six layers, from top to bottom, respectively, the application layer (application, APP), the application framework layer (framework, FWK), the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware abstraction layer, HAL), the kernel layer and the hardware layer.

[0086] The application layer may include a series of application packages. Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. The application layer may also include system UI (systemUI), which is used to display the interface of the electronic device, such as the video call interface, etc. In the embodiment of the present application, the call refers to the call application, and the call application may have call functions such as voice call, multi-party voice conference, video call, multi-party video conference, etc.

[0087] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc., and the embodiments of the present application do not impose any restrictions on this.

[0088] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0089] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0090] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0091] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0092] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0093] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0094] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0095] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0096] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0097] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0098] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0099] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats.

[0100] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0101] A 2D graphics engine is a drawing engine for 2D drawings.

[0102] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. The hardware abstraction layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware. Figure 3 As shown, the hardware abstraction layer may include Wi-Fi HAL, audio HAL, camera HAL, etc. Among them, the camera HAL is the core software framework of the Camera.

[0103] The kernel layer is the layer between hardware and software. It is the core of an operating system and the first layer of software expansion based on hardware. It provides the most basic functions of the operating system and is the basis for the operation of the operating system. It is responsible for managing the system's processes, memory, device drivers, files, and network systems, and determines the performance and stability of the system. The kernel layer can include display drivers, audio drivers, camera drivers, sensor drivers, etc. Among them, the camera driver is the driver layer of the Camera device, which is mainly responsible for interacting with the hardware.

[0104] The hardware layer includes a display, a camera, a sensor, etc., and the embodiments of the present application do not impose any restrictions on this.

[0105] Based on the above, a data transmission method provided by an embodiment of the present application is further described in detail below. Figure 4 As shown, the data transmission method includes the following steps S401 to S403. Figure 4 The method execution subject shown can be the electronic device mentioned above. Or, Figure 4 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 4 The method is described by taking an electronic device as an example of an execution subject.

[0106] S401. In case of a video call, at a first time point, an electronic device sends first video data of a first user to a second user.

[0107] In an embodiment of the present application, the first video data includes multiple user identification information, and the second user is other users in the video call except the first user.

[0108] It can be understood that, Figure 5A As shown, when a first user uses a call APP (i.e., an application capable of making video calls and / or voice calls) in an electronic device to make a video call (such as a multi-party video conference) with a second user (such as second user 1, second user 2, and second user 3), at a first time point, surrounding users at the first user's call site appear on the first user's screen. At this time, the image data captured by the camera in the first video data contains facial image information of multiple users; Figure 5B As shown, the first user is talking to surrounding users at the scene of the first user's call. At this time, the audio data collected by the microphone in the first video data contains voiceprint information of multiple users. The voiceprint information and / or facial image information of multiple users mentioned here can be considered as information that can identify the user, that is, multiple user identification information. Since the first user did not turn on the privacy protection function in the call APP, the first video data will be exposed to other users in the video call except the first user, that is, the electronic device sends the first video data of the first user to the second user.

[0109] In a possible implementation, the user identification information includes voiceprint information and / or face image information. The following is a specific description for different situations:

[0110] Case 1: The user identification information includes voiceprint information.

[0111] In a specific implementation, the so-called human voiceprint information refers to the general term for the voice features contained in the speech that can characterize and identify the speaker, as well as the voice model established based on these features (parameters). In the process of learning to speak, people will gradually form their own voiceprint features by imitating the way different people around them speak. Therefore, theoretically speaking, voiceprints are like fingerprints. Different people have different voiceprint features, and human voiceprint information can be used to identify users. In an embodiment of the present application, the electronic device can use personalized voiceactivity detection (PVAD) technology to extract the human voiceprint information in the first video data. At this time, since multiple human voiceprint information is extracted from the first video data, it can be considered that the first video data includes multiple user identification information.

[0112] Case 2: The user identification information includes facial image information.

[0113] In a specific implementation, the so-called facial image information refers to the facial features of the user. Different people have different facial information, and facial image information can be used for identity recognition. In an embodiment of the present application, the electronic device can use facial recognition technology to extract facial image information from the first video data. At this time, since multiple facial image information is extracted from the first video data, it can be considered that the first video data includes multiple user identification information. It should be noted that the camera can only collect facial image information when the user is facing the screen. If the user only appears sideways on the screen, he cannot be recognized.

[0114] Case 3: The user identification information includes voiceprint information and face image information.

[0115] In a specific implementation, the electronic device can use voice print detection technology and facial recognition technology to extract voice print information and facial image information from the first video data. This can be considered a combination of situation 1 and situation 2 above. Multiple voice print information and facial image information corresponding to each voice print information can be extracted from the first video data. Therefore, it can be considered that the first video data includes multiple user identification information.

[0116] S402. At a second time point, the electronic device responds to the operation of turning on the privacy protection function and stops sending the first video data to the second user based on the fact that the first video data includes multiple user identification information.

[0117] In the embodiment of the present application, the second time point is later than the first time point. At the second time point, the user turns on the privacy protection function in the call APP, and the electronic device detects whether the first video data contains multiple user identification information. If the first video data contains multiple user identification information, then the first video data will stop being sent to the second user to avoid privacy leakage and save power consumption.

[0118] In one possible implementation, the method also includes: the electronic device displays a video call interface; the electronic device displays a notification bar in response to pulling down the video call interface; the notification bar includes an activation button for the privacy protection function; and the electronic device activates the privacy protection function in response to clicking the activation button.

[0119] The implementation method for users to enable the privacy protection function can be: Fig. 6A As shown, when the first user uses the call APP in the electronic device to make a video call with the second user (i.e., the second user 1, the second user 2, and the second user 3), the electronic device will display a video call interface; the first user can pull down the video call interface, and a notification bar will pop up; as shown in FIG. Figure 6BAs shown, the notification bar has a privacy protection function on button and a privacy protection function off button; the first user clicks the on button to turn on the privacy protection function. Of course, the user can also click the off button at any time to turn off the privacy protection function.

[0120] It should be noted that since the camera can only collect facial image information when the user is facing the screen, if the user only appears on the screen from the side or back, the electronic device cannot recognize it. Therefore, when the privacy protection function is turned on, the video data can still contain the side or back images of the surrounding users, which will not affect privacy leakage.

[0121] S403. When the privacy protection function is turned on, at a third time point, based on the second video data including a user identification information, and the user identification information matches the pre-saved first user identification information, the electronic device sends the second video data to the second user.

[0122] In the embodiment of the present application, the third time point is later than the second time point. The pre-stored first user identification information here refers to the identification information of the first user collected in advance by the electronic device, such as the first voiceprint information of the first user, the first face image information of the first user, etc.

[0123] When the privacy protection function is turned on, Figure 6C As shown, at the third time point, the surrounding users at the first user's call site leave the first user's screen. At this time, the image data collected by the camera in the second video data only contains the facial image information of one user, and the facial image information of the one user matches the pre-saved first facial image information; the first user stops talking with the surrounding users at the first user's call site. At this time, the audio data collected by the microphone in the second video data only contains the voiceprint information of one user, and the voiceprint information of the one user matches the pre-saved first voiceprint information. In this case, the second video data is allowed to be exposed to other users in the video call except the first user, that is, the electronic device can send the second video data to the second user. It can be understood that at the third time point, since the second video data only contains the identification information of the first user, it does not involve the leakage of other users' privacy information, and the second video data can be sent to the second user.

[0124] In a possible implementation, the method further includes: when the electronic device detects that the number of repetitions of the first user's vocal content reaches a second preset threshold within a preset time period, turning off the privacy protection function; or, when a sound event for turning off the privacy protection function is detected, turning off the privacy protection function. It can be understood that for the following two scenarios, the electronic device needs to execute a rollback mechanism for the privacy protection function, that is, when the privacy protection function fails, the electronic device needs to automatically turn off the privacy protection function to ensure the normal conduct of the video call.

[0125] Scenario 1: The electronic device will monitor the number of repetitions of the first user's vocal content in real time within a preset time period (i.e., the first preset time period). When the number of repetitions of the first user's vocal content reaches a second preset threshold, it indicates that the privacy protection function may have failed. For example, when the electronic device cannot send the first user's video data to the second user, the second user will prompt the first user that he cannot hear the first user's voice or cannot see the first user's video screen at this time, and cannot make a video call with the first user. Then the first user will repeat the speaking content. When the electronic device detects that the number of repetitions of the first user's vocal content reaches the second preset threshold, it will automatically turn off the privacy protection function to ensure the normal progress of the video call.

[0126] Scenario 2: During a video call, when the electronic device detects a sound event that turns off the privacy protection function, it will execute the operation of turning off the privacy protection function. The sound event here can refer to a voice, a sound effect, etc., which is not limited here. For example, the first user directly says the voice "turn off the privacy protection function", and the electronic device detects the voice content and executes the operation of turning off the privacy protection function.

[0127] In a possible implementation, the method further includes: after the electronic device turns off the privacy protection function, within a second preset time period, based on the presence of multiple user identification information in the fourth video data, displaying a prompt box; the prompt box includes a confirmation button and prompt information, the prompt information prompting whether to re-enable the privacy protection function; the electronic device responds to the operation of clicking the confirmation button and re-enables the privacy protection function. In other words, after the privacy protection function is automatically turned off, the electronic device can still continue to monitor the video data, and multiple user identification information has always existed in the fourth video data within the second preset time period. In order to ensure privacy security, the electronic device can prompt the user to restart the privacy protection function.

[0128] It can be seen that based on the method described above, in the case of a video call, at a first time point, the electronic device sends the first video data of the first user to the second user, and the first video data includes multiple user identification information; at a second time point, the electronic device responds to the operation of turning on the privacy protection function, and since the first video data includes multiple user identification information, the electronic device needs to stop sending the first video data to the second user; in the case of turning on the privacy protection function, at a third time point, since the second video data includes one user identification information, and the one user identification information matches the pre-saved first user identification information, the electronic device can send the second video data to the second user. It can be understood that after turning on the privacy protection function, only when there is one user identification information in the video data, and the one user identification information matches the pre-saved first user identification information, will the video data be sent to other users in the video call, so that the conversation audio and facial images of the surrounding users at the call site (such as the surrounding users at the call site of the first user) will not be exposed to other users in the video call, thereby avoiding privacy leakage and saving power consumption.

[0129] The following describes in detail the data transmission method of the user identification information in different situations.

[0130] 1. The user identification information includes voiceprint information.

[0131] like Figure 7 As shown, the data transmission method includes the following steps S701 to S703. Figure 7 The method execution subject shown can be the electronic device mentioned above. Or, Figure 7 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 7 The method is described by taking an electronic device as an example of an execution subject.

[0132] S701. In case of a video call, at a first time point, an electronic device sends first video data of a first user to a second user.

[0133] In an embodiment of the present application, the first video data includes multiple voiceprint information of human voices, and the second user is a user other than the first user in the video call. Specifically, the first video data includes first audio data, and the first audio data includes multiple voiceprint information of human voices. At a first time point, the electronic device sends the first video data of the first user to the second user, which can also be understood as, at the first time point, the electronic device sends the first audio data of the first user to the second user.

[0134] It should be noted that when the first user uses the call APP in the electronic device (i.e., an application that can make video calls and / or voice calls) to make a video call (such as a multi-party video conference) with the second user, at the first time point, the surrounding users at the call site of the first user are talking, and at this time, the first audio data collected by the microphone in the first video data contains multiple human voiceprint information. Since the first user has not turned on the privacy protection function in the call APP, the first audio data in the first video data will be exposed to other users in the video call except the first user, that is, the electronic device sends the first video data of the first user to the second user. Similarly, when the first user uses the call APP in the electronic device to make a voice call (such as a multi-party voice conference) with the second user, at the first time point, the electronic device may also expose the first audio data to other users in the voice call except the first user.

[0135] S702. At a second time point, the electronic device responds to the operation of turning on the privacy protection function, and stops sending the first audio data to the second user based on the fact that the loudness of the first audio data in the first video data is greater than or equal to the first preset threshold and the first audio data includes multiple human voiceprint information.

[0136] In an embodiment of the present application, the second time point is later than the first time point. At the second time point, the user turns on the privacy protection function in the call APP. At this time, the electronic device will detect whether the loudness of the first audio data in the first video data is greater than or equal to the first preset threshold, and will also detect whether the first audio data contains multiple human voiceprint information. If the loudness of the first audio data in the first video data is greater than or equal to the first preset threshold, and the first audio data includes multiple human voiceprint information, then the electronic device will stop sending the first audio data to the second user to avoid privacy leakage and save power consumption.

[0137] S703. When the privacy protection function is turned on, at a third time point, the electronic device sends the second audio data to the second user based on the fact that the loudness of the second audio data in the second video data is greater than or equal to the first preset threshold, the second audio data includes a human voiceprint information, and the human voiceprint information matches the pre-saved first human voiceprint information.

[0138] In the embodiment of the present application, the third time point is later than the second time point. The pre-stored first person's voiceprint information here refers to the first user's voiceprint information pre-collected by the electronic device.

[0139] When the privacy protection function is turned on, at the third time point, the surrounding users at the first user's call site stop talking. At this time, the loudness of the second audio data collected by the microphone in the second video data is greater than or equal to the first preset threshold, indicating that the second audio data is valid audio data, excluding small sounds or noises; in addition, the second audio data only contains one person's voiceprint information, and the voiceprint information of the person matches the pre-saved first person's voiceprint information. In this case, the electronic device allows the second audio data in the second video data to be exposed to other users in the video call except the first user, that is, the electronic device can send the second audio data to the second user. It can be understood that at the third time point, since the second audio data only contains the voiceprint information of the first user, it does not involve the leakage of other users' privacy information, and the second audio data can be sent to the second user.

[0140] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, the electronic device does not retain the third audio data based on the loudness of the third audio data in the third video data being less than a first preset threshold, wherein the fourth time point is later than the third time point.

[0141] It can be understood that, when the privacy protection function is turned on, at the fourth time point, the electronic device further collects the third video data, and detects that the loudness of the third audio data collected by the microphone in the third video data is less than the first preset threshold, indicating that the third audio data is invalid audio data, that is, a sound with a low volume or noise, so there is no need to retain the third audio data to save power consumption.

[0142] In one possible implementation, the method also includes: when the privacy protection function is turned on, at a fourth time point, the electronic device does not retain the third audio data based on the fact that the loudness of the third audio data in the third video data is greater than or equal to the first preset threshold and the third audio data does not contain human voiceprint information.

[0143] It can be understood that, when the privacy protection function is turned on, at the fourth time point, the electronic device further collects the third video data, and detects that the loudness of the third audio data collected by the microphone in the third video data is greater than or equal to the first preset threshold, indicating that the third audio data is valid audio data, and small sounds or noises are excluded; however, through the human voiceprint detection technology, it is detected that there is no human voiceprint information in the third audio data, indicating that no one is speaking at this time, and there is no need to send the third audio data to the second user. Therefore, there is no need to retain the third audio data to save power consumption.

[0144] In a possible implementation, the method also includes: when the privacy protection function is turned on, at a fourth time point, the electronic device does not retain the third audio data based on the loudness of the third audio data in the third video data being greater than or equal to a first preset threshold, the third audio data including a human voiceprint information, and the human voiceprint information does not match the pre-saved first human voiceprint information.

[0145] It can be understood that, when the privacy protection function is turned on, at the fourth time point, the electronic device further collects the third video data, and detects that the loudness of the third audio data collected by the microphone in the third video data is greater than or equal to the first preset threshold, indicating that the third audio data is valid audio data, and small sounds or noises are excluded; in addition, through the human voiceprint detection technology, it is detected that the third audio data contains a human voiceprint information, but this human voiceprint information does not match the pre-saved first human voiceprint information, indicating that at this time there is a surrounding user at the call scene of the first user speaking, and it is not the collected human voice information of the first user. In order to avoid leaking other users' privacy information, there is no need to send the third audio data to the second user, and therefore there is no need to retain the third audio data to save power consumption.

[0146] In general, if Figure 8 As shown, the specific implementation process for audio data detection in the data transmission method can be: in the case of a video call or a voice call, the electronic device responds to the operation of turning on the privacy protection function, collects and detects audio data (such as the second audio data in the second video data, the third audio data in the third video data). First, it is detected whether the loudness of the audio data is greater than or equal to the first preset threshold; if the loudness of the audio data is less than the first preset threshold, the audio data is not retained; if the loudness of the audio data is greater than or equal to the first preset threshold, the audio data is further detected whether there is human voiceprint information.

[0147] Furthermore, if the audio data does not contain human voiceprint information, the audio data is not retained; if the audio data contains human voiceprint information, it is further detected whether the audio data contains multiple human voiceprint information.

[0148] Furthermore, if the audio data contains multiple voice print information, the audio data will not be retained; if the audio data contains only one voice print information, the voice print information of the one person will be further detected to see whether it matches the pre-saved voice print information of the first user (ie, the first voice print information).

[0149] Finally, if the personal voiceprint information does not match the pre-saved personal voiceprint information of the first user, the audio data will not be retained; if the personal voiceprint information matches the pre-saved personal voiceprint information of the first user, the audio data will be retained and sent to the second user (i.e., other users in the video call or voice call except the first user).

[0150] In a possible implementation, the method further includes: the electronic device displays a video call interface; the electronic device displays a notification bar in response to pulling down the video call interface; the notification bar includes an activation button for the privacy protection function; the electronic device activates the privacy protection function in response to clicking the activation button. The specific implementation can be referred to the description in the above step S402, which will not be described in detail here.

[0151] In a possible implementation, the method further includes: when the electronic device detects that the number of repetitions of the first user's vocal content reaches a second preset threshold within a preset time period, turning off the privacy protection function; or, when a sound event for turning off the privacy protection function is detected, turning off the privacy protection function. The specific implementation method can be referred to the description in the above step S403, which will not be repeated here.

[0152] It can be seen that based on the method described above, in the case of a video call or a voice call, at a first time point, the electronic device sends the first video data of the first user to the second user, and the first audio data in the first video data includes multiple human voiceprint information; at a second time point, the electronic device responds to the operation of turning on the privacy protection function, because the loudness of the first audio data in the first video data is greater than or equal to the first preset threshold, and the first audio data includes multiple human voiceprint information, the electronic device needs to stop sending the first audio data to the second user; when the privacy protection function is turned on, at a third time point, because the loudness of the second audio data in the second video data is greater than or equal to the first preset threshold, and in addition, the second audio data includes a human voiceprint information, and the human voiceprint information matches the pre-saved first human voiceprint information, the electronic device can send the second audio data to the second user. It can be understood that after the privacy protection function is turned on, the audio data will only be sent to other users in the video call or voice call when the loudness of the audio data is greater than or equal to the first preset threshold, and there is only one human voice print information in the audio data, and the human voice print information matches the pre-saved human voice print information of the first user, thereby preventing the conversation audio of surrounding users at the call site (such as surrounding users at the call site of the first user) from being exposed to other users in the video call or voice call, thereby avoiding privacy leakage and saving power consumption.

[0153] 2. The user identification information includes facial image information.

[0154] like Fig. 9 As shown, the data transmission method includes the following steps S901 to S903. Fig. 9 The method execution subject shown can be the electronic device mentioned above. Or, Fig. 9 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Fig. 9 The method is described by taking an electronic device as an example of an execution subject.

[0155] S901. In case of a video call, at a first time point, an electronic device sends first video data of a first user to a second user.

[0156] In an embodiment of the present application, the first video data includes multiple facial image information, and the second user is a user other than the first user in the video call. Specifically, the first video data includes first image data, and the first image data includes multiple facial image information. At a first time point, the electronic device sends the first video data of the first user to the second user, which can also be understood as, at the first time point, the electronic device sends the first image data of the first user to the second user.

[0157] It should be noted that when the first user uses the call APP (i.e., an application capable of making video calls) in an electronic device to make a video call (such as a multi-party video conference) with the second user, at the first time point, the surrounding users at the call site of the first user appear on the screen of the first user, and at this time, the image data collected by the camera in the first video data contains facial image information of multiple users. Since the first user has not turned on the privacy protection function in the call APP, the first image data in the first video data will be exposed to other users in the video call except the first user, that is, the electronic device sends the first image data of the first user to the second user.

[0158] S902. At a second time point, the electronic device responds to the operation of turning on the privacy protection function, and stops sending the first image data to the second user based on the fact that the first image data in the first video data includes multiple facial image information.

[0159] In an embodiment of the present application, the second time point is later than the first time point. At the second time point, the user turns on the privacy protection function in the call APP, and the electronic device detects whether the first image data in the first video data includes multiple facial image information. If the first image data in the first video data includes multiple facial image information, the electronic device stops sending the first image data to the second user to avoid privacy leakage and save power consumption.

[0160] S903. When the privacy protection function is turned on, at a third time point, the electronic device sends the second image data to the second user based on the fact that the second image data in the second video data includes facial image information and the facial image information matches the pre-saved first facial image information.

[0161] In the embodiment of the present application, the third time point is later than the second time point. The first facial image information pre-stored here refers to the facial image information of the first user pre-collected by the electronic device.

[0162] When the privacy protection function is turned on, at the third time point, the surrounding users at the first user's call site leave the first user's screen. At this time, the second image data collected by the camera in the second video data only contains one face image information, and the face image information matches the pre-saved first face image information. In this case, the electronic device allows the second image data in the second video data to be exposed to other users in the video call except the first user, that is, the electronic device can send the second image data to the second user. It can be understood that at the third time point, since the second audio data only contains the face image information of the first user, it does not involve the leakage of other users' privacy information, and the second image data can be sent to the second user.

[0163] In a possible implementation, the method further includes: when the privacy protection function is turned on, at a fourth time point, the electronic device does not retain the third image data based on the absence of facial image information in the third image data in the third video data.

[0164] It can be understood that when the privacy protection function is turned on, at the fourth time point, the electronic device further collects the third video data, and through the face recognition technology, it detects that the third image data collected by the camera in the third video data does not contain facial image information, indicating that no one appears on the screen at this time, so there is no need to retain the third image data to save power consumption.

[0165] In one possible implementation, the method also includes: when the privacy protection function is turned on, at a fourth time point, the electronic device does not retain the third image data based on the third image data in the third video data including facial image information, and the facial image information does not match the pre-saved first facial image information.

[0166] It can be understood that, when the privacy protection function is turned on, at the fourth time point, the electronic device further collects the third video data, and through the facial recognition technology, it is detected that there is only one facial image information in the third image data collected by the camera in the third video data, and the facial image information does not match the pre-saved first facial image information, indicating that at this time a surrounding user at the call site of the first user appears on the screen. In order to avoid leaking the privacy information of other users, there is no need to send the third image data to the second user, and therefore there is no need to retain the third image data to save power consumption.

[0167] In general, if Fig.10 As shown, the specific implementation process for image data detection in the data transmission method can be: in the case of a video call, the electronic device responds to the operation of turning on the privacy protection function, collects and detects image data (such as the second image data in the second video data, the third image data in the third video data). First, it is detected whether there is facial image information in the image data; if there is no facial image information in the image data, the image data is not retained; if there is facial image information in the image data, it is further detected whether there are multiple facial image information in the image data.

[0168] Further, if the image data contains multiple facial image information, the image data will not be retained; if the image data contains one facial image information, the facial image information will be further detected to see whether it matches the pre-saved facial image information of the first user (ie, the first facial image information).

[0169] Finally, if the facial image information does not match the facial image information of the first user saved in advance, the image data will not be retained; if the facial image information matches the facial image information of the first user saved in advance, the image data will be retained and sent to the second user (i.e., other users in the video call except the first user).

[0170] In a possible implementation, the method further includes: the electronic device displays a video call interface; the electronic device displays a notification bar in response to pulling down the video call interface; the notification bar includes an activation button for the privacy protection function; the electronic device activates the privacy protection function in response to clicking the activation button. The specific implementation can be referred to the description in the above step S402, which will not be described in detail here.

[0171] In a possible implementation, the method further includes: when the electronic device detects that the number of repetitions of the first user's vocal content reaches a second preset threshold within a preset time period, turning off the privacy protection function; or, when a sound event for turning off the privacy protection function is detected, turning off the privacy protection function. The specific implementation method can be referred to the description in the above step S403, which will not be repeated here.

[0172] It can be seen that based on the method described above, in the case of a video call, at a first time point, the electronic device sends the first video data of the first user to the second user, and the first image data in the first video data includes multiple facial image information; at a second time point, the electronic device responds to the operation of turning on the privacy protection function, because the first image data in the first video data includes multiple facial image information, the electronic device needs to stop sending the first image data to the second user; in the case of turning on the privacy protection function, at a third time point, because the second image data in the second video data includes a facial image information, and the facial image information matches the pre-stored first facial image information, the electronic device can send the second image data to the second user. It can be understood that after turning on the privacy protection function, only when the image data has a facial image information, and the facial image information matches the pre-stored facial image information of the first user, will the image data be sent to other users in the video call, so that the facial images of the surrounding users at the call site (such as the surrounding users at the call site of the first user) will not be exposed to other users in the video call, thereby avoiding privacy leakage and saving power consumption.

[0173] 3. The user identification information includes voiceprint information and face image information.

[0174] like Fig.11 As shown, the data transmission method includes the following steps S1101 to S1103. Fig.11 The method execution subject shown can be the electronic device mentioned above. Or, Fig.11 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Fig.11 The method is described by taking an electronic device as an example of an execution subject.

[0175] S1101. In case of a video call, at a first time point, an electronic device sends first video data of a first user to a second user.

[0176] In an embodiment of the present application, the first video data includes multiple voiceprint information and multiple face image information, and the second user is a user other than the first user in the video call. Specifically, the first video data includes first audio data and first image data, the first audio data includes multiple voiceprint information, and the first image data includes multiple face image information. It should be noted that each voiceprint information and each face image information may or may not be in a one-to-one correspondence, which is not limited here.

[0177] It can be understood that when the first user uses the call APP (i.e., an application that can make video calls) in the electronic device to make a video call (such as a multi-party video conference) with the second user, at the first time point, the surrounding users at the call site of the first user are talking, and at this time, the first audio data collected by the microphone in the first video data contains multiple voiceprint information; the surrounding users at the call site of the first user appear on the screen of the first user, and at this time, the image data collected by the camera in the first video data contains multiple users' facial image information. Since the first user has not turned on the privacy protection function in the call APP, the first audio data and the first image data in the first video data will be exposed to other users in the video call except the first user, that is, the electronic device sends the first audio data and the first image data of the first user to the second user.

[0178] S1102. At a second time point, the electronic device responds to the operation of turning on the privacy protection function, and based on the loudness of the first audio data in the first video data being greater than or equal to the first preset threshold and the first audio data including multiple voiceprint information, stops sending the first audio data to the second user; based on the first image data in the first video data including multiple facial image information, stops sending the first image data to the second user.

[0179] In the embodiment of the present application, the electronic device stops sending the first audio data to the second user based on the loudness of the first audio data in the first video data being greater than or equal to the first preset threshold value, and the first audio data including multiple voiceprint information. The specific implementation method can refer to the description of the above step S702, which is not repeated here. The specific implementation method can refer to the description of the above step S902, which is not repeated here, based on the first image data in the first video data including multiple facial image information, and the first image data is stopped from being sent to the second user.

[0180] S1103. When the privacy protection function is turned on, at a third time point, the electronic device sends the second audio data to the second user based on the fact that the loudness of the second audio data in the second video data is greater than or equal to the first preset threshold, the second audio data includes a human voiceprint information, and the human voiceprint information matches the pre-saved first human voiceprint information; the electronic device sends the second image data to the second user based on the fact that the second image data in the second video data includes a human face image information, and the human face image information matches the pre-saved first human face image information.

[0181] In an embodiment of the present application, the electronic device sends the second audio data to the second user based on the loudness of the second audio data in the second video data being greater than or equal to the first preset threshold, the second audio data including a human voiceprint information, and the human voiceprint information matching the pre-saved first human voiceprint information. The specific implementation method can refer to the description of the above step S703, which is not repeated here. The electronic device sends the second image data to the second user based on the second image data in the second video data including a human face image information, and the human face image information matching the pre-saved first human face image information. The specific implementation method can refer to the description of the above step S903, which is not repeated here.

[0182] In general, if Fig.12 As shown, the specific implementation process of detecting audio data and image data in video data in the data transmission method can be: in the case of a video call, the electronic device responds to the operation of turning on the privacy protection function, and simultaneously collects and detects audio data (such as the second audio data in the second video data, the third audio data in the third video data) and image data (such as the second image data in the second video data, the third image data in the third video data).

[0183] For audio data detection, the first step is to detect whether the loudness of the audio data is greater than or equal to a first preset threshold; if the loudness of the audio data is less than the first preset threshold, the audio data is not retained; if the loudness of the audio data is greater than or equal to the first preset threshold, the audio data is further detected to determine whether there is human voiceprint information.

[0184] Furthermore, if the audio data does not contain human voiceprint information, the audio data is not retained; if the audio data contains human voiceprint information, it is further detected whether the audio data contains multiple human voiceprint information.

[0185] Furthermore, if the audio data contains multiple voice print information, the audio data will not be retained; if the audio data contains only one voice print information, the voice print information of the one person will be further detected to see whether it matches the pre-saved voice print information of the first user (ie, the first voice print information).

[0186] Finally, if the personal voiceprint information does not match the pre-saved personal voiceprint information of the first user, the audio data will not be retained; if the personal voiceprint information matches the pre-saved personal voiceprint information of the first user, the audio data will be retained and sent to the second user (i.e., other users in the video call except the first user).

[0187] For image data detection, the first step is to detect whether there is facial image information in the image data; if there is no facial image information in the image data, the image data is not retained; if there is facial image information in the image data, the image data is further detected whether there are multiple facial image information.

[0188] Further, if the image data contains multiple facial image information, the image data will not be retained; if the image data contains one facial image information, the facial image information will be further detected to see whether it matches the pre-saved facial image information of the first user (ie, the first facial image information).

[0189] Finally, if the facial image information does not match the facial image information of the first user saved in advance, the image data will not be retained; if the facial image information matches the facial image information of the first user saved in advance, the image data will be retained and sent to the second user (i.e., other users in the video call except the first user).

[0190] In a possible implementation, the method further includes: the electronic device displays a video call interface; the electronic device displays a notification bar in response to pulling down the video call interface; the notification bar includes an activation button for the privacy protection function; the electronic device activates the privacy protection function in response to clicking the activation button. The specific implementation can be referred to the description in the above step S402, which will not be described in detail here.

[0191] In a possible implementation, the method further includes: when the electronic device detects that the number of repetitions of the first user's vocal content reaches a second preset threshold within a preset time period, turning off the privacy protection function; or, when a sound event for turning off the privacy protection function is detected, turning off the privacy protection function. The specific implementation method can be referred to the description in the above step S403, which will not be repeated here.

[0192] It can be seen that based on the method described above, in the case of a video call, at a first time point, the electronic device sends the first video data of the first user to the second user, and at this time, the first audio data in the first video data includes multiple voice print information, and the first image data in the first video data includes multiple face image information; at a second time point, the electronic device responds to the operation of turning on the privacy protection function, because the loudness of the first audio data in the first video data is greater than or equal to the first preset threshold, and the first audio data includes multiple voice print information, the electronic device needs to stop sending the first audio data to the second user, and at the same time, because the first image data in the first video data includes Multiple facial image information, so the electronic device needs to stop sending the first image data to the second user; when the privacy protection function is turned on, at a third time point, since the loudness of the second audio data in the second video data is greater than or equal to the first preset threshold, the second audio data includes a human voiceprint information, and the human voiceprint information matches the pre-saved first human voiceprint information, the electronic device can send the second audio data to the second user; at the same time, since the second image data in the second video data includes a human facial image information, and the human facial image information matches the pre-saved first facial image information, the electronic device can send the second image data to the second user. It can be understood that after the privacy protection function is turned on, the audio data will only be sent to other users in the video call when the loudness of the audio data is greater than or equal to the first preset threshold, and the audio data only contains one voice voiceprint information, and the voice voiceprint information matches the pre-saved voice voiceprint information of the first user, and the image data will only be sent to other users in the video call when there is one face image information in the image data, and the face image information matches the pre-saved face image information of the first user, so that the conversation audio and facial images of the surrounding users at the call site (such as the surrounding users at the call site of the first user) will not be exposed to other users in the video call, thereby avoiding privacy leakage and saving power consumption.

[0193] See also Fig.13 , Fig.13 A schematic structural diagram of a data transmission device 1300 according to an embodiment of the present application is shown. Fig.13 The data transmission device shown may be an electronic device, or a device in an electronic device, or a device that can be used in conjunction with an electronic device. Fig.13 The data transmission device shown may include a processing unit 1301 and a transmission unit 1302. Among them:

[0194] The transmission unit 1302 is used to send first video data of a first user to a second user at a first time point in the case of a video call; the first video data includes multiple user identification information, and the second user is a user other than the first user in the video call;

[0195] The transmission unit 1302 is configured to, at a second time point, in response to an operation of enabling the privacy protection function, stop sending the first video data to the second user based on the fact that the first video data includes a plurality of user identification information;

[0196] Transmission unit 1302 is used to send the second video data to the second user at a third time point when the privacy protection function is turned on, based on the second video data including a user identification information, and the user identification information matches the pre-saved first user identification information; the second time point is later than the first time point, and the third time point is later than the second time point.

[0197] In a possible implementation, the user identification information includes voiceprint information and / or facial image information.

[0198] In a possible implementation, the user identification information includes human voiceprint information, and the transmission unit 1302, when stopping sending the first video data to the second user based on the first video data including multiple user identification information, is specifically used to: stop sending the first audio data to the second user based on the loudness of the first audio data in the first video data being greater than or equal to a first preset threshold, and the first audio data including multiple human voiceprint information.

[0199] In a possible implementation, the transmission unit 1302, when sending the second video data to the second user based on the second video data including user identification information and the user identification information matching the pre-stored first user identification information, is specifically configured to: send the second audio data to the second user based on the loudness of the second audio data in the second video data being greater than or equal to the first preset threshold, the second audio data including a human voiceprint information, and the human voiceprint information matching the pre-stored first human voiceprint information.

[0200] In one possible implementation, the processing unit 1301 is used to: when the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being less than a first preset threshold, not retain the third audio data; the fourth time point is later than the third time point.

[0201] In one possible implementation, the processing unit 1301 is used to: when the privacy protection function is turned on, at a fourth time point, based on the fact that the loudness of the third audio data in the third video data is greater than or equal to the first preset threshold and the third audio data does not contain human voiceprint information, not retain the third audio data; the fourth time point is later than the third time point.

[0202] In one possible implementation, the processing unit 1301 is used to: when the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being greater than or equal to the first preset threshold, the third audio data includes a human voiceprint information, and the human voiceprint information does not match the pre-saved first human voiceprint information, the third audio data is not retained; and the fourth time point is later than the third time point.

[0203] In one possible implementation, the user identification information includes facial image information, and the transmission unit 1302, when stopping sending the first video data to the second user based on the fact that the first video data includes multiple user identification information, is specifically used to: stop sending the first image data to the second user based on the fact that the first image data in the first video data includes multiple facial image information.

[0204] In one possible implementation, the transmission unit 1302, when sending the second video data to the second user based on the fact that the second video data includes a user identification information and the user identification information matches the pre-saved first user identification information, is specifically used to: send the second image data to the second user based on the fact that the second image data in the second video data includes a facial image information and the facial image information matches the pre-saved first facial image information.

[0205] In a possible implementation, the processing unit 1301 is further used to: when the privacy protection function is turned on, at a fourth time point, based on the absence of facial image information in the third image data in the third video data, not retain the third image data.

[0206] In one possible implementation, the processing unit 1301 is also used to: when the privacy protection function is turned on, at a fourth time point, based on the fact that the third image data in the third video data includes facial image information, and the facial image information does not match the pre-saved first facial image information, the third image data is not retained.

[0207] In one possible implementation, the processing unit 1301 is further used to: turn off the privacy protection function when it is detected that the number of repetitions of the vocal content of the first user reaches a second preset threshold within a preset time period; or turn off the privacy protection function when a sound event that turns off the privacy protection function is detected.

[0208] In one possible implementation, the processing unit 1301 is further used to: display a video call interface; in response to pulling down the video call interface, display a notification bar; the notification bar includes an activation button for the privacy protection function; and in response to clicking the activation button, activate the privacy protection function.

[0209] For the case where the data transmission device may be a chip or a chip system, see Fig.14 Schematic diagram of the chip structure shown. Fig.14 The chip 1400 shown includes a processor 1401 and an interface 1402. Optionally, it may also include a memory 1403. The number of the processor 1401 may be one or more, and the number of the interface 1402 may be multiple.

[0210] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0211] The interface 1402 is used to receive or output signals;

[0212] The processor 1401 is used to execute data processing operations of the electronic device.

[0213] It is understandable that some optional features in the embodiments of the present application may be implemented independently in certain scenarios without relying on other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects, or may be combined with other features according to needs in certain scenarios. Accordingly, the data transmission device provided in the embodiments of the present application may also implement these features or functions accordingly, which will not be described in detail here.

[0214] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0215] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0216] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0217] The present application also provides a computer program product. When the computer program product is executed on a computer, the computer can implement the functions of any of the above method embodiments.

[0218] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0219] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data transmission method, characterized in that: The method comprises: In the case of a video call, at a first time point, first video data of a first user is sent to a second user; the first video data includes multiple user identification information, and the second user is a user other than the first user in the video call; At a second time point, in response to an operation of enabling the privacy protection function, based on the first video data including a plurality of user identification information, stopping sending the first video data to the second user; When the privacy protection function is turned on, at a third time point, based on the second video data including a user identification information, and the user identification information matches the pre-saved first user identification information, the second video data is sent to the second user; the second time point is later than the first time point, and the third time point is later than the second time point.

2. The method according to claim 1, characterized in that The user identification information includes voiceprint information and / or face image information.

3. The method according to claim 2, characterized in that The user identification information includes voiceprint information, and based on the first video data including a plurality of user identification information, stopping sending the first video data to the second user includes: Based on the fact that the loudness of the first audio data in the first video data is greater than or equal to a first preset threshold, and the first audio data includes a plurality of voiceprint information, the first audio data is stopped from being sent to the second user.

4. The method according to claim 3, characterized in that The step of sending the second video data to the second user based on the second video data including user identification information, and the user identification information matches the pre-stored first user identification information, comprises: Based on the fact that the loudness of the second audio data in the second video data is greater than or equal to the first preset threshold, the second audio data includes a human voiceprint information, and the human voiceprint information matches the pre-saved first human voiceprint information, the second audio data is sent to the second user.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: When the privacy protection function is turned on, at a fourth time point, based on the loudness of the third audio data in the third video data being less than the first preset threshold, the third audio data is not retained; the fourth time point is later than the third time point.

6. The method according to claim 3 or 4, characterized in that: The method further comprises: When the privacy protection function is turned on, at a fourth time point, based on the fact that the loudness of the third audio data in the third video data is greater than or equal to the first preset threshold and the third audio data does not contain any human voiceprint information, the third audio data is not retained; the fourth time point is later than the third time point.

7. The method according to claim 3 or 4, characterized in that: The method further comprises: When the privacy protection function is turned on, at a fourth time point, based on that the loudness of the third audio data in the third video data is greater than or equal to the first preset threshold, the third audio data includes a human voiceprint information, and the human voiceprint information does not match the pre-saved first human voiceprint information, the third audio data is not retained; the fourth time point is later than the third time point.

8. The method according to any one of claims 1 to 7, characterized in that The user identification information includes face image information, and based on the first video data including a plurality of user identification information, stopping sending the first video data to the second user includes: Based on the fact that the first image data in the first video data includes a plurality of facial image information, the sending of the first image data to the second user is stopped.

9. The method according to claim 8, characterized in that The step of sending the second video data to the second user based on the second video data including user identification information, and the user identification information matches the pre-stored first user identification information, comprises: Based on the fact that the second image data in the second video data includes a face image information, and the face image information matches the pre-stored first face image information, the second image data is sent to the second user.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: When the privacy protection function is turned on, at the fourth time point, based on the fact that the third image data in the third video data does not contain facial image information, the third image data is not retained.

11. The method according to claim 8 or 9, characterized in that: The method further comprises: When the privacy protection function is turned on, at the fourth time point, based on the fact that the third image data in the third video data includes facial image information, and the facial image information does not match the pre-saved first facial image information, the third image data is not retained.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: When the number of repetitions of the first user's vocal content detected within a preset time period reaches a second preset threshold, the privacy protection function is turned off; or when a sound event that turns off the privacy protection function is detected, the privacy protection function is turned off.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: Display the video call interface; In response to the operation of pulling down the video call interface, displaying a notification bar; the notification bar includes a button for turning on the privacy protection function; In response to an operation of clicking the enable button, the privacy protection function is enabled.

14. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-13.

15. A chip, characterized in that: It comprises a processor and an interface, the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 13 is executed.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 13.

Images