Video call method, device, equipment and storage medium

Abstract

The present application proposes a video call method, apparatus, device and storage medium. The video call method displays a call setting page when a video call application is started, and responds to the setting instructions of the call setting page, so that the user can determine the mask area and mask style parameters in the screen based on his or her own needs. When the video call starts, a target call video stream is generated according to the mask area and the mask style parameters. In the target call video stream, the mask area is masked using the mask style parameters, and the target call video stream is sent. Based on the user's needs, the mask area is masked using the mask style parameters in the target call video stream, which is beneficial to protecting the user's privacy during the video call and improving the user's video call experience.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a video call method, apparatus, device and storage medium. Background Art

[0002] With the promotion of 5G networks, VoLTE video calls have become an increasingly important way of daily communication. However, in some specific environments, such as home and office, the background of the call may involve sensitive information, such as private photos or sensitive files on the bookshelf, etc., which may easily cause privacy leakage during the video call and affect the user's video call experience. Summary of the invention

[0003] The embodiments of the present application provide a video call method, apparatus, device, and storage medium to solve at least one problem existing in the related art. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for making a video call, including:

[0005] When the video call application is launched, the call settings page is displayed;

[0006] In response to a setting instruction on the call setting page, determining a mask area and mask style parameters in the screen;

[0007] At the start of a video call, a target call video stream is generated according to the mask area and the mask style parameters, wherein the mask area is masked using the mask style parameters in the target call video stream;

[0008] Send the target call video stream.

[0009] In one implementation, in response to a setting instruction on a call setting page, determining a mask area and mask style parameters on a screen includes:

[0010] In response to a first instruction on the call setting page, selecting at least one area from the screen as a mask area;

[0011] In response to the second instruction on the call setting page, determining a mask style for each area respectively to obtain a mask style parameter;

[0012] The mask style includes one of blur, blur, mosaic and pattern.

[0013] In one embodiment, generating a target call video stream according to a mask area and a mask style parameter at the start of a video call includes:

[0014] When a video call starts, capture the call video stream in real time;

[0015] According to the mask style parameters, the mask area is masked to generate a masked video stream;

[0016] Generate a target call video stream based on the mask video stream and the call video stream.

[0017] In one implementation, masking the mask area according to the mask style parameters to generate the masked video stream includes:

[0018] According to the mask style parameters, a corresponding mask effect is generated in the mask area;

[0019] According to the masking effect, a video stream is generated, and the video stream is time-aligned with the call video stream to generate a masked video stream.

[0020] In one implementation, generating a target call video stream according to the mask video stream and the call video stream includes:

[0021] Through the GPU, the video encoder and one of the FFmpeg tools, the mask video stream is overlaid into the mask area corresponding to the call video stream to generate a target call video stream.

[0022] In one implementation, generating a target call video stream according to the mask video stream and the call video stream includes:

[0023] Use the mask video stream as an enhancement layer or overlay;

[0024] By using a GPU, a video encoder, and one of the FFmpeg tools, a mask effect is added to the call video stream of the base layer by using an enhancement layer or an overlay layer to generate a target call video stream.

[0025] In one embodiment, the method further comprises:

[0026] When the video call ends, clear the mask area and mask style parameters.

[0027] In a second aspect, an embodiment of the present application provides a video call device, including:

[0028] A display module, used to display a call setting page when the video call application is started;

[0029] A setting module, configured to determine a mask area and mask style parameters in the screen in response to a setting instruction on the call setting page;

[0030] A generating module, used for generating a target call video stream according to the mask area and the mask style parameters when a video call starts, wherein the mask area is masked by using the mask style parameters in the target call video stream;

[0031] The sending module is used to send the target call video stream.

[0032] In one embodiment, the setting module is further used to:

[0033] When the video call ends, clear the mask area and mask style parameters.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned embodiments.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the method in any one of the above-mentioned embodiments is implemented.

[0036] The beneficial effects of the above technical solution include at least:

[0037] By displaying a call setting page when starting a video call application and responding to setting instructions on the call setting page, the user can determine the mask area and mask style parameters in the screen based on his or her own needs. When the video call starts, a target call video stream is generated based on the mask area and the mask style parameters. In the target call video stream, the mask area is masked using the mask style parameters, and the target call video stream is sent. Based on user needs, the mask area is masked using the mask style parameters in the target call video stream, which is beneficial to protecting the user's privacy during the video call and improving the user's video call experience.

[0038] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0040] Figure 1 This is a schematic diagram of the steps of a video calling method according to an embodiment of the present application;

[0041] Figure 2 This is a structural block diagram of a video calling device according to an embodiment of the present application;

[0042] Figure 3 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0044] In the related art, in the face of the risk of privacy leakage during video calls, full-screen blur processing is usually adopted, which may affect the call quality and user experience and is not conducive to the promotion of video call services.

[0045] Reference Figure 1 , a flowchart of a video calling method according to an embodiment of the present application is shown, and the video calling method may at least include steps S100-S400:

[0046] S100: When a video call application is started, a call setting page is displayed.

[0047] S200: In response to a setting instruction on a call setting page, determine a mask area and mask style parameters on the screen.

[0048] S300. When a video call starts, a target call video stream is generated according to the mask area and the mask style parameters. In the target call video stream, the mask area is masked using the mask style parameters.

[0049] S400: Send the target call video stream.

[0050] The technical solution of the embodiment of the present application is to display a call setting page when starting a video call application, and respond to the setting instructions of the call setting page to enable the user to determine the mask area and mask style parameters in the screen based on his or her own needs. When the video call starts, a target call video stream is generated according to the mask area and the mask style parameters. In the target call video stream, the mask area is masked using the mask style parameters, and the target call video stream is sent. Based on the user's needs, the mask area is masked using the mask style parameters in the target call video stream, which is beneficial to protecting the user's privacy during the video call and improving the user's video call experience.

[0051] In one implementation, when a video call is needed, the user uses the first terminal to start a video call application. At this time, starting the video call application will display a call setting page for the user to set the mask.

[0052] In one embodiment, step S200 includes steps S210-S230:

[0053] S210: In response to a first instruction on the call setting page, select at least one area on the screen as a mask area.

[0054] Optionally, the setting instruction may include a first instruction and a second instruction, and both instructions may be generated by operating a mouse, keyboard, touch screen, etc. In the embodiment of the present application, the first terminal responds to the first instruction on the call setting page, and the user may select at least one area from the display screen of the first terminal as a mask area based on actual needs.

[0055] S220 . In response to the second instruction on the call setting page, determine the mask style of each area respectively to obtain mask style parameters.

[0056] Optionally, the first terminal responds to the second instruction of the call setting page, and the user determines the mask style of each area based on actual needs, and the mask styles of each area can be the same or different to meet personalized needs. It should be noted that the mask style includes one of blur, blur, mosaic and pattern.

[0057] In one implementation, step S300 includes steps S310-S330:

[0058] S310: When a video call starts, capture the call video stream in real time.

[0059] In an embodiment of the present application, a user selects a call partner on a video call application to initiate a video call request, and the first terminal establishes a 5G VoLTE connection with a PBX media processor in the cloud to ensure high-quality transmission of the call video stream.

[0060] S320: Mask the mask area according to the mask style parameters to generate a masked video stream.

[0061] Optionally, steps S3201-S3202 are included:

[0062] S3201. Generate a corresponding mask effect in the mask area according to the mask style parameters.

[0063] Optionally, information such as mask style parameters and mask area can be processed locally on the first terminal to generate a mask video stream, or can be transmitted to a PBX media processor for processing to generate a mask video stream. In the embodiment of the present application, the PBX media processor is used as an example for explanation. Specifically, the PBX media processor uses computer vision technology and mask style parameters to generate a corresponding mask effect in the mask area. For example, if there are two areas A and B, and the corresponding mask styles are blur and pattern, respectively, then area A generates a blur effect and area B generates a pattern covering effect.

[0064] S3202. Generate a video stream according to the mask effect, and time-align the video stream with the call video stream to generate a masked video stream.

[0065] Optionally, after the masking effect is determined, it is equivalent to determining a frame of video, so repeating the video frame can generate a corresponding video stream; then, based on the real-time time of the call video stream, the PBX media processor uses timestamp technology to keep the video stream time-aligned with the call video stream, thereby generating a masked video stream, ensuring that subsequent masked video streams are always aligned with the masked video streams, avoiding subsequent lags or misalignment problems.

[0066] S330. Generate a target call video stream according to the mask video stream and the call video stream.

[0067] Optionally, the PBX media processor receives the call video stream, sends the mask video stream to the first terminal, and synthesizes the mask video stream and the call video stream to generate a target call video stream.

[0068] Optionally, S330 may include S3301 or S3302:

[0069] S3301. Through the GPU, video encoder and one of the FFmpeg tools, the mask video stream is overlaid into the mask area corresponding to the call video stream to generate a target call video stream.

[0070] In some implementations, in order to adapt to different equipment conditions, the PBX media processor can perform video synthesis encoding through hardware (such as GPU, video encoder) or software (such as FFmpeg tool). For example, GPU or video encoder usually supports multi-layer synthesis of streams, and two streams can be combined in real time through hardware, or two streams can be synthesized at the frame level through FFmpeg tool to generate the final video frame. Therefore, through the above means, the mask video stream can be covered in the mask area corresponding to the call video stream, and the target call video stream can be finally generated.

[0071] S3302. Use the mask video stream as an enhancement layer or an overlay layer, and use the GPU, video encoder, and one of the FFmpeg tools to add a mask effect to the call video stream of the base layer using the enhancement layer or the overlay layer to generate a target call video stream.

[0072] In some embodiments, in order to avoid excessive changes to the call video stream of the base layer that may affect the quality of the call video stream, the PBX media processor uses the mask video stream as an enhancement layer or an overlay layer, and uses the enhancement layer or the overlay layer through a GPU, a video encoder, and one of the FFmpeg tools to add a mask effect to the call video stream of the base layer, thereby finally generating a target call video stream.

[0073] In one embodiment, after determining the target call video stream, the PBX media processor sends the target call video stream to the second terminal in real time, decodes and renders the target call video stream, so that the user of the second terminal sees the video after the mask area is masked, which is beneficial to protecting the user's privacy and ensuring that private information is not leaked.

[0074] In one implementation, the video call method of the embodiment of the present application may further include step S500:

[0075] S500: When the video call ends, clear the mask area and mask style parameters.

[0076] In an embodiment of the present application, when the first terminal or the second terminal chooses to end a call, the capture and processing of the call video stream are stopped. At the same time, the first terminal and the PBX media processor will clear the user mask area and mask style parameters to avoid leaving records and protect the user's privacy.

[0077] The method of the embodiment of the present application can be used for privacy processing of one-way or two-way video streams to ensure the privacy of one or both parties; based on the user's own needs, the mask area can be selected as needed on the setting page to achieve dynamic masking, real-time identification and masking of specific areas, and different styles are provided for easy selection, providing personalized privacy protection options to meet the diverse needs of users and improve security; based on the stream injection technology, the mask area of ​​the call video stream is replaced to avoid the impact of full-screen blur processing on the call quality and ensure the clarity of the video call.

[0078] Reference Figure 2 , shows a structural block diagram of a video call device according to an embodiment of the present application, and the device may include:

[0079] A display module, used to display a call setting page when the video call application is started;

[0080] A setting module, configured to determine a mask area and mask style parameters in the screen in response to a setting instruction on the call setting page;

[0081] A generating module, used for generating a target call video stream according to the mask area and the mask style parameters when a video call starts, wherein the mask area is masked by using the mask style parameters in the target call video stream;

[0082] The sending module is used to send the target call video stream.

[0083] In one embodiment, the setting module is further used to:

[0084] When the video call ends, clear the mask area and mask style parameters.

[0085] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0086] Reference Figure 3 , shows a structural block diagram of an electronic device according to an embodiment of the present application, the electronic device includes: a memory 310 and a processor 320, the memory 310 stores instructions that can be run on the processor 320, and the processor 320 loads and executes the instructions to implement the video call method in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0087] In one embodiment, the electronic device also includes a communication interface 330 for communicating with external devices and performing data exchange transmission. If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be interconnected through a bus and complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0088] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0089] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the video call method provided in the above embodiment.

[0090] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0091] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0092] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced RISC machines (ARM) architecture.

[0093] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0094] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0096] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0097] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0098] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0099] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0100] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0101] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video calling method, characterized in that: include: When the video call application is launched, the call settings page is displayed; In response to a setting instruction on the call setting page, determining a mask area and mask style parameters in the screen; At the start of a video call, a target call video stream is generated according to the mask area and the mask style parameters, and in the target call video stream, the mask area is masked using the mask style parameters; Send the target call video stream.

2. The video calling method according to claim 1, characterized in that: The determining of the mask area and mask style parameters in the screen in response to the setting instruction of the call setting page includes: In response to a first instruction on the call setting page, selecting at least one area from the screen as a mask area; In response to the second instruction on the call setting page, determining a mask style for each area respectively to obtain a mask style parameter; The mask style includes one of blur, blur, mosaic and pattern.

3. The video calling method according to claim 1 or 2, characterized in that: When the video call starts, generating the target call video stream according to the mask area and the mask style parameters includes: When a video call starts, capture the call video stream in real time; According to the mask style parameters, the mask area is masked to generate a masked video stream; Generate a target call video stream based on the mask video stream and the call video stream.

4. The video calling method according to claim 3, characterized in that: The step of masking the mask area according to the mask style parameters to generate the masked video stream comprises: According to the mask style parameters, a corresponding mask effect is generated in the mask area; According to the masking effect, a video stream is generated, and the video stream is time-aligned with the call video stream to generate a masked video stream.

5. The video calling method according to claim 3, characterized in that: The generating a target call video stream according to the mask video stream and the call video stream comprises: Through the GPU, the video encoder and one of the FFmpeg tools, the mask video stream is overlaid into the mask area corresponding to the call video stream to generate a target call video stream.

6. The video calling method according to claim 3, characterized in that: The generating a target call video stream according to the mask video stream and the call video stream comprises: Use the mask video stream as an enhancement layer or overlay; By using a GPU, a video encoder, and one of the FFmpeg tools, a mask effect is added to the call video stream of the base layer by using an enhancement layer or an overlay layer to generate a target call video stream.

7. The video calling method according to claim 3, characterized in that: The method further comprises: When the video call ends, clear the mask area and mask style parameters.

8. A video call device, characterized in that: include: A display module, used to display a call setting page when the video call application is started; A setting module, configured to determine a mask area and mask style parameters in the screen in response to a setting instruction on the call setting page; A generating module, used for generating a target call video stream according to the mask area and the mask style parameters when a video call starts, wherein the mask area is masked by using the mask style parameters in the target call video stream; The sending module is used to send the target call video stream.

9. An electronic device, characterized in that: include: A processor and a memory, wherein instructions are stored in the memory and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images