Video conference processing method and terminal

Abstract

The invention discloses a video conference processing method and a terminal, and the method comprises the steps: after a sending connection and a receiving connection are established between a target terminal and a server, the server receives audio and video data sent by other terminals, and then processes the audio and video data according to a session description protocol message representing that only the audio data is received; the server can process the audio and video data sent by other terminals according to the session description protocol message representing that only the audio data is received to obtain the audio data, and the audio data or the video data is sent to the target terminal to be played, so that the audio and video data can be processed by the server according to the session description protocol message representing that only the audio data is received. Compared with the prior art of sending the audio and video data, the bandwidth waste can be reduced, and the data transmission speed in the video conference can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wireless communications, and in particular to a video conference processing method and terminal. Background Art

[0002] In the command and dispatch industry, in order to realize multi-party calls and videos, an audio and video conference is usually established between multiple people. Since some videos do not need to be paid attention to, the relevant technology provides a method for multiple people to watch only one person's video screen. This method hides the video interface that does not need to be paid attention to, but in fact, the party will still receive the video of other parties that do not need to be paid attention to. This method not only wastes bandwidth, but also affects the transmission speed. Summary of the invention

[0003] The present invention provides a video conference processing method and terminal, which are used to solve the problem of wasting bandwidth and affecting transmission speed in video conference in the prior art.

[0004] In a first aspect, the present application provides a method for processing a video conference, the method comprising:

[0005] After receiving the request for establishing a sending connection sent by the server, the target terminal generates a first session description protocol message representing sending audio and video data;

[0006] The target terminal sends the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message;

[0007] After receiving the creation and reception connection request sent by the server, the target terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0008] The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received;

[0009] The target terminal sends the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and sends audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message.

[0010] In a possible implementation manner, the target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received, including:

[0011] The target terminal deletes the last character in the second session description protocol message, or deletes the last character string in the second session description protocol message according to the format of the second session description protocol message.

[0012] In a possible implementation, the method further includes:

[0013] The target terminal determines an identifier representing the video in the second session description protocol message;

[0014] The target terminal deletes the data after the identifier in the second session description protocol message.

[0015] In a possible implementation manner, before the target terminal receives the request for establishing a connection sent by the server, the method further includes:

[0016] The target terminal receives a modification request sent by the server and indicating modification of the second session description protocol message, wherein the modification request carries a modification identifier.

[0017] In a possible implementation manner, the target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received, including:

[0018] The target terminal determines that the modification identifier carried in the modification request indicates that only audio data is received;

[0019] The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received.

[0020] In a second aspect, the present application also provides a terminal, including a processor, a display screen, and a speaker;

[0021] The processor is configured to generate a first session description protocol message representing the sending of audio and video data after receiving a request for establishing a sending connection sent by a server; send the first session description protocol message to the server so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; generate a second session description protocol message representing the sending and receiving of audio and video data after receiving a request for creating a receiving connection sent by the server; modify the format of the second session description protocol message to obtain a third session description protocol message representing the receiving of only audio data; send the third session description protocol message to the server so that a receiving connection is established between the server and the target terminal, and send audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message;

[0022] The display screen is used to display the video screen generated by the target terminal;

[0023] The loudspeaker is used to play audio data sent by other terminals.

[0024] In a possible implementation manner, the processor is specifically configured to:

[0025] According to the format of the second session description protocol message, the last character in the second session description protocol message is deleted, or the last character string in the second session description protocol message is deleted.

[0026] In a possible implementation manner, the processor is further configured to:

[0027] Determining an identifier representing the video in the second session description protocol message;

[0028] The data after the flag in the second session description protocol message is deleted.

[0029] In a possible implementation manner, before receiving the request for establishing a connection sent by the server, the processor is further configured to:

[0030] A modification request sent by the server and indicating modification of the second session description protocol message is received, wherein the modification request carries a modification identifier.

[0031] In a possible implementation manner, the processor is specifically configured to:

[0032] Determining that the modification identifier carried in the modification request indicates that only audio data is received;

[0033] The format of the second session description protocol message is modified to obtain a third session description protocol message indicating that only audio data is received.

[0034] In a third aspect, the present application further provides a terminal, the terminal comprising:

[0035] A first generating module, configured to generate a first session description protocol message representing the sending of audio and video data after receiving a request for establishing a sending connection sent by a server;

[0036] a first sending module, configured to send the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message;

[0037] A second generating module is used to generate a second session description protocol message representing sending and receiving audio and video data after receiving the creation receiving connection request sent by the server;

[0038] A modification module, used for modifying the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received;

[0039] The second sending module is used to send the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and to send audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message.

[0040] In a possible implementation, the modification module is specifically used to:

[0041] According to the format of the second session description protocol message, the last character in the second session description protocol message is deleted, or the last character string in the second session description protocol message is deleted.

[0042] In a possible implementation manner, the terminal further includes:

[0043] A determination module, used to determine an identifier representing a video in the second session description protocol message;

[0044] The deleting module is used to delete the data after the identifier in the second session description protocol message.

[0045] In a possible implementation, the terminal further includes a receiving module, and before receiving the establishment connection request sent by the server, the receiving module is used to:

[0046] A modification request sent by the server and indicating modification of the second session description protocol message is received, wherein the modification request carries a modification identifier.

[0047] In a possible implementation, the modification module is specifically used to:

[0048] Determining that the modification identifier carried in the modification request indicates that only audio data is received;

[0049] The format of the second session description protocol message is modified to obtain a third session description protocol message indicating that only audio data is received.

[0050] The beneficial effects of the present invention are as follows:

[0051] The present application provides a method and terminal for processing a video conference, the method comprising: after the target terminal receives a request for establishing a sending connection sent by a server, a first session description protocol message representing the sending of audio and video data is generated, and then the first session description protocol message is sent to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; after the target terminal receives the request for creating a receiving connection sent by the server, a second session description protocol message representing the sending and receiving of audio and video data is generated, the second session description protocol message is modified to obtain a third session description protocol message representing the receiving of only audio data; the target terminal sends a third session description protocol message to the server, so that the server establishes a receiving connection between the server and the target terminal, and the audio data in the audio and video data sent by other terminals is sent to the target terminal according to the third session description protocol message. Since the server can send the audio data in the audio data sent by other terminals to the target terminal according to the third session description protocol message, bandwidth waste can be reduced and the data transmission speed in the video conference can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0054] Figure 2 A flowchart of a method for processing a video conference provided in an embodiment of the present application;

[0055] Figure 3 An interactive schematic diagram of a video conference processing method provided in an embodiment of the present application;

[0056] Figure 4 An interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0057] Figure 5 An interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0058] Figure 6 An interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0059] Figure 7An interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0060] Figure 8 An interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of the structure of another terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The method in the embodiment of the present invention can be applied to various scenarios, and is particularly applicable to the scenario of scheduling a terminal to conduct a video conference with another terminal. Figure 1 , which is a schematic diagram of a scenario in which a scheduling terminal and a terminal conduct a video conference. The scenario includes a scheduling terminal 101 and one or more terminals, such as terminal 1021, terminal 1022, and terminal 1023.

[0065] When performing multi-party video conferencing, the scheduling terminal 101, the terminal 1021, the terminal 1022 and the terminal 1023 can perform multi-party video conferencing.

[0066] based on Figure 1 The scene, Figure 2 The flowchart corresponding to a method for processing a video conference provided by an embodiment of the present invention is exemplarily shown, and specifically includes the following steps:

[0067] S201, after receiving a request for establishing a sending connection sent by a server, the target terminal generates a first session description protocol message representing sending audio and video data;

[0068] S202: The target terminal sends the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message;

[0069] S203, after receiving the creation and reception connection request sent by the server, the target terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0070] S204: The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received;

[0071] S205: The target terminal sends the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and sends audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message.

[0072] In the embodiment of the present application, after the target terminal receives the request for establishing a sending connection sent by the server, it generates a first session description protocol message representing the sending of audio and video data, and then sends the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives audio and video data according to the first session description protocol message. After the target terminal receives the request for creating a receiving connection sent by the server, it generates a second session description protocol message representing the sending and receiving of audio and video data, modifies the second session description protocol message, and obtains a third session description protocol message representing the receiving of only audio data. The target terminal sends the third session description protocol message to the server, so that the server establishes a receiving connection between the server and the target terminal, and sends the audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message. Since the server can send the audio data in the audio data sent by other terminals according to the third session description protocol message, the target terminal can only receive the audio data sent by other terminals, thereby reducing the waste of bandwidth and improving the data transmission speed in the video conference.

[0073] The target terminal in the embodiment of the present application can be Figure 1 The scheduling terminal 101 may also be one of the terminals 1021, 1022, and 1023, which is not limited in this application.

[0074] In a specific implementation, the target terminal first establishes a sending connection with the server, and then establishes a receiving connection with the server. In the process of establishing the sending connection, the server sends a sending connection establishment request to the target terminal. After the terminal receives the connection sending connection request sent by the server, it generates a first session description protocol message representing the sending of audio and video data. Then, the target terminal sends the first session description protocol message to the server, so that after receiving the first session description protocol message, the server establishes a sending connection with the target terminal, and receives the audio and video data sent by the target terminal according to the first session description protocol message.

[0075] In the process of establishing a receiving connection, the server sends a request to establish a receiving connection to the target terminal. After receiving the request to establish a receiving connection, the target terminal generates a second session description protocol message representing the sending and receiving of audio and video data. Then the target terminal modifies the second session description protocol message into a third session description protocol message representing the reception of only audio data, and sends the third session description protocol message to the server. After receiving the third session description protocol message, the server establishes a receiving connection with the target terminal, and sends the audio data in the audio data sent by other terminals to the server to the target terminal according to the third session description protocol message, thereby realizing that the target terminal only receives the audio data sent by other terminals.

[0076] It should be noted that the first session description protocol message in the embodiment of the present application represents that the target terminal sends audio and video data to the server, and the third session description protocol message not only represents that the target terminal receives the audio data sent by the server, but also has a corresponding relationship with other terminals. For example, when the target terminal is Figure 1 When the scheduling terminal 101 is used, when the target terminal and the server establish a receiving connection, there is a third session description protocol message corresponding to the terminal 1021, a third session description protocol message corresponding to the terminal 1022, and a third session description protocol message corresponding to the terminal 1023. When the terminal 1021 sends audio and video data to the server, the server sends the audio data according to the third session description protocol message corresponding to the terminal 1021 sent by the target terminal; when the terminal 1022 sends audio and video data to the server, the server sends the audio data according to the third session description protocol message corresponding to the terminal 1022 sent by the target terminal; when the terminal 1023 sends audio and video data to the server, the server sends the audio data according to the third session description protocol message corresponding to the terminal 1023 sent by the target terminal.

[0077] for Figure 1The same is true for other terminals except the scheduling terminal 101. For example, when the target terminal is terminal 1021, when terminal 1021 establishes a receiving connection with the server, there is a session description protocol message corresponding to the scheduling terminal 101, a session description protocol message corresponding to terminal 1022, and a session description protocol message corresponding to terminal 1023. If terminal 1021 receives the video and audio of the scheduling terminal 101, the session description protocol message corresponding to terminal 1021 and the scheduling terminal 101 is a session description protocol message representing support for audio and video. If terminal 1021 only receives the audio of terminal 1022, the session description protocol message corresponding to terminal 1021 and terminal 1022 is a session description protocol message representing only receiving audio. If terminal 1021 only receives the audio of terminal 1023, the session description protocol message corresponding to terminal 1021 and terminal 1023 is a session description protocol message representing only receiving audio.

[0078] It should also be noted that the target terminal and the server may also establish a receiving connection first and then establish a sending connection. The specific establishment method may refer to the above method and will not be repeated here.

[0079] like Figure 3 , which is an interactive schematic diagram of a video conference processing method provided in an embodiment of the present application;

[0080] S301, the server sends a request to establish a connection to the target terminal;

[0081] S302, after receiving the request to establish a sending connection, the target terminal generates a first session description protocol message representing the sending of audio and video data; S303, the target terminal sends the first session description protocol message to the server;

[0082] S304: After receiving the first session description protocol message, the server establishes a sending connection between the server and the target terminal;

[0083] S305, the server sends a request to establish a receiving connection to the target terminal;

[0084] S306: After receiving the request to establish a receiving connection, the target terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0085] S307, the target terminal modifies the second session description protocol message to a third session description protocol message for receiving only audio data;

[0086] S308, the target terminal sends the third session description protocol message to the server;

[0087] S309: After receiving the third session description protocol message, the server establishes a receiving connection with the target terminal;

[0088] S310, the server receives audio and video data sent by other terminals;

[0089] S311. The server processes the received audio data according to the third session description protocol message to obtain audio data;

[0090] S312, the server sends the audio data to the target terminal;

[0091] S313: Play audio information.

[0092] like Figure 4 , which is an interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0093] S401, the server sends a request to establish a receiving connection to the target terminal;

[0094] S402: After receiving the request to establish a receiving connection, the target terminal generates a fourth session description protocol message representing receiving audio and video data;

[0095] S403: The target terminal modifies the fourth session description protocol message into a fifth session description protocol message indicating that only audio data is received;

[0096] S404, the target terminal sends the fifth session description protocol message to the server;

[0097] S405. After receiving the fifth session description protocol message, the server establishes a receiving connection with the target terminal;

[0098] S406, the server sends a request to establish a connection to the target terminal;

[0099] S407: After receiving the request to establish a sending connection, the target terminal generates a sixth session description protocol message representing receiving and sending audio and video data;

[0100] S408. The target terminal sends the sixth session description protocol message to the server;

[0101] S409: After receiving the sixth session description protocol message, the server establishes a receiving connection between the server and the target terminal;

[0102] S410, the server receives audio and video data sent by other terminals;

[0103] S411. The server processes the received audio data according to the fifth session description protocol message to obtain audio data;

[0104] S412, the server sends the audio data to the target terminal;

[0105] S413: Play audio information.

[0106] In one embodiment, the target terminal modifies the second session description protocol message representing the reception and transmission of audio and video data into a third session description message representing the reception of audio data only in the following two cases.

[0107] Case 1: if the format of the second session description protocol message is the UNIFIED_PLAN format, the format of the second session description protocol message is modified to a=group:BUNDLE 0, that is, the format of the third session description protocol message is a=group:BUNDLE 0.

[0108] Case 2: if the format of the second session description protocol message is the PLAN_B format, the format of the second session description protocol message is modified to a=group:BUNDLE audio, that is, the format of the third session description protocol message is a=group:BUNDLE audio.

[0109] Specifically, the target terminal may determine the identifier representing the audio in the second session description protocol message, and delete the data after the identifier in the second session description protocol message.

[0110] In the embodiment of the present application, since there is still a lot of information about the video in the original SDP (i.e., the second session description protocol message), it also needs to be cleared. Usually, the information about the video starts from m=video, and from it to the end of the entire SDP are all video data about the video, so all of them need to be cleared. SDP is generated by line, and the end characters are all "\r\n", so we just delete them by line. In this way, we generate a suitable SDP, i.e., the third session description protocol message that only receives audio data.

[0111] In a specific implementation, the target terminal first receives a request to modify a session description protocol message sent by the server, and then a sending connection is established between the target terminal and the server, and then a receiving connection is established between the target terminal and the server. When the receiving connection is established between the target terminal and the server, the target terminal modifies the second session description protocol message so that the target terminal only receives audio data sent by other terminals.

[0112] Specifically, if the request to modify the session description protocol message sent by the server carries an identifier for receiving only video data, the target terminal modifies the second session description protocol message into a session description protocol message indicating that only video data is received; if the request to modify the session description protocol message sent by the server carries an identifier for receiving only audio data, the target terminal modifies the second session description protocol message into a session description protocol message indicating that only audio data is received.

[0113] like Figure 5 FIG. 1 is a flow chart of another method for establishing a receiving connection provided in an embodiment of the present application, which specifically includes the following steps:

[0114] S501, the server sends a request to establish a receiving connection to the target terminal;

[0115] S502: After receiving the request for establishing a receiving connection sent by the server, the target terminal generates a session description protocol message representing sending and receiving audio and video data;

[0116] S503, the target terminal modifies the format of the generated session description protocol message to obtain a session description protocol message indicating that only video data is received;

[0117] S504, the target terminal sends a session description protocol message indicating that only video data is received to the server;

[0118] S505, after receiving the session description protocol message indicating that only video data is to be received, the server establishes a receiving connection between the server and the target terminal;

[0119] S506, the server receives audio and video data sent by other terminals;

[0120] S507, the server processes the received audio data according to the session description protocol message indicating that only video data is received to obtain video data;

[0121] S508: The server sends the video data to the target terminal.

[0122] In addition to receiving audio data or video data sent by other terminals, the target terminal can also send audio and video data to the server.

[0123] like Figure 6 , which is an interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0124] S601, the server sends a request to establish a connection to the target terminal;

[0125] S602: After receiving the request to establish a sending connection, the target terminal generates a first session description protocol message representing the sending of audio and video data;

[0126] S603: The target terminal sends a first session description protocol message to the server;

[0127] S604: After receiving the first session description protocol message, the server establishes a sending connection between the server and the target terminal;

[0128] S605, the server sends a request to establish a receiving connection to the target terminal;

[0129] S606: After receiving the request to establish a receiving connection, the target terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0130] S607: The target terminal modifies the second session description protocol message to a third session description protocol message for receiving only audio data;

[0131] S608. The target terminal sends the third session description protocol message to the server;

[0132] S609: After receiving the third session description protocol message, the server establishes a receiving connection with the target terminal;

[0133] S610, the target terminal sends audio and video data to the server;

[0134] S611. The server receives audio data according to the first session description protocol message.

[0135] like Figure 7 , which is an interactive schematic diagram of another video conference processing method provided in an embodiment of the present application;

[0136] S701, the server sends a request to establish a receiving connection to the target terminal;

[0137] S702: After receiving the request to establish a receiving connection, the target terminal generates a fourth session description protocol message representing receiving audio and video data;

[0138] S703: The target terminal modifies the fourth session description protocol message into a fifth session description protocol message indicating that only audio data is received;

[0139] S704, the target terminal sends the fifth session description protocol message to the server;

[0140] S705. After receiving the fifth session description protocol message, the server establishes a receiving connection with the target terminal;

[0141] S706, the server sends a request to establish a connection to the target terminal;

[0142] S707: After receiving the request to establish a sending connection, the target terminal generates a sixth session description protocol message representing receiving and sending audio and video data;

[0143] S708. The target terminal sends the sixth session description protocol message to the server;

[0144] S709: After receiving the sixth session description protocol message, the server establishes a sending connection between the server and the target terminal;

[0145] S710, the target terminal sends audio and video data to the server;

[0146] S711. The server receives audio and video data according to the sixth session description protocol message.

[0147] The following describes the embodiments of the present application in detail using a multi-party video conference as a scenario.

[0148] First, select the terminal to enter the meeting, put the terminal ID into memberList, and set the ID of the scheduling terminal as presenter, which is also the ID of the video that other terminals must view. In order to distinguish the different contents of different instructions sent each time, the scheduling terminal does not generate a room ID (room number), but the server generates the room ID, which will be returned to other terminals in memberList in the next step. The purpose of this is to prevent the generated meeting numbers from overlapping. The meeting numbers generated by the server can avoid duplication.

[0149] After the server receives the creation signaling sent by the scheduling terminal, it notifies each terminal one by one according to the terminal information in the memberList, and the scheduling terminal also needs to be notified. That is, after the server receives the creation signaling sent by the scheduling terminal, it sends a request to establish a sending connection to the scheduling terminal and each terminal. After the scheduling terminal and each terminal receive the request to establish a sending connection, the scheduling terminal and each terminal generate a first session description protocol message representing the sending of audio and video data. After the scheduling terminal and each terminal generate their respective corresponding first session description protocol messages, they send the generated first session description protocol messages to the server.

[0150] After receiving the first session description protocol message, the server sends a first reply message to the scheduling terminal and each terminal to establish a sending connection between the server and the scheduling terminal, and a sending connection between the server and each terminal, and then sends a request to establish a receiving connection to the scheduling terminal and each terminal.

[0151] After receiving the request to establish a receiving connection, the scheduling terminal and each terminal generate a second session description protocol message representing the reception and transmission of audio and video data. The scheduling terminal and each terminal determine whether to receive only audio data or audio and video data between themselves and other terminals, and then modify the second session description protocol message. For example, the scheduling terminal determines that itself and other terminals only receive audio data from other terminals, then the scheduling terminal modifies the second session description protocol message corresponding to each other terminal into a third session description protocol message for receiving only audio data, and then sends the third session description protocol message to the server. After receiving the third session description protocol message, the server sends media stream data to the target terminal according to the format of the third session description protocol message. For another example, terminal 1 receives the audio and video data of the scheduling terminal and the audio data of terminal 2, then terminal 1 does not make any modification to the second session description protocol message corresponding to the scheduling terminal, and modifies the second session description protocol message corresponding to terminal 2 into a third session description protocol message representing the reception of only audio data.

[0152] In the embodiment of the present application, after the terminal and the scheduling terminal receive the creation notification sent by the server, a session description protocol message is generated. The session protocol description message includes a first session protocol description message and a second session protocol description message. The session protocol description message contains content in accordance with the SDP protocol of WebRTC, which is used to negotiate supported audio and video. Usually WebRTC supports one peerConnection (object connection), and one peerConnection supports sending and receiving audio and video at the same time. However, in the multi-party conference mode, in order to find the audio and video of the sender to the receiver, the receiver will create two peerConnections, one for sending, i.e. sending peerConnection, and one for receiving, i.e. receiving peerConnection. After separation, the server uses the sent peerConnection data to find the received peerConnection, which is logically easier to handle.

[0153] In the specific implementation, first create and send peerConnection, generate sdp (session description protocol) that only supports sending audio and video, and then send this sdp to the server.

[0154] The scheduling terminal and each terminal need to send audio and video by default, so the scheduling terminal and each terminal send their generated sdp to the server.

[0155] According to the interactive protocol of WebRTC, after the scheduling terminal and each terminal send SDP, the server needs to reply with an answer, and both parties negotiate the supported audio and video, ports, etc. so that the audio and video recognized by both parties can be sent normally.

[0156] Since in the embodiment of the present application, sending and receiving are separate, there can be reception only if there is sending. When the server receives the sent SDP, it needs to inform the scheduling terminal and each terminal that they need to receive it. It cannot directly send a receiving notification without sending, which will result in a blank time.

[0157] The scheduling terminal and each terminal are notified to create a receiving instruction. These two are usually separated because the timing of triggering the scheduling terminal and each terminal may be different at different time points.

[0158] When the scheduling terminal and each terminal receive the notification of establishing reception, they need to generate a receiving sdp. At this time, the sdp should only receive audio and the video of the scheduling terminal. However, because the information for sending video has been established when the offer was just created, webrtc will not generate the onlyReceive tag. It will generate sendreceive in the tag in sdp. If this is sent directly, audio and video will be received. Therefore, sdp is modified in this application. In webrtc, sdp has two formats: one is PLAN_B format, and the other is UNIFIED_PLAN format. If it is UNIFIED_PLAN format, it will generate a=group:BUNDLE 0 1 field. If it is PLAN_B, it will generate a=group:BUNDLE audio video field, which means that audio and video are supported at the same time.

[0159] The embodiment of the present application makes the following judgment: If presenter=0000001, it means that the scheduling terminal is 0000001. Because we need to receive the audio and video of the scheduling terminal, there is no need to modify the sdp of the scheduling terminal. If it is not 0000001, it means other terminals, only the audio of other terminals, so the sdp of other terminals needs to be modified. If it is in UNIFIED_PLAN format, it is modified to a=group:BUNDLE 0. If it is in PLAN_B format, it is modified to a=group:BUNDLE audio, which means only receiving audio. Send the generated modified sdp to the server.

[0160] The server generates a new answer based on the modified sdp, and both parties reconnect to receive information such as the audio and video format and port.

[0161] After the scheduling terminal and each terminal receive the answer, they initialize the sending of audio and video media data and the receiving of media information for playback, and send the generated data to the server according to the negotiated SDP-related port and channel. The server finds the relevant receiver based on the received SDP and forwards the data.

[0162] After receiving the data, the terminal can play it, thus realizing a video and multi-audio session mode.

[0163] In the embodiment of the present application, by changing the default interaction mode of webrtc, two peerConnections are established, the sending peerConnection and the receiving peerConnection, so that the server can quickly find the receiving peerConnection according to the sending peerConnection. If Webrtc has set the video parameters, even if the default API of webrtc is called newMediaConstraints.KeyValuePair("OfferToReceiveAudio","true"), video information will be generated in sdp. Therefore, the embodiment of the present application needs to modify sdp to solve the problem of fast response and reduce the bandwidth impact caused by unnecessary audio and video.

[0164] like Figure 8 FIG. 1 is a flow chart of a method for processing a video conference provided in an embodiment of the present application, which specifically includes the following steps:

[0165] S801, the scheduling terminal sends a meeting request to the server;

[0166] S802, the server sends a request to establish a connection to the scheduling terminal and other terminals respectively;

[0167] S803: After receiving the request to establish a sending connection, the scheduling terminal generates a first session description protocol message representing the sending of audio and video data;

[0168] S804, the scheduling terminal sends the first session description protocol message to the server;

[0169] S805: After receiving the first session description protocol message, the server establishes a sending connection between the server and the scheduling terminal;

[0170] S806. The server sends a first response message to the dispatch terminal;

[0171] S807, after receiving the request to establish a sending connection, the other terminal generates a first session description protocol message representing the sending of audio and video data;

[0172] S808. The other terminal sends the first session description protocol message to the server;

[0173] S809: After receiving the first session description protocol message, the server establishes a sending connection between the server and the other terminal;

[0174] S810, the server sends a first response message to other terminals;

[0175] S811, the server sends a request to establish a receiving connection to the scheduling terminal and other terminals respectively;

[0176] S812: After receiving the request to establish a receiving connection, the scheduling terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0177] S813. The scheduling terminal modifies the second session description protocol message into a third session description protocol message indicating that only audio data is received.

[0178] S814. The scheduling terminal sends the third session description protocol message to the server;

[0179] S815. After receiving the third session description protocol message, the server establishes a receiving connection between the server and the scheduling terminal;

[0180] S816. The server sends a second response message to the scheduling terminal;

[0181] S817. After receiving the request to establish a receiving connection, the other terminal generates a second session description protocol message representing sending and receiving audio and video data;

[0182] S818. The other terminal sends the second session description protocol message to the server;

[0183] S819: After receiving the second session description protocol message, the server establishes a receiving connection between the server and the other terminal;

[0184] S820. The server sends a second response message to other terminals.

[0185] S821, the dispatching terminal generates audio and video data;

[0186] S822, the dispatching terminal sends the audio and video data to the server through the sending connection;

[0187] S823. After receiving the audio and video data, the server sends the audio and video data to other terminals according to the second session description protocol message;

[0188] S824. The other terminal displays the video information in the audio and video data, and plays the audio information in the audio and video data;

[0189] S825. Other terminals generate audio and video data;

[0190] S826. Other terminals send the audio and video data to the server via a sending connection;

[0191] S827. The server sends the audio data in the audio and video data to the scheduling terminal through a sending connection according to the third session description protocol message;

[0192] S828. After receiving the audio data, the dispatch terminal plays the audio data.

[0193] Based on the same inventive concept, an embodiment of the present application also provides a terminal, the principle of solving the problem of the terminal is similar to the above-mentioned video conference processing method. The implementation of the terminal can refer to the implementation of the method, and the repeated parts will not be repeated.

[0194] like Fig. 9 As shown, a terminal provided in an embodiment of the present application includes a processor 701, a display screen 702 and a speaker 703;

[0195] The processor 701 is configured to generate a first session description protocol message representing the sending of audio and video data after receiving a request for establishing a sending connection sent by a server; send the first session description protocol message to the server so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; generate a second session description protocol message representing the sending and receiving of audio and video data after receiving a request for creating a receiving connection sent by the server; modify the format of the second session description protocol message to obtain a third session description protocol message representing the receiving of only audio data; send the third session description protocol message to the server so that a receiving connection is established between the server and the target terminal, and send the audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message;

[0196] The display screen 702 is used to display the video screen generated by the target terminal;

[0197] The speaker 703 is used to play audio data sent by other terminals.

[0198] In a possible implementation manner, the processor 701 is specifically configured to:

[0199] According to the format of the second session description protocol message, the last character in the second session description protocol message is deleted, or the last character string in the second session description protocol message is deleted.

[0200] In a possible implementation manner, the processor 701 is further configured to:

[0201] Determining an identifier representing the video in the second session description protocol message;

[0202] The data after the flag in the second session description protocol message is deleted.

[0203] In a possible implementation manner, before receiving the establishment connection request sent by the server, the processor 701 is further configured to:

[0204] A modification request sent by the server and indicating modification of the second session description protocol message is received, wherein the modification request carries a modification identifier.

[0205] In a possible implementation manner, the processor 701 is specifically configured to:

[0206] Determining that the modification identifier carried in the modification request indicates that only audio data is received;

[0207] The format of the second session description protocol message is modified to obtain a third session description protocol message indicating that only audio data is received.

[0208] Based on the same inventive concept, an embodiment of the present application also provides a terminal, the principle of solving the problem of the terminal is similar to the above-mentioned video conference processing method. The implementation of the terminal can refer to the implementation of the method, and the repeated parts will not be repeated.

[0209] like Fig.10 As shown, a terminal provided in an embodiment of the present application includes:

[0210] The first generating module 801 is used to generate a first session description protocol message representing the sending of audio and video data after receiving a request for establishing a sending connection sent by the server;

[0211] A first sending module 802 is used to send the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message;

[0212] The second generating module 803 is used to generate a second session description protocol message representing sending and receiving audio and video data after receiving the creation and receiving connection request sent by the server;

[0213] A modification module 804 is used to modify the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received;

[0214] The second sending module 805 is used to send the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and to send audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message.

[0215] In a possible implementation, the modification module 804 is specifically configured to:

[0216] According to the format of the second session description protocol message, the last character in the second session description protocol message is deleted, or the last character string in the second session description protocol message is deleted.

[0217] In a possible implementation manner, the terminal further includes:

[0218] A determination module, used to determine an identifier representing a video in the second session description protocol message;

[0219] The deleting module is used to delete the data after the identifier in the second session description protocol message.

[0220] In a possible implementation, the terminal further includes a receiving module, and before receiving the establishment connection request sent by the server, the receiving module is used to:

[0221] A modification request sent by the server and indicating modification of the second session description protocol message is received, wherein the modification request carries a modification identifier.

[0222] In a possible implementation, the modification module is specifically used to:

[0223] Determining that the modification identifier carried in the modification request indicates that only audio data is received;

[0224] The format of the second session description protocol message is modified to obtain a third session description protocol message indicating that only audio data is received.

[0225] The present application provides a method and terminal for processing a video conference, the method comprising: after the target terminal receives a request for establishing a sending connection sent by a server, a first session description protocol message representing the sending of audio and video data is generated, and then the first sending description protocol message is sent to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; after the target terminal receives the request for establishing a receiving connection sent by the server, a second session description protocol message representing the sending and receiving of audio and video data is generated, the second session description protocol message is modified to obtain a third session description protocol message representing the receiving of only audio data; the target terminal sends a third session description protocol message to the server, so that the server establishes a receiving connection between the server and the target terminal, and the audio data in the audio and video data sent by other terminals is sent to the target terminal according to the third session description protocol message. Since the server can send the audio data in the audio data sent by other terminals according to the third session description protocol message, it can reduce the waste of bandwidth and improve the data transmission speed in the video conference.

[0226] The present application is described above with reference to the block diagrams and / or flow charts showing the methods, devices (systems) and / or computer program products according to the embodiments of the present application. It should be understood that a block of the block diagram and / or flow chart diagram and a combination of blocks of the block diagram and / or flow chart diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer and / or other programmable data processing device to produce a machine, so that the instructions executed by the computer processor and / or other programmable data processing device create a method for implementing the functions / actions specified in the block diagram and / or flow chart block.

[0227] Accordingly, the present application may also be implemented with hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, the present application may take the form of a computer program product on a computer-usable or computer-readable storage medium, which has a computer-usable or computer-readable program code implemented in the medium, for use by an instruction execution system or in conjunction with an instruction execution system. In the context of the present application, a computer-usable or computer-readable medium may be any medium that may contain, store, communicate, transmit, or convey a program for use by an instruction execution system, device, or apparatus, or in conjunction with an instruction execution system, device, or apparatus.

[0228] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for processing a video conference, characterized in that: The method includes: After receiving the request for establishing a sending connection sent by the server, the target terminal generates a first session description protocol message representing sending audio and video data; The target terminal sends the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; After receiving the creation and reception connection request sent by the server, the target terminal generates a second session description protocol message representing sending and receiving audio and video data; The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received; The target terminal sends the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and sends audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message.

2. The method according to claim 1, characterized in that The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received, including: The target terminal deletes the last character in the second session description protocol message, or deletes the last character string in the second session description protocol message according to the format of the second session description protocol message.

3. The method according to claim 2, characterized in that The method further includes: The target terminal determines an identifier representing the video in the second session description protocol message; The target terminal deletes the data after the identifier in the second session description protocol message.

4. The method according to any one of claims 1 to 3, characterized in that: Before the target terminal receives the establishment connection request sent by the server, the method further includes: The target terminal receives a modification request sent by the server and indicating modification of the second session description protocol message, wherein the modification request carries a modification identifier.

5. The method according to claim 4, characterized in that The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received, including: The target terminal determines that the modification identifier carried in the modification request indicates that only audio data is received; The target terminal modifies the format of the second session description protocol message to obtain a third session description protocol message indicating that only audio data is received.

6. A terminal, characterized in that: Includes processor, display and speakers; The processor is configured to generate a first session description protocol message representing the sending of audio and video data after receiving a request for establishing a sending connection sent by the server; Sending the first session description protocol message to the server, so that the server establishes a sending connection according to the first session description protocol message, and receives the audio and video data sent by the target terminal according to the first session description protocol message; After receiving the creation and reception connection request sent by the server, a second session description protocol message representing the sending and receiving of audio and video data is generated; a format of the second session description protocol message is modified to obtain a third session description protocol message representing the reception of only audio data; Sending the third session description protocol message to the server to establish a receiving connection between the server and the target terminal, and sending audio data in the audio and video data sent by other terminals to the target terminal according to the third session description protocol message; The display screen is used to display the video screen generated by the target terminal; The loudspeaker is used to play audio data sent by other terminals.

7. The terminal according to claim 6, characterized in that The processor is specifically used for: According to the format of the second session description protocol message, the last character in the second session description protocol message is deleted, or the last character string in the second session description protocol message is deleted.

8. The terminal according to claim 7, characterized in that The processor is further configured to: Determining an identifier representing the video in the second session description protocol message; The data after the flag in the second session description protocol message is deleted.

9. The terminal according to any one of claims 6 to 8, characterized in that: Before receiving the establishment connection request sent by the server, the processor is further configured to: A modification request sent by the server and indicating modification of the second session description protocol message is received, wherein the modification request carries a modification identifier.

10. The terminal according to claim 9, characterized in that The processor is specifically used for: Determining that the modification identifier carried in the modification request indicates that only audio data is received; The format of the second session description protocol message is modified to obtain a third session description protocol message indicating that only audio data is received.

Images