Multimedia data stream transmission method, device, electronic device and storage medium

Abstract

The present invention discloses a multimedia data stream transmission method, apparatus, electronic device, and storage medium. The multimedia data stream transmission method includes the following steps: receiving a multimedia data status transmitted by a target terminal via a signaling channel, wherein the multimedia data status relates to the content of the multimedia data itself; determining a target terminal that meets a target condition as a selected target terminal, wherein the target condition includes: the multimedia data status transmitted by the target terminal matches a preset target state of the terminal; and pulling the multimedia data stream corresponding to the selected target terminal. This method avoids the transmission of multimedia data without substantive content, reduces ineffective data transmission, and significantly conserves bandwidth.

Description

Technical Field

[0001] The present invention relates to the technical field of multimedia data instant communication, and more specifically, to a multimedia data stream transmission method, device, electronic device and storage medium. Background Art

[0002] Among existing instant messaging applications, multimedia communication based on audio and video is a very common application. In existing technologies, for example, during a remote video conference, a user communicates with one or more other users via a terminal device. During the communication, the user can select and play the video and audio of other participants on the terminal device as needed.

[0003] In current video conferencing, a user terminal must pull all other users' audio and video streams to its own terminal. Simultaneously, each user terminal must also upload its own audio and video data to the server. However, in a video conference, users often only focus on a few specific speakers. Therefore, a significant portion of both the audio and video data downloaded to the terminal and uploaded to the server contains meaningless content. This insignificant data consumes both the user terminal's download bandwidth and the server's forwarding bandwidth, wasting bandwidth resources. Summary of the Invention

[0004] The present invention is to overcome at least one of the above-mentioned drawbacks of the prior art and provides a method, device, electronic device and storage medium for transmitting multimedia data streams. The technical solution adopted by the present invention is as follows.

[0005] In a first aspect, the present invention provides a method for transmitting multimedia data streams, comprising the steps of:

[0006] receiving a multimedia data status sent by a target terminal through a signaling channel, wherein the multimedia data status is a status about the content of the multimedia data itself;

[0007] Determine a target terminal that meets the target condition as a selected object terminal, wherein the target condition includes: the multimedia data state sent by the target terminal matches the target state preset by the terminal;

[0008] Pull the multimedia data stream corresponding to the selected object terminal.

[0009] In one embodiment, the method further comprises the steps of: receiving a call terminal list;

[0010] The target condition also includes: the target terminal is on the call terminal list.

[0011] In one embodiment, before pulling the multimedia data stream corresponding to the selected target terminal, the method further includes the following steps:

[0012] Sending a stream pull request to the selected target terminal, wherein the stream pull request is used to instruct the selected target terminal to send a corresponding multimedia data stream to the streaming media server;

[0013] The process of pulling the multimedia data stream corresponding to the selected target terminal is to pull the multimedia data stream from the streaming media server.

[0014] In one embodiment, before pulling the multimedia data stream corresponding to the selected target terminal, the method further includes the step of determining the data type of the multimedia data to be pulled from the selected target terminal.

[0015] In one embodiment, the steps are further included:

[0016] Analyze the state of the content of the multimedia data collected by the input device connected to the terminal to obtain the multimedia data state of the terminal;

[0017] The multimedia data status of the terminal is sent to the terminal that selects the terminal as the target terminal through the signaling channel.

[0018] In one embodiment, the method further includes the step of displaying the multimedia data status of the target terminal.

[0019] In one embodiment, the pulled multimedia data includes: video data;

[0020] The multimedia data stream transmission method further comprises the steps of:

[0021] Detect whether the video image change of the pulled video data within a preset time length exceeds a preset threshold;

[0022] When the video picture change does not exceed a preset threshold, the multimedia data status of the corresponding target terminal is displayed.

[0023] In one embodiment, the multimedia data is audio data, and the preset target state is a state representing the presence of human voice in the audio.

[0024] In one embodiment, the multimedia data is video data, and the preset target state is a state representing the presence of a person in the video.

[0025] In one embodiment, the multimedia data is audio and video data, and the preset target state is a state representing that someone is speaking in the audio and video.

[0026] In a second aspect, the present invention provides a method for transmitting multimedia data streams, comprising the steps of:

[0027] Get the terminal list that selects this terminal as the target terminal;

[0028] Acquiring multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of content of the multimedia data itself;

[0029] According to the terminal list, the multimedia data status is sent to the corresponding terminal through the signaling channel to instruct the terminal that selects the terminal as the target terminal to pull the multimedia data stream corresponding to the terminal according to the multimedia data status.

[0030] In one embodiment, the step is further included: when receiving a stream pulling request from a terminal that selects the terminal as the target terminal, sending a corresponding multimedia data stream to a streaming media server; wherein the stream pulling request is feedback information made by the terminal that selects the terminal as the target terminal based on the multimedia data status.

[0031] In a third aspect, the present invention provides a device for transmitting multimedia data streams, characterized by comprising:

[0032] a receiving module, configured to receive a multimedia data status sent by a target terminal through a signaling channel, wherein the multimedia data status is a status about the content of the multimedia data itself;

[0033] a processing module, configured to determine a target terminal that meets a target condition as a selected object terminal, wherein the target condition includes: a multimedia data state sent by the target terminal matches a target state preset by the terminal;

[0034] The stream pulling module is used to pull the multimedia data stream corresponding to the selected target terminal.

[0035] In a fourth aspect, the present invention provides a device for transmitting multimedia data streams, comprising:

[0036] An acquisition module is used to obtain a terminal list that selects the terminal as a target terminal;

[0037] a detection module, configured to obtain a multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of the content of the multimedia data itself;

[0038] The sending module is used to send the multimedia data status to the corresponding terminal through the signaling channel according to the terminal list, so as to instruct the terminal that selects the terminal as the target terminal to pull the multimedia data stream corresponding to the terminal according to the multimedia data status.

[0039] In a fifth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of the above-mentioned embodiments when executing the program.

[0040] In a sixth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method of any of the above-mentioned embodiments when the program is executed by a processor.

[0041] In the present invention, multimedia data status information obtained by judging based on the content of the multimedia data itself is received through a signaling channel. When the multimedia data status meets the preset requirements, the corresponding multimedia data stream is pulled, thereby avoiding the transmission of multimedia data without substantial content, reducing invalid data transmission, and greatly saving bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of embodiment 1.

[0043] Figure 2 It is a structural diagram of an implementation method of Example 1.

[0044] Figure 3 It is a flow chart of the second embodiment.

[0045] Figure 4 It is a structural diagram of embodiment three.

[0046] Figure 5 It is a structural diagram of embodiment 4. DETAILED DESCRIPTION

[0047] The following describes exemplary embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0048] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first, second, ..." may interchangeably represent a specific order or precedence, where permitted. It should be understood that the objects distinguished by "first, second, ..." may be interchangeable, where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0049] Example 1

[0050] See Figure 1 , Figure 1 1 is a flowchart of a multimedia data stream transmission method provided in the first embodiment of the present invention, the method includes step S110, step S120, and step S130. It should be noted that step S110, step S120, and step S130 are only reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps of the method in this embodiment.

[0051] Step S110: receiving a multimedia data status sent by a target terminal through a signaling channel, wherein the multimedia data status is a status related to the content of the multimedia data itself.

[0052] When users join a video conference through their terminals, they may be particularly interested in the speeches of specific participants, so they may pre-select one or more target terminals based on their needs. Of course, if the user does not select a target terminal, each terminal has a default program that selects certain terminals as target terminals, or simply defaults to all participating terminals as target terminals.

[0053] Specifically, the multimedia data status is the status about the content of the multimedia data itself. For example, when the multimedia data is audio data, it can be the status about whether someone is speaking, then whether someone is speaking or not is the status about the content of the audio data itself. Similarly, for video data, the presence / absence of people or the presence / absence of specific people in the video content itself is the status of the video data. Similarly, for audio and video data, it is the status of the combination of audio content and video content, such as the status of whether someone is speaking in the video screen. It should be pointed out here that the process of analyzing the content of the multimedia data itself to obtain the result is not limited in this method. It can be obtained by the target terminal's own analysis, or obtained by the target terminal from other devices, or obtained by the user through labeling on the terminal.

[0054] By receiving the multimedia data status of the target terminal, you can know the content of the multimedia data of the target terminal without receiving the entire multimedia data stream. You can pull the stream only when the content meets the requirements, exclude unnecessary audio and video data, and avoid invalid data transmission. This is especially true in scenarios where there are too many people in a video conference and group discussions are required, or when listening to a host's speech, it can reduce the impact on other terminals and greatly save bandwidth.

[0055] It should be noted that in this step, the status information may be received and forwarded by a signaling server. When there are relatively few terminals, a server may not be required, and the terminals may communicate directly through a signaling channel.

[0056] Step S120: Determine a target terminal that meets the target condition as a selected object terminal, wherein the target condition includes: the multimedia data status sent by the target terminal matches the target status preset by the terminal.

[0057] Target terminals are terminals used by specific individuals that the user is particularly interested in. However, multimedia data streams from these target terminals are not always pulled. Instead, the corresponding multimedia data streams are pulled only when these target terminals meet the target conditions. Target conditions include: the multimedia data state emitted by the target terminal matches the target state preset for this terminal. For example, if the preset target state for this terminal is someone speaking, and the multimedia data state emitted by target terminal A is also someone speaking, then the two match and meet the target conditions, and target terminal A is then determined as the selected target terminal. When the multimedia data state emitted by the target terminal matches the preset target state for this terminal, it can be assumed that the content being outputted is the content that the user truly needs.

[0058] In one embodiment, the multimedia data stream transmission method further includes: receiving a call terminal list;

[0059] The target condition also includes: the target terminal is on the call terminal list.

[0060] Typically, during a conference, users will constantly change the layout on their terminals to display the desired audio and video content. Therefore, not all target terminals will display the desired audio and video. Therefore, the user's layout information is also received. This layout information is known as the call terminal list. This call terminal list effectively represents the target terminals that the user is currently focused on, and the content being displayed by these terminals is the content the user truly needs. Therefore, this call terminal list effectively constitutes the second target condition: the target terminal must also be on the call terminal list. Only when both the multimedia data status emitted by the target terminal matches the terminal's preset target state and the target terminal is on the call terminal list, is the data from the target terminal considered valid for the user.

[0061] This embodiment readjusts the selected target terminal according to the real-time needs of the user, further avoiding invalid data transmission and saving bandwidth.

[0062] Step S130: Pull the multimedia data stream corresponding to the selected target terminal.

[0063] After the selected target terminals are determined, it is equivalent to screening out the content that the user really needs. At this time, the corresponding multimedia data streams can be pulled for these specific selected target terminals.

[0064] It should be pointed out here that the multimedia data stream in this method can be a single type of data stream or a data stream containing multiple types, so the corresponding multimedia data stream here can be all multimedia data streams, or it can be one or several types of data streams in the multimedia data stream. For example, the multimedia data includes audio and video data, but only audio data or video data is pulled when pulling.

[0065] It should be noted that in this step, the corresponding multimedia data can be pulled from the streaming media server. When there are relatively few terminals, the server may not be required and the streaming can be pulled between the terminals.

[0066] In this method, multimedia data status information obtained by judging the content of the multimedia data itself is received through a signaling channel. When the multimedia data status meets the preset requirements, the corresponding multimedia data stream is pulled, avoiding the transmission of multimedia data without substantial content, reducing invalid data transmission, and greatly saving bandwidth.

[0067] In one embodiment, before step S130, step S131 is further included.

[0068] Step S131: Send a stream pull request to the selected target terminal, wherein the stream pull request is used to instruct the selected target terminal to send a corresponding multimedia data stream to a streaming media server.

[0069] like Figure 2 As shown, this embodiment is a scenario where corresponding multimedia data is pulled from the streaming media server 100. In reality, before a terminal becomes a selected target terminal for any other terminal, its data is actually not wanted by the user and is invalid data. If its data is continuously sent to the streaming media server at this time, it will occupy the bandwidth of itself and the streaming media server. Therefore, in this embodiment, each terminal does not send its own multimedia data stream to the streaming media server before being determined as a selected target terminal. Only when it receives a request to send a stream to the selected target terminal, that is, when it determines that its data is needed, will it send the multimedia data stream to the streaming media server, thus saving bandwidth for itself and the streaming media server. Then, in step S130, the corresponding multimedia data stream is pulled from the streaming media server.

[0070] In one embodiment, before step S130, step S132 is further included.

[0071] Step S132: determining the data type of the multimedia data to be pulled from the selected target terminal.

[0072] When the multimedia data is purely audio or video, the data type of the pulled multimedia data does not need to be selected; it is audio or video data. However, when the multimedia data is audio or video, it is possible to determine the data status by combining the audio and video content, but only pull the audio data or only pull the video data. In this case, this step is required. In this step, the data type of the multimedia data to be pulled from the selected target terminal is determined.

[0073] In this step, there is no restriction on the basis for determination, and those skilled in the art can specify the basis for determination according to actual circumstances. For example, when determining the target terminal, the type of data to be pulled by the terminal has been preset, or when receiving the call terminal list, the list includes the type of data to be pulled by the call terminal, or after determining the selected object terminal, the result is returned to the user, allowing the user to decide the type of data pulled by each terminal from the result.

[0074] It should be noted here that when this embodiment is implemented simultaneously with the previous embodiment, it is preferred that this embodiment be implemented before the previous embodiment. This can avoid sending data streams of unnecessary data types to the streaming media server, thereby further saving bandwidth of itself and the streaming media server.

[0075] In one embodiment, the multimedia data stream transmission method further includes: step S140 and step S150.

[0076] Step S140, analyzing the state of the content of the multimedia data collected by the input device connected to the terminal to obtain the multimedia data state of the terminal;

[0077] Step S150: Send the multimedia data status of the terminal to the terminal that selects the terminal as the target terminal through the signaling channel.

[0078] In a video conference, the conference terminal is not only the receiver of multimedia data but also the sender of its own multimedia data. Therefore, while receiving the multimedia data status through the signaling channel, it also needs to send the multimedia data status to the terminal that has selected the terminal as the target terminal.

[0079] Multimedia data is typically transmitted to the conference terminal via a connected input device. Upon receiving multimedia data from the input device, the conference terminal analyzes the content and determines the terminal's multimedia data status. This independent analysis eliminates the need to retransmit multimedia data to an analysis device, saving bandwidth for the conference terminal and the analysis device (such as a streaming media server). It also mitigates the impact of end-to-end packet loss during network transmission, preventing the accuracy of multimedia data status analysis from being compromised.

[0080] For analysis of multimedia data content, those skilled in the art can adopt appropriate measures based on actual circumstances. For example, if the multimedia data is audio data, the terminal can determine whether there is a person speaking in the audio data based on characteristics such as decibels, timbre, and duration of silence. For video data, facial detection can be used to determine whether there is a person in the video, or further, facial recognition can be used to determine whether there is a specific person in the video, thereby obtaining the status of the video content itself (presence / absence of a person or presence / absence of a specific person). For audio and video data, the audio content and video content can be combined for analysis to obtain the analyzed status, such as combining the determination of whether there is a person in the video image and the presence of a voice in the audio to determine whether there is a person speaking in the video data.

[0081] In one embodiment, the multimedia data is audio data, and the preset target state is a state representing the presence of human voice in the audio.

[0082] In a video conference, most people do not speak, so data without voice is invalid and has no substantial content. In this embodiment, audio data with voice is pulled.

[0083] In one embodiment, the multimedia data is video data, and the preset target state is a state representing the presence of a person in the video.

[0084] In a video conference, there are usually participants in front of each terminal. The area without participants may be an empty space, so there is no need to pull the video stream of this part.

[0085] In one embodiment, the multimedia data is audio and video data, and the preset target state is a state representing that someone is speaking in the audio and video.

[0086] In one embodiment, the multimedia data stream transmission method further includes the step of displaying the multimedia data status of the target terminal.

[0087] In practice, whether a passive no-signal state is caused by network anomalies or an active no-signal state is caused by not pulling multimedia data streams, the displayed effect is the same, making it difficult for users to distinguish between the two. Therefore, this embodiment displays the multimedia data status of the target terminal, such as outputting a label such as "No signal input from the other party", allowing users to quickly determine whether the problem is due to extreme network conditions or the other party's data input, facilitating rapid troubleshooting of video conferencing.

[0088] In one embodiment, the pulled multimedia data includes video data, and the multimedia data stream transmission method further includes steps S160 and S170.

[0089] Step S160, detecting whether the video image change amount of the pulled video data within a preset time length exceeds a preset threshold;

[0090] Step S170 : When the video picture change amount does not exceed a preset threshold, the multimedia data status of the corresponding target terminal is displayed.

[0091] A screen freeze due to network problems will result in minor screen changes. A screen change will also result in minor screen changes if the content of the video captured by the subscription terminal has not changed much. However, it is difficult for users to distinguish between the above two situations. Therefore, this embodiment displays the multimedia data status of the corresponding terminal to let users know the reason for the minor screen changes, which is convenient for users to quickly troubleshoot.

[0092] Example 2

[0093] See Figure 3 , Figure 3 The flowchart of a multimedia data stream transmission method provided in the second embodiment of the present invention includes steps S210, S220, and S230. It should be noted that steps S210, S220, and S230 are only reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 3 The corresponding relationship does not limit the order of the steps of the method in this embodiment.

[0094] Step S210, obtaining a terminal list that selects the terminal as a target terminal;

[0095] Step S220, obtaining a multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of the content of the multimedia data itself;

[0096] Step S230 : sending the multimedia data status to the corresponding terminal through the signaling channel according to the terminal list, so as to instruct the terminal that has selected the terminal as the target terminal to pull the multimedia data stream corresponding to the terminal according to the multimedia data status.

[0097] Corresponding to Example 1, Example 2 is a scenario where the local multimedia data status is actively transmitted. In Example 2, the status of the local multimedia data content is obtained and sent to the terminal that has selected the local terminal as the target terminal, allowing the corresponding terminal to pull the corresponding multimedia data stream, avoiding the transmission of multimedia data without substantive content, reducing invalid data transmission, and significantly saving bandwidth.

[0098] It should be noted that the multimedia data status obtained by this method is an analysis result, and the method does not limit how this result is obtained. The result can be obtained by the target terminal itself or obtained from other devices.

[0099] In one embodiment, the multimedia data stream transmission method further includes step S240.

[0100] Step S240: upon receiving a stream pulling request from a terminal that selects the terminal as a target terminal, sending a corresponding multimedia data stream to a streaming media server; wherein the stream pulling request is feedback information made by the terminal that selects the terminal as a target terminal based on the multimedia data status.

[0101] This embodiment utilizes a streaming media server to forward multimedia data. After a terminal has sent its multimedia data status, its status may not necessarily indicate a state where the other terminal deems it is in need of streaming. At this point, it is not the other terminal's selected target terminal. Before becoming the selected target terminal for any other terminal, the data on this terminal is actually unwanted and invalid. Continuing to send this data to the streaming media server would occupy both its own and the server's bandwidth. Therefore, in this embodiment, the terminal does not send its own multimedia data stream to the streaming media server until it receives a stream-pull request to the selected target terminal, i.e., when it determines that its data is needed. This conserves bandwidth for both the terminal and the streaming media server.

[0102] It should be noted that after executing the steps in this embodiment, you can continue to execute the steps in the first embodiment.

[0103] Example 3

[0104] Corresponding to the method of embodiment 1, as Figure 4 As shown, the present invention further provides a multimedia data stream transmission device 3, comprising: a receiving module 301, a processing module 302 and a stream pulling module 303.

[0105] The receiving module 301 is configured to receive a multimedia data status sent by a target terminal through a signaling channel, wherein the multimedia data status is a status about the content of the multimedia data itself;

[0106] The processing module 302 is configured to determine a target terminal that meets a target condition as a selected target terminal, wherein the target condition includes: a multimedia data state sent by the target terminal matches a target state preset by the terminal;

[0107] The stream pulling module 303 is used to pull the multimedia data stream corresponding to the selected target terminal.

[0108] In one embodiment, the receiving module is further configured to receive a call terminal list;

[0109] The target condition also includes: the target terminal is on the call terminal list.

[0110] In one embodiment, the apparatus for transmitting multimedia data streams further includes a request module;

[0111] A request module is used to send a stream pull request to a selected target terminal, wherein the stream pull request is used to instruct the selected target terminal to send a corresponding multimedia data stream to a streaming media server;

[0112] The stream pulling module pulls the multimedia data stream from the streaming media server.

[0113] In one embodiment, the apparatus for transmitting multimedia data streams further includes an object type determination module;

[0114] The object type determination module is used to determine the data type of the multimedia data to be pulled from the selected object terminal.

[0115] In one embodiment, the apparatus for transmitting multimedia data streams further includes an analysis and sending module;

[0116] The analysis and sending module is used to analyze the status of the content of the multimedia data collected by the input device connected to the terminal to obtain the multimedia data status of the terminal; and send the multimedia data status of the terminal to the terminal that selects the terminal as the target terminal through the signaling channel.

[0117] In one embodiment, the multimedia data is audio data, and the preset target state is a state representing the presence of human voice in the audio.

[0118] In one embodiment, the multimedia data is video data, and the preset target state is a state representing the presence of a person in the video.

[0119] In one embodiment, the multimedia data is audio and video data, and the preset target state is a state representing that someone is speaking in the audio and video.

[0120] In one embodiment, the apparatus for transmitting multimedia data streams further includes a presentation module;

[0121] The display module is used to display the multimedia data status of the target terminal.

[0122] In one embodiment, the pulled multimedia data includes: video data; the apparatus for transmitting multimedia data streams further includes a picture detection module;

[0123] The picture detection module is used to detect whether the video picture change of the pulled video data within a preset time length exceeds a preset threshold. When the video picture change does not exceed the preset threshold, the multimedia data status of the corresponding target terminal is displayed.

[0124] In this device, multimedia data status information obtained by judging the content of the multimedia data itself is received through a signaling channel. When the multimedia data status meets the preset requirements, the corresponding multimedia data stream is pulled, avoiding the transmission of multimedia data without substantial content, reducing invalid data transmission, and greatly saving bandwidth.

[0125] Example 4

[0126] Corresponding to the method of embodiment 2, as Figure 5 As shown, the present invention further provides a multimedia data stream transmission device 4, comprising: an acquisition module 401, a detection module 402 and a sending module 403.

[0127] The acquisition module 401 is used to obtain a terminal list that selects the terminal as a target terminal;

[0128] The detection module 402 is configured to obtain a multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of the content of the multimedia data itself;

[0129] The sending module 403 is used to send the multimedia data status to the corresponding terminal through the signaling channel according to the terminal list, so as to instruct the terminal that selects the terminal as the target terminal to pull the multimedia data stream corresponding to the terminal according to the multimedia data status.

[0130] In one embodiment, the sending module is also used to send the corresponding multimedia data stream to the streaming media server when receiving a stream pulling request from a terminal that selects this terminal as the target terminal; wherein the stream pulling request is feedback information made by the terminal that selects this terminal as the target terminal based on the multimedia data status.

[0131] Example 5

[0132] An embodiment of the present invention further provides a storage medium storing computer instructions, which, when executed by a processor, implement the multimedia data stream transmission method of any of the above embodiments.

[0133] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0134] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0135] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, an encoder, and a computer program stored in the memory and runnable on the encoder, wherein when the encoder executes the program, any one of the multimedia data stream transmission methods in the above-mentioned embodiments is implemented.

[0136] The above-mentioned computer device receives multimedia data status information obtained by judging the content of the multimedia data itself through a signaling channel. When the multimedia data status meets the preset requirements, it pulls the corresponding multimedia data stream, avoiding the transmission of multimedia data without substantial content, reducing invalid data transmission, and greatly saving bandwidth.

[0137] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A multimedia data stream transmission method, characterized in that: The method is used for one of the terminals in a video conference, and comprises the steps of: Get the terminal list that selects this terminal as the target terminal; Acquiring multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of content of the multimedia data itself; According to the terminal list, the multimedia data status is sent to the terminal that has selected the terminal as the target terminal through a signaling channel, so as to instruct the terminal that has selected the terminal as the target terminal to decide whether to respond with a stream pull request based on the multimedia data status; wherein the stream pull request is used to instruct the terminal to send a corresponding multimedia data stream to the streaming media server; When receiving a stream pull request from a terminal that selects the terminal as a target terminal, the corresponding multimedia data stream is sent to the streaming media server; Receiving a call terminal list for a video conference, and selecting a target terminal from the call terminal list; receiving a multimedia data status sent by a target terminal through a signaling channel, wherein the multimedia data status is a status about the content of the multimedia data itself; Determine a target terminal that meets the target condition as a selected object terminal, wherein the target condition includes: the multimedia data state sent by the target terminal matches the target state preset by the terminal; Send a stream pull request to the selected target terminal; Pull the multimedia data stream corresponding to the selected target terminal from the streaming media server.

2. The multimedia data stream transmission method according to claim 1, wherein: Before pulling the multimedia data stream corresponding to the selected object terminal, the method further includes the step of determining the data type of the multimedia data to be pulled from the selected object terminal.

3. The multimedia data stream transmission method according to claim 1, wherein: The method further includes the step of displaying the multimedia data status of the target terminal.

4. The multimedia data stream transmission method according to claim 1, wherein: The pulled multimedia data includes: video data; The multimedia data stream transmission method further comprises the steps of: Detect whether the video image change of the pulled video data within a preset time length exceeds a preset threshold; When the video picture change does not exceed a preset threshold, the multimedia data status of the corresponding target terminal is displayed.

5. The multimedia data stream transmission method according to any one of claims 1 to 3, characterized in that: The multimedia data is audio data, and the preset target state is a state representing the presence of human voice in the audio.

6. The multimedia data stream transmission method according to any one of claims 1 to 4, characterized in that: The multimedia data is video data, and the preset target state is a state representing that there are people in the video.

7. The multimedia data stream transmission method according to any one of claims 1 to 4, characterized in that: The multimedia data is audio and video data, and the preset target state is a state representing that someone is speaking in the audio and video.

8. A device for transmitting multimedia data streams, characterized in that: The device is used in one of the terminals of a video conference, and includes: An acquisition module is used to obtain a terminal list that selects the terminal as a target terminal; a detection module, configured to obtain a multimedia data status of multimedia data collected by an input device connected to the terminal, wherein the multimedia data status is a status of the content of the multimedia data itself; a sending module, configured to send the multimedia data status to a terminal that has selected the terminal as a target terminal via a signaling channel based on the terminal list, so as to instruct the terminal that has selected the terminal as the target terminal to decide whether to respond to a stream pull request based on the multimedia data status; wherein the stream pull request is used to instruct the terminal to send a corresponding multimedia data stream to a streaming media server; and upon receiving a stream pull request from a terminal that has selected the terminal as a target terminal, send the corresponding multimedia data stream to the streaming media server; a receiving module, configured to receive a call terminal list for a video conference, select a target terminal from the call terminal list, and receive a multimedia data status sent by the target terminal via a signaling channel, wherein the multimedia data status is a status of the multimedia data content itself; a processing module, configured to determine a target terminal that meets a target condition as a selected object terminal, wherein the target condition includes: a multimedia data state sent by the target terminal matches a target state preset by the terminal; The request module is used to send a stream pull request to the selected target terminal; The stream pulling module pulls the multimedia data stream corresponding to the selected target terminal from the streaming media server.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images