Data stream calling method and device, equipment and storage medium
By automatically judging and retrieving the terminal data stream that was not successfully received, the problem that the terminal cannot receive the data stream in a network conference is solved, and rapid recovery and sensorless user experience is achieved.
Patent Information
- Application Number
- CN202410839854.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-26
- Publication Date
- 2024-11-01
AI Technical Summary
In online meetings, some terminals may not be able to receive data streams from the conference speakers, resulting in the inability to display the conference screen or the conference audio cannot be heard. The prior art requires users to manually re-enter the conference or restart the terminal, resulting in a long recovery delay.
By judging the stream regulating trigger event, the data flow identification information of the currently successfully received terminal and the identification information of the expected receiving terminal are obtained, and the terminal that has not been successfully received is automatically locked and retrieved to automatically restore the data flow.
It realizes the rapid recovery of data flow without user participation in online meetings, shortens recovery time and improves user experience.
Smart Images

Figure CN118890428A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for retrieving data streams. Background Art
[0002] Network conferencing uses the Internet as a medium, allowing users to achieve face-to-face communication through the Internet without the limitations of time and space. The network conferencing system uses advanced audio and video encoding and decoding technology to ensure clear voice and video effects of the product. In this way, participants can access the network conference through the terminal, see the conference screen, and hear the conference audio.
[0003] However, in an online conference, some terminals of the participants may not be able to receive the data stream of the terminal of the conference speaker for some reason, so that the conference screen cannot be displayed and the conference audio cannot be heard. In the prior art, in this case, the user needs to rejoin the conference or restart the terminal, which results in a long delay for the terminal of the participant to resume displaying the audio and video of the online conference. Summary of the invention
[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus, device and storage medium for retrieving a data stream that overcome the above problems or at least partially solve the above problems.
[0005] In order to solve the above problem, an embodiment of the present invention discloses a method for retrieving a data stream, which is applied to a first device. The method includes:
[0006] Step 1: Determine whether a flow regulation triggering event occurs, and if the flow regulation triggering event occurs, execute the following steps 2 to 4:
[0007] Step 2: Acquire first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0008] Step 3: determining, based on the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0009] Step 4: If the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
[0010] Optionally, the flow regulation triggering event includes at least one of the following:
[0011] A first trigger event is used to indicate that a first link is disconnected, where the first link is a link between a network domain accessed by the first device and another network domain, where the other network domain is a network domain accessed by a terminal that expects to receive a data stream of the first terminal;
[0012] A second trigger event, used to indicate that the traffic of the first link reaches a first threshold;
[0013] A third trigger event is used to instruct setting the first terminal to send a data stream;
[0014] The fourth trigger event is used to indicate that after the first terminal is set to send the data stream, there is a terminal that fails to successfully receive the data stream of the first terminal.
[0015] Optionally, the determining whether the first trigger event, the second trigger event, and the fourth trigger event occur includes:
[0016] In the case of receiving third information, determining that at least one of the first trigger event, the second trigger event, and the fourth trigger event occurs, the third information being used to indicate that a decoding channel for at least one third terminal to receive the data stream of the first terminal is cleared;
[0017] In a case where the third information is not received, it is determined that the first trigger event, the second trigger event, and the fourth trigger event have not occurred.
[0018] Optionally, in the case where the second terminal exists, calling the data stream of the first terminal for the second terminal includes:
[0019] When the second terminal exists and the first trigger event or the second trigger event occurs, the link between the network domain accessed by the first device and the network domain accessed by the second terminal is reselected, and the data stream of the first terminal is called for the second terminal through the reselected link.
[0020] Optionally, the method further comprises:
[0021] After the execution of step 4 is completed, the method returns to step 2 at a predetermined interval, and executes step 3 and step 4 in sequence until the execution times of step 2 to step 4 reach a second threshold, and then stops executing step 2 to step 4.
[0022] Optionally, the step of calling the data stream of the first terminal for the second terminal includes:
[0023] Sending a first instruction to a third device;
[0024] The third device is a server in the network domain accessed by the first device;
[0025] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0026] Optionally, the step of calling the data stream of the first terminal for the second terminal includes:
[0027] Sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0028] receiving a first instruction sent by the second device;
[0029] Sending the first instruction to a third device;
[0030] The third device is a server in the network domain accessed by the first device;
[0031] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0032] An embodiment of the present invention discloses a method for retrieving a data stream, which is applied to a second device. The method includes:
[0033] Receiving fourth information sent by the first device, wherein the fourth information is used to indicate that the second terminal currently expects to receive a data stream of the first terminal but fails;
[0034] A first instruction is sent to the first device, wherein the first instruction is used to instruct a third device to obtain a data stream of the first terminal and send the data stream of the first terminal to the second terminal, and the third device is a server in a network domain accessed by the first device.
[0035] An embodiment of the present invention discloses a method for retrieving a data stream, which is applied to a fourth device. The method includes:
[0036] In a case where the third terminal fails to successfully receive the data stream of the first terminal, sending third information to the first device, so that the first device obtains the first information according to the third information, and determines, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but fails to receive it successfully, and in the case where the second terminal exists, calls the data stream of the first terminal for the second terminal;
[0037] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared;
[0038] The first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0039] The second information includes identification information of a terminal that desires to receive the data stream of the first terminal.
[0040] An embodiment of the present invention discloses a data stream retrieving device, which is applied to a first device, and the device includes:
[0041] The judgment module is used to execute step 1: judge whether a flow adjustment triggering event occurs, and in the case of the flow adjustment triggering event, trigger the acquisition module, the determination module, and the data flow calling module to respectively execute the following steps 2 to 4:
[0042] The acquisition module is used to execute step 2: acquire first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0043] The determination module is configured to execute step three: determining, based on the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0044] The data stream retrieving module is used to execute step 4: when the determining module determines that the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
[0045] Optionally, the flow regulation triggering event includes at least one of the following:
[0046] A first trigger event is used to indicate that a first link is disconnected, where the first link is a link between a network domain accessed by the first device and another network domain, where the other network domain is a network domain accessed by a terminal that expects to receive a data stream of the first terminal;
[0047] A second trigger event, used to indicate that the traffic of the first link reaches a first threshold;
[0048] A third trigger event is used to instruct setting the first terminal to send a data stream;
[0049] The fourth trigger event is used to indicate that after the first terminal is set to send the data stream, there is a terminal that fails to successfully receive the data stream of the first terminal.
[0050] Optionally, the judging module is specifically used for:
[0051] In the case of receiving third information, determining that at least one of the first trigger event, the second trigger event, and the fourth trigger event occurs, the third information being used to indicate that a decoding channel for at least one third terminal to receive the data stream of the first terminal is cleared;
[0052] In a case where the third information is not received, it is determined that the first trigger event, the second trigger event, and the fourth trigger event have not occurred.
[0053] Optionally, the data stream calling module is specifically used for:
[0054] When the second terminal exists and the first trigger event or the second trigger event occurs, the link between the network domain accessed by the first device and the network domain accessed by the second terminal is reselected, and the data stream of the first terminal is called for the second terminal through the reselected link.
[0055] Optionally, the device further comprises:
[0056] The control module is used to control the data stream retrieval module to execute step 4 after completion, control the acquisition module to execute step 2 at a predetermined interval, and sequentially control the determination module to execute step 3 and the data stream retrieval module to execute step 4, until the execution times of steps 2 to 4 reach a target threshold, and then stop executing steps 2 to 4.
[0057] Optionally, when the data stream calling module calls the data stream of the first terminal for the second terminal, it is specifically used to:
[0058] Sending a first instruction to a third device;
[0059] The third device is a server in the network domain accessed by the first device;
[0060] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0061] Optionally, when the data stream calling module calls the data stream of the first terminal for the second terminal, it is specifically used to:
[0062] Sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0063] receiving a first instruction sent by the second device;
[0064] Sending the first instruction to a third device;
[0065] The third device is a server in the network domain accessed by the first device;
[0066] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0067] An embodiment of the present invention discloses a data stream retrieving device, which is applied to a second device, and the device includes:
[0068] A receiving module, configured to receive fourth information sent by the first device, wherein the fourth information is used to indicate that the second terminal currently expects to receive a data stream of the first terminal but fails;
[0069] The first sending module is used to send a first instruction to the first device, wherein the first instruction is used to instruct a third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal, and the third device is a server in the network domain accessed by the first device.
[0070] The embodiment of the present invention discloses a data stream scheduling device, which is applied to a fourth device. The device includes:
[0071] a second sending module, configured to send third information to the first device when the third terminal fails to successfully receive the data stream of the first terminal, so that the first device obtains the first information according to the third information, and determines, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but fails to receive the data stream, and retrieves the data stream of the first terminal for the second terminal when the second terminal exists;
[0072] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared;
[0073] The first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0074] The second information includes identification information of a terminal that desires to receive the data stream of the first terminal.
[0075] The embodiment of the present invention further discloses an electronic device, including:
[0076] one or more processors; and
[0077] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the above-mentioned method for retrieving the data stream.
[0078] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the above-mentioned method for retrieving data streams.
[0079] An embodiment of the present invention discloses a data stream retrieving system, the data stream retrieving system comprising a first device and a third device, the third device being a server in a network domain accessed by the first device;
[0080] Wherein, the first device is used for:
[0081] Determine whether a flow adjustment triggering event occurs, and if the flow adjustment triggering event occurs, obtain first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0082] Determine, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0083] In the case where the second terminal exists, sending a first instruction to the third device, wherein the first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal;
[0084] The third device is used for:
[0085] The first instruction is received, and based on the first instruction, a data stream of the first terminal is acquired, and the data stream of the first terminal is sent to the second terminal.
[0086] Optionally, the system further includes a fourth device, configured to: when the third terminal fails to successfully receive the data stream of the first terminal, send third information to the first device, so that the first device determines whether a flow adjustment triggering event occurs according to the third information;
[0087] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared.
[0088] An embodiment of the present invention discloses a data stream retrieving system, the data stream retrieving system comprising a first device, a second device and a third device, the third device being a server in a network domain accessed by the first device;
[0089] Wherein, the first device is used for:
[0090] Determine whether a flow adjustment triggering event occurs, and if the flow adjustment triggering event occurs, obtain first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0091] Determine, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0092] In a case where the second terminal exists, sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0093] The second device is used for:
[0094] receiving the fourth information;
[0095] Sending a first instruction to the first device, wherein the first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal;
[0096] The first device is also used for:
[0097] receiving the first instruction sent by the second device, and sending the first instruction to the third device;
[0098] The third device is used for:
[0099] The first instruction is received, and based on the first instruction, a data stream of the first terminal is acquired, and the data stream of the first terminal is sent to the second terminal.
[0100] Optionally, the system further includes a fourth device, configured to: when the third terminal fails to successfully receive the data stream of the first terminal, send third information to the first device, so that the first device determines whether a flow adjustment triggering event occurs according to the third information;
[0101] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared.
[0102] The embodiments of the present invention include the following advantages:
[0103] In an embodiment of the present invention, a first device is capable of determining whether a flow modulation triggering event occurs, and in the event of a flow modulation triggering event, obtaining first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal, thereby determining a second terminal that currently expects to receive the data stream of the first terminal but fails to do so based on the first information and the stored second information, wherein the second information includes identification information of a terminal that expects to receive the data stream of the first terminal; and then, in the event that it is determined that a second terminal exists, the data stream of the first terminal is retrieved for the second terminal.
[0104] It can be seen that in an embodiment of the present invention, the identification information of the terminal that expects to receive the data stream of the first terminal can be stored, and in the event of a flow adjustment event, by comparing it with the identification information of the terminal that currently successfully receives the data stream of the first terminal, it can be known whether there is a second terminal that expects to receive the data stream of the first terminal but has not successfully received it. If there is a second terminal, it is triggered to call the data stream of the first terminal for the second terminal. Therefore, in an embodiment of the present invention, in an online conference, when a terminal of a participant fails to successfully receive the data stream of the terminal of the conference speaker, the first device can lock the terminal that needs flow adjustment by automatically comparing the expected recipient and the current successful recipient, and the first device can trigger automatic flow adjustment for the terminal. This process does not require user participation, not only takes a short time, but also is imperceptible to the user. BRIEF DESCRIPTION OF THE DRAWINGS
[0105] Figure 1 is a schematic diagram of a ring network scenario to which an embodiment of the present invention is applicable;
[0106] Figure 2 It is a schematic diagram of the architecture of a visual network to which an embodiment of the present invention is applicable;
[0107] Figure 3 It is one of the step flow charts of an embodiment of a method for retrieving a data stream of the present invention;
[0108] Figure 4 This is the second step flow chart of an embodiment of a method for retrieving a data stream of the present invention;
[0109] Figure 5 This is a step flow chart of a method for retrieving a data stream according to an embodiment of the present invention;
[0110] Figure 6 It is one of the schematic diagrams involved in the specific implementation of the method for retrieving data streams of the present invention;
[0111] Figure 7 This is the second schematic diagram of the specific implementation of the method for retrieving data streams of the present invention;
[0112] Figure 8It is one of the structural block diagrams of an embodiment of a data stream retrieving device of the present invention;
[0113] Fig. 9 This is the second structural block diagram of the data stream calling device embodiment of the present invention;
[0114] Fig.10 This is the third structural block diagram of the data stream calling device embodiment of the present invention. DETAILED DESCRIPTION
[0115] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0116] At present, in an online conference, some terminals of the participants may not be able to receive the data stream of the terminal of the conference speaker for some reason, so that the conference screen cannot be displayed and the conference audio cannot be heard. In the prior art, in this case, the user needs to rejoin the conference or restart the terminal, which results in a long delay for the terminal of the participant to resume displaying the audio and video of the online conference.
[0117] For example, in a visual network, multiple nodes can form a ring network, and a visual network terminal can access the visual network by accessing a certain node, so that multiple visual network terminals accessing different nodes can participate in a visual network conference. Among them, an autonomous domain in the visual network can be used as a node.
[0118] For example, in Figure 1 In the example, after nodes A, B, and C form a ring network, the link from node A to node C is: A→C or A→B→C. If terminal A1 is connected to node A, and terminals C1 and C2 are connected to node C; if terminal A1 is the sender of the data stream (i.e., the user of terminal A1 is the speaker), and the user of terminal C1 is the chairman, then terminals C1 and C2 need to watch the video and sound of terminal A1, and terminal C2 needs to hear the voice of terminal C1. When the link is switched from A→C to A→B→C, terminal C1 and terminal side 2 on node C will not exit, but the decoding channel for terminals C1 and C2 on node C to receive the data stream of terminal A1 will be cleared. In this way, when terminals C1 and C2 watch terminal A1, no stream is displayed, but terminal C2 can hear the voice of the chairman of terminal C1.
[0119] At present, when the conference management client that initiates the conference actively switches the speaker, it will adjust the stream for all participants. When the terminal exits the conference and joins the conference, it will adjust the stream for the participant to watch the chairman by default. It can be seen that the scenario described in the above example is not within the functional scope of the current conference management client. That is, the current conference management client does not know the decoding channel status of each terminal, does not know that the terminal cannot receive the data stream of the speaker's terminal, and will not re-adjust the stream.
[0120] It can be seen from this that for terminals connected to the ring network, when participating in an online conference, if the link between a terminal and the terminal to which the data stream it receives belongs is disconnected, the terminal will display no stream; and if it needs to be restored, the user can only manually rejoin the conference, but this not only requires user intervention, but also takes a long time to recover.
[0121] In addition, when the conference management client that initiates the meeting actively switches the speaker, some terminals may not be able to successfully receive the data stream of the switched speaker's terminal. In this case, the user can only manually rejoin the meeting. Similarly, this not only requires user intervention, but also takes a long time to recover.
[0122] The embodiments of the present invention can store the identification information of the terminal that expects to receive the data stream of the first terminal, and in the event of a flow adjustment event, by comparing it with the identification information of the terminal that currently successfully receives the data stream of the first terminal, it can be known whether there is a second terminal that expects to receive the data stream of the first terminal but has not successfully received it. If there is a second terminal, it triggers the second terminal to call the data stream of the first terminal. Therefore, in an embodiment of the present invention, in an online conference, when a terminal of a participant fails to successfully receive the data stream of the terminal of the conference speaker, the first device can lock the terminal that needs flow adjustment by automatically comparing the expected recipient and the current successful recipient, and the first device can trigger automatic flow adjustment for the terminal. This process does not require user participation and is not only time-consuming but also imperceptible to the user.
[0123] Therefore, the embodiments of the present invention can solve the problem of no streaming on the participating terminals of the network conference in the above two scenarios, and shorten the recovery time so that users do not feel it.
[0124] It can be understood that the embodiments of the present invention can be applied to the visual network described above, and can also be applied to other networks; the embodiments of the present invention can be applied to the ring network scenario described above, and can also be applied to other link scenarios.
[0125] The data processing method provided by the embodiment of the present invention is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.
[0126] An embodiment of the present invention provides a method for retrieving a data stream, which is applied to a first device.
[0127] For example, when the embodiments of the present invention are applied to a visual network, they can be applied to Figure 2 The visual network architecture shown in FIG. 1 includes: a management client, a management server, multiple nodes ( Figure 2A network topology structure is formed by nodes A, B, and C as an example, wherein each node includes a fusion control unit. The visual network terminal can access the corresponding node by accessing the fusion control unit. The conference management server accesses one of the nodes in the network topology structure. The conference management client can communicate with the fusion control unit in the node accessed by the conference management server through the conference management server, or directly communicate with the fusion control unit in the node accessed by the conference management server.
[0128] In addition, when the data scheduling method of the embodiment of the present invention is applied to the visual networking architecture described above, the first device may be a device installed with a management server, or a device installed with a management client.
[0129] like Figure 3 As shown, the method includes the following steps 301 to 304:
[0130] Step 301: Determine whether a flow regulation triggering event occurs.
[0131] It can be seen that in an embodiment of the present invention, a flow adjustment trigger event is used to trigger the terminal that currently successfully receives the data stream of the first terminal, thereby comparing it with the stored terminal that expects to receive the data stream of the first terminal, and obtaining the second terminal that currently expects to receive the data stream of the first terminal but has not successfully received it, thereby realizing flow adjustment for the second terminal.
[0132] Step 302: When the flow regulation triggering event occurs, obtain first information.
[0133] That is, after executing step 301, when the flow regulation triggering event occurs, execute steps 302 to 304.
[0134] The first information includes identification information of a terminal that currently successfully receives the data stream of the first terminal.
[0135] For example, in an online conference, the first terminal serves as the conference speaker's terminal, and other terminals participating in the conference need to receive the data stream of the first terminal; the terminal that receives the data stream of the first terminal can display it based on the received data stream; for example, when the data stream includes video data, the video screen can be displayed; when the data stream includes audio data, the audio can be played.
[0136] In addition, the identification information of the terminal may be the communication number of the terminal.
[0137] Step 303: Determine, based on the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not received it successfully.
[0138] Among them, the second information includes identification information of the terminal that expects to receive the data stream of the first terminal; it can be seen that by comparing the first information and the second information, the terminal identification information that exists in the second information but does not exist in the first information can be obtained, and the identification information is the identification information of the terminal that expects to receive the data stream of the first terminal but has not been successfully received at present.
[0139] Therefore, if there is a second terminal that wants to receive the data stream of the first terminal but has not yet successfully received the data stream, it is necessary to retrieve the data stream of the first terminal for the second terminal.
[0140] For example, in a network conference, there is a second terminal that expects to receive the data stream of the first terminal but fails to receive the data stream of the first terminal. If the data stream includes video data, the second terminal cannot display the video screen; if the data stream includes audio data, the second terminal cannot play the audio. In the prior art, the user needs to re-operate the second terminal to join the conference before the data stream of the first terminal can be received again, which will delay for a long time. In this case, the embodiment of the present invention can automatically adjust the stream for the second terminal after determining the existence of the second terminal, so as to promptly restore the second terminal to display the conference screen and play the conference audio.
[0141] It is understandable that the second terminal may include one or more terminals. When the second terminal includes multiple terminals (i.e., there are multiple terminals that expect to receive the data stream of the first terminal but have not yet successfully received it), the data stream of the first terminal may be retrieved for these terminals in batches.
[0142] Step 304: If the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
[0143] It can be seen from the above steps 301 to 304 that in an embodiment of the present invention, the identification information of the terminal that expects to receive the data stream of the first terminal can be stored, and in the event of a flow adjustment event, by comparing it with the identification information of the terminal that currently successfully receives the data stream of the first terminal, it can be known whether there is a second terminal that expects to receive the data stream of the first terminal but has not successfully received it. If there is a second terminal, it is triggered to call the data stream of the first terminal for the second terminal. Therefore, in an embodiment of the present invention, in an online conference, when a terminal of a participant fails to successfully receive the data stream of the terminal of the conference speaker, the first device can lock the terminal that needs flow adjustment by automatically comparing the expected recipient and the current successful recipient, and the first device can trigger automatic flow adjustment for the terminal. This process does not require user participation, not only takes a short time, but also is imperceptible to the user.
[0144] In an optional embodiment of the present invention, the flow regulation triggering event includes at least one of the following:
[0145] A first trigger event is used to indicate that a first link is disconnected, where the first link is a link between a network domain accessed by the first device and another network domain, where the other network domain is a network domain accessed by a terminal that expects to receive a data stream of the first terminal;
[0146] A second trigger event, used to indicate that the traffic of the first link reaches a first threshold;
[0147] A third trigger event is used to instruct setting the first terminal to send a data stream;
[0148] The fourth trigger event is used to indicate that after the first terminal is set to send the data stream, there is a terminal that fails to successfully receive the data stream of the first terminal.
[0149] It can be seen that when at least one of the first trigger event, the second trigger event, the third trigger event, and the fourth trigger event occurs, the above steps 302 to 304 can be triggered to implement flow adjustment. It can be seen that in the embodiment of the present invention, the method of automatically adjusting the flow for the terminal by comparing the expected receiver and the current successful receiver to lock the terminal that needs flow adjustment and triggering the first device to automatically adjust the flow for the terminal can be applied to many flow adjustment scenarios.
[0150] In one embodiment of the present invention, the determining whether the first trigger event, the second trigger event, and the fourth trigger event occur includes:
[0151] In the case of receiving third information, determining that at least one of the first trigger event, the second trigger event, and the fourth trigger event occurs, the third information being used to indicate that a decoding channel for at least one third terminal to receive the data stream of the first terminal is cleared;
[0152] In a case where the third information is not received, it is determined that the first trigger event, the second trigger event, and the fourth trigger event have not occurred.
[0153] It should be noted that when the third terminal fails to successfully receive the data stream of the first terminal within a predetermined time period, the decoding channel for the third terminal to receive the data stream of the first terminal will be cleared; therefore, when the first device receives the above-mentioned third information, it can be determined that at least one of the above-mentioned first trigger event, second trigger event, and fourth trigger event has occurred.
[0154] Therefore, illustratively, for the above step 302, the obtaining of the first information may include the following step D-1 or D-2:
[0155] Step D-1: upon receiving third information, acquiring the first information according to the third information, wherein the third information is used to indicate that a decoding channel for at least one third terminal to receive a data stream of the first terminal is cleared;
[0156] For example, embodiments of the present invention are applied to Figure 2 In the visual network architecture shown, the first device is a device installed with a conference management client or a conference management server. If the terminal C1 and the terminal C2 connected to the C node fail to successfully receive the data stream of the terminal A1 within a predetermined time period, the fusion control unit of the C node can clear the decoding channel for the terminal C1 and the terminal C2 to receive the data stream of the terminal A1, and report to the conference management server or the conference management client that the decoding channel for the terminal C1 and the terminal C2 to receive the data stream of the terminal A1 is cleared.
[0157] The path in which the fusion control unit of the C node reports to the conference management server that the decoding channel of the data stream of the terminal C1 and the terminal C2 receiving the terminal A1 is cleared is: the fusion control unit of the C node → the fusion control unit of the A node → the conference management server;
[0158] The fusion control unit of node C reports to the conference management client that the decoding channel of the data stream received by terminals C1 and C2 from terminal A1 is cleared. The path is: fusion control unit of node C → fusion control unit of node A → conference management server → conference management client.
[0159] It should be noted that if terminal C1 and terminal C2 fail to successfully receive the data stream of terminal A1 within the predetermined time period, it is because the link from node A to node C is disconnected. In this case, the reporting path of the information "the decoding channel of terminal C1 and terminal C2 receiving terminal A1 is cleared" from node C to node A is: the fusion control unit of node C → the fusion control unit of node B → the fusion control unit of node A.
[0160] It can be seen from this that when the first device receives the above-mentioned third information and learns that the decoding channel of at least one third terminal for receiving the data stream of the first terminal is cleared, the identification information of the terminal that currently successfully receives the data stream of the first terminal can be updated according to the third information, that is, the identification information of the third terminal is removed from the identification information of the terminal that currently successfully receives the data stream of the first terminal, and then the accurate first information is obtained, thereby triggering the execution of the above-mentioned steps 303 to 304, and realizing automatic flow adjustment of the second terminal that expects to receive the data stream of the first terminal but has not successfully received it.
[0161] It can be seen that in the embodiment of the present invention, the first device receives the third information to trigger automatic flow adjustment of the data stream. In this way, the flow adjustment timing is more timely, and the terminal that needs flow adjustment determined based on the third information is more accurate.
[0162] It should be noted that if, based on the above-mentioned third information, it can be determined which specific terminals have their decoding channels for receiving the data stream of the first terminal cleared, for example, the third information indicates the decoding channels affected and the terminals using these decoding channels, then based on the third information, it can be determined which terminals currently expect to receive the data stream of the first terminal but have not successfully received it. In this case, this flow adjustment may not involve comparing the expected viewing party and the party currently viewing, and the flow may be directly adjusted to the terminal determined according to the third information; however, if multiple flow adjustments are required to ensure the success rate of the flow adjustment, then after receiving the third information, the receiving party may be updated according to the third information, so that the next time the flow adjustment is performed by comparing the expected viewing party with the party currently viewing.
[0163] However, if it is impossible to determine based on the third information which terminals' decoding channels for receiving the data stream of the first terminal have been cleared, for example, the third information only indicates the identifiers of the cleared decoding channels, and the terminals using these channels to receive the data stream of the first terminal cannot be determined based solely on the identifiers of these cleared decoding channels, then after receiving the third information, it is necessary to adjust the stream by comparing the expected viewer and the current viewer.
[0164] Step D-2: Acquire the first information when the first terminal is set to send a data stream.
[0165] It should be noted that if the first terminal is set to send a data stream, then the data stream of the first terminal needs to be retrieved to other terminals. For example, in a network conference, if the first terminal is set to send a data stream, it means that the user of the first terminal is a speaker, and other terminals need to receive the data stream of the first terminal.
[0166] For example, embodiments of the present invention are applied to Figure 2 In the visual network architecture shown, the first device is a device installed with a conference management client or a conference management server, terminals C1 and C2 are connected to the C node, terminal A1 is connected to the A node, and terminals C1, C2, and A1 participate in the same visual network conference. If terminal A1 is set to send a data stream (that is, the user of terminal A1 is set as a speaker), then terminals C1 and C2 need to receive the data stream of terminal A1. In this case, the terminal that currently successfully receives the data stream of terminal A1 can be triggered to obtain the terminal, thereby comparing it with the terminal that expects to receive the data stream of terminal A1, and then the data stream of terminal A1 is retrieved for the terminals that expect to receive the data stream of terminal A1 (that is, terminals C1 and C2).
[0167] It can be seen that in the embodiments of the present invention, when setting a terminal for sending a data stream, a method of obtaining first information and then comparing the first information with the second information can be adopted to achieve automatic flow adjustment of the data stream.
[0168] It can be seen from the above D-1 to D-2 that the method of locking the terminal that needs to be adjusted by comparing the expected recipient and the current successful recipient, and triggering the first device to automatically adjust the flow for the terminal is not only applicable to the situation where the decoding channel of the data stream received by the first terminal is cleared, but also applicable to the situation where the terminal for sending the data stream is set, thereby making the flow adjustment method more applicable to more scenarios.
[0169] In an optional embodiment of the present invention, the method further comprises:
[0170] After step 304 is executed, the process returns to step 302 at a predetermined interval, and executes steps 303 and 304 in sequence until the number of executions of steps 302 to 304 reaches a target threshold, at which time the execution of steps 302 to 304 is stopped.
[0171] It can be seen from this that when it is necessary to call the data stream of the first terminal for the second terminal, in an embodiment of the present invention, the flow is adjusted a total of a number of times indicated by the target threshold, and a predetermined time interval is set between two adjacent times. If it still fails after calling the target threshold indicated number of times, the flow will not be adjusted further, thereby reducing unnecessary interaction instructions and further reducing the impact on the service performance of related devices.
[0172] In an optional embodiment of the present invention, in the above step 304, calling the data stream of the first terminal for the second terminal includes:
[0173] Sending a first instruction to a third device;
[0174] The third device is a server in the network domain accessed by the first device;
[0175] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0176] It can be seen from this that when the first device retrieves the data stream of the first terminal for the second terminal, the first device can instruct the server in the network domain it accesses to obtain the data stream of the first terminal, thereby sending the obtained data stream of the first terminal to the second terminal.
[0177] It can be seen that in an embodiment of the present invention, after the first device determines the second terminal that expects to receive but fails to successfully receive the data stream of the first terminal, the first device can decide whether to adjust the flow, that is, the first device issues a first instruction for adjusting the flow, so that the device receiving the first instruction adjusts the flow for the second terminal; a specific and feasible flow adjustment process is provided when the first device triggers the flow adjustment according to the comparison result of the first information and the second information.
[0178] Exemplarily, embodiments of the present invention are applied to Figure 2In the visual networking architecture shown, if the first device is a device installed with a management server, the third device is a fusion control unit in the A node (i.e., the autonomous domain) to which the management server is connected. In this case, if the terminal C1 and the terminal C2 in the C node call the data stream of the terminal A1, the specific implementation process can be:
[0179] The management server sends a first instruction to the fusion control unit of node A, so that the fusion control unit of node A requests data stream from terminal A1. After the fusion control unit of node A receives the data stream of terminal A1, the data stream of terminal A1 is sent to the fusion control unit of node C, and the fusion control unit of node C sends the data stream of terminal A1 to terminals C1 and C2.
[0180] In an optional embodiment of the present invention, in the above step 304, calling the data stream of the first terminal for the second terminal includes:
[0181] Sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0182] receiving a first instruction sent by the second device;
[0183] Sending the first instruction to a third device;
[0184] The third device is a server in the network domain accessed by the first device;
[0185] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0186] In addition, when the second terminal fails to successfully receive the data stream of the first terminal, the decoding channel for the second terminal to receive the data stream of the first terminal will be cleared. Therefore, the fourth information can be represented by the following content: the decoding channel for the second terminal to receive the data stream of the first terminal is cleared.
[0187] It can be seen from this that when the first device calls the data stream of the first terminal for the second terminal, the first device can report to the second device the information that the second terminal has not successfully received the data stream of the first terminal, so that the second device sends the above-mentioned first instruction to the server in the network domain connected to the first device through the first device, instructing the server in the network domain connected to the first device to obtain the data stream of the first terminal, and then sending the obtained data stream of the first terminal to the second terminal.
[0188] It can be seen that in an embodiment of the present invention, after the first device determines the second terminal that expects to receive but fails to successfully receive the data stream of the first terminal, the situation can be reported to the second device, and the second device decides whether to adjust the flow, that is, the second device issues a first instruction for adjusting the flow, so that the device receiving the first instruction adjusts the flow for the second terminal; another specific and feasible flow adjustment process is provided when the first device triggers the flow adjustment according to the comparison result of the first information and the second information.
[0189] It should be noted that the specific forms of the first instruction sent by the second device to the first device and the first instruction sent by the first device to the third device may be the same or different.
[0190] Exemplarily, embodiments of the present invention are applied to Figure 2 In the visual networking architecture shown, if the first device is a device installed with a conference management server, the second device is a device installed with a conference management client, and the third device is a fusion control unit in the A node (i.e., the autonomous domain) to which the conference management server is connected, in this case, if the terminal C1 and the terminal C2 in the C node call the data stream of the terminal A1, the specific implementation process can be:
[0191] The conference management server notifies the conference management client that terminals C1 and C2 have not successfully received the data stream of terminal A1 (for example, the decoding channel of terminal A1 received by terminals C1 and C2 can be cleared to indicate this information). In this way, the conference management client can send a first instruction for adjusting the flow to the conference management server, so that the conference management server sends the first instruction to the fusion control unit of node A, thereby causing the fusion control unit of node A to request the data stream from terminal A1. After the fusion control unit of node A receives the data stream of terminal A1, the data stream of terminal A1 is sent to the fusion control unit of node C, thereby causing the fusion control unit of node C to send the data stream of terminal A1 to terminals C1 and C2.
[0192] The embodiment of the present invention further provides a method for retrieving a data stream, which is applied to a second device. For example, the embodiment of the present invention is applied to Figure 2 In the visual networking architecture shown, the second device may be a device with a management client installed, and the first device may be a device with a management server installed.
[0193] like Figure 4 As shown, the method includes the following steps 401 to 402:
[0194] Step 401: Receive fourth information sent by the first device.
[0195] The fourth information is used to indicate that the second terminal currently expects to receive the data stream of the first terminal but fails.
[0196] In addition, the first device can determine the second terminal that has not successfully received the data stream of the first terminal by comparing the first information and the second information, and thus send the above-mentioned fourth information to the second device when the second terminal exists; the first information includes the identification information of the terminal that currently successfully receives the data stream of the first terminal; the second information includes the identification information of the terminal that expects to receive the data stream of the first terminal.
[0197] It can be seen that by comparing the first information and the second information, the terminal identification information that exists in the second information but not in the first information can be obtained. The identification information is the identification information of the terminal that expects to receive the data stream of the first terminal but has not successfully received it.
[0198] In addition, when the second terminal fails to successfully receive the data stream of the first terminal, the decoding channel for the second terminal to receive the data stream of the first terminal will be cleared. Therefore, the above fourth information can be represented by the following content: the decoding channel for the second terminal to receive the data stream of the first terminal is cleared.
[0199] Step 402: Send a first instruction to the first device.
[0200] The first instruction is used to instruct a third device to obtain a data stream of the first terminal and send the data stream of the first terminal to the second terminal. The third device is a server in a network domain accessed by the first device.
[0201] It can be seen from steps 401 to 402 that in an embodiment of the present invention, after the first device determines the second terminal that expects to receive but fails to successfully receive the data stream of the first terminal, the situation can be reported to the second device, and the second device decides whether to adjust the flow, that is, the second device issues a first instruction for adjusting the flow, so that the device receiving the first instruction adjusts the flow for the second terminal. It can be seen that in an embodiment of the present invention, in an online conference, when a terminal of a participant fails to successfully receive the data stream of the terminal of the conference speaker, the first device can notify the second device of the situation, and the second device can trigger automatic flow adjustment for the terminal. This process does not require user participation, not only takes a short time, but also is imperceptible to the user.
[0202] The embodiment of the present invention further provides a method for retrieving a data stream, which is applied to a fourth device, where the fourth device may be a server in a network domain accessed by a third terminal. Figure 2 In the visual networking architecture shown, the fourth device can be a fusion control unit in the autonomous domain to which the third terminal accesses.
[0203] like Figure 5 As shown, the method includes the following step 501:
[0204] Step 501: When the third terminal fails to successfully receive the data stream of the first terminal, third information is sent to the first device, so that the first device obtains the first information based on the third information, and determines the second terminal that currently expects to receive the data stream of the first terminal but fails to receive it successfully based on the first information and the stored second information, and when the second terminal exists, retrieves the data stream of the first terminal for the second terminal.
[0205] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared;
[0206] The first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0207] The second information includes identification information of a terminal that desires to receive the data stream of the first terminal.
[0208] It should be noted that, when the third terminal fails to successfully receive the data stream of the first terminal within a predetermined time period, the decoding channel through which the third terminal receives the data stream of the first terminal will be cleared.
[0209] As can be seen from the above step 501, in an embodiment of the present invention, the first device can store the identification information of the terminal that expects to receive the data stream of the first terminal, and when receiving the above third information, by comparing it with the identification information of the terminal that currently successfully receives the data stream of the first terminal, it can be known whether there is a second terminal that expects to receive the data stream of the first terminal but has not successfully received it, and if there is a second terminal, it is triggered to retrieve the data stream of the first terminal for the second terminal. Therefore, in an embodiment of the present invention, in an online conference, when a terminal of a participant fails to successfully receive the data stream of the terminal of the conference speaker, the first device can lock the terminal that needs to adjust the flow by automatically comparing the expected recipient and the current successful recipient, and the first device is triggered to automatically adjust the flow for the terminal. This process does not require user participation, not only takes a short time, but also is imperceptible to the user.
[0210] In summary, the specific implementation of the method for retrieving data streams according to the embodiment of the present invention can be described as follows:
[0211] 1. Application Architecture
[0212] The current visual network has eliminated the restrictions that autonomous domains can only be cascaded up to 4 layers and the maximum number of accessible devices in each autonomous domain is a fixed value. In theory, it can support cascading of autonomous domains with more than 8 layers. Among them, autonomous domains support multiple topology types, including star, ring, cone, multi-cone, access, etc. For example, the aggregation layer autonomous domain supports star, ring, cone, multi-cone and other topologies, the core layer autonomous domain supports ring, cone and other topologies, and the access layer autonomous domain supports access topology.
[0213] The following takes a ring-type visual network topology structure composed of three nodes as an example to introduce the specific implementation method of the data stream calling method of an embodiment of the present invention.
[0214] like Figure 6 As shown, the architecture of the visual network includes: a management client, a management server, a ring network composed of nodes A, B, and C, and multiple terminals (i.e., visual network terminals).
[0215] Among them, an autonomous domain of a visual network is regarded as a node; an autonomous domain is a network with unified management and unified strategy, and is the smallest subnetting unit of a visual network. A visual network is often composed of one or more autonomous domains. Autonomous domains support tree-shaped cascading. Currently, autonomous domains are divided into ring network autonomous domains and access autonomous domains. The ring network autonomous domain supports multiple topological structures such as chain and tree to realize mesh networking.
[0216] The fusion control unit of each node is composed of two boards, one board is the access control unit, and the other board is the ring network control unit. The access control unit is responsible for the communication between the server and the ring network control unit, which is mainly responsible for ring network communication and cross-domain services.
[0217] That is, it can be understood that the fusion control unit: also known as the core server, is the data forwarding device of the visual network; it can be divided into three types: one is the ring network control unit, which is used to form a ring network, referred to as the ring network side; the second is the access control unit; the third is an N-in-one device in which several ring network control units and access control units are integrated in one frame;
[0218] In addition, the conference management client (i.e., conference management client) and the conference management server (i.e., conference management service) constitute the conference management system, referred to as conference management.
[0219] Meeting Management Client: It is a video conferencing scheduling software with comprehensive functions, efficient execution and stable operation. It has powerful video conferencing management functions and supports scheduling of video conferencing terminals and monitoring. It has super flexibility and can meet the video scheduling needs of various video conferences and emergency command.
[0220] Conference management server: It consists of network service, conference management service, audit management service, unified interface service and conference scheduling service.
[0221] Conference Scheduling Service: It can be referred to as Scheduling Service for short. It completely removes the Internet Protocol Address (IP) and interacts with the access control unit's visual network autonomous service. It is mainly responsible for the conference management client as the service initiator. The conference scheduling service performs service scheduling and interacts with the visual network autonomous service. The service process is divided into two parts: Content Management Server (CMS) and Distributor Management Server (DMS). CMS is responsible for the interface interaction with the conference management client and the database interaction. DMS is responsible for the underlying conference and control logic. DMS consists of three libraries: Cascade Library, Conference Service Scheduling Library, and VVSDK Library.
[0222] Cascade library: responsible for cross-autonomous conference business-related communications and cross-domain business processing.
[0223] Conference business scheduling library: responsible for the scheduling of core conference business.
[0224] VVSDK library: encapsulates the protocol library for communicating with autonomous services and is responsible for communicating with the fusion control unit. All functions related to autonomous communication or terminal communication need to call this library, such as starting a meeting, stopping a meeting, removing a participant, adding a participant, switching data streams, etc.
[0225] The management client can communicate with the Transmission Control Protocol (TCP) router (ie, Tcprouter) of the fusion control unit by using a virtual machine to virtual machine migration (Virtual to Virtual, V2V) method or an IP method.
[0226] In addition, the terminal can access a node by accessing the fusion control unit.
[0227] 2. Execution process, such as Figure 7 As shown:
[0228] (1) Membership process:
[0229] When the conference management client opens a conference with three node terminals, for example, terminal C1 in node C is the chairperson, it sends a reservation meeting\approval meeting request to the conference management server, which saves the request to the reservation and approval meeting, and returns a reservation meeting\approval meeting response to the conference management client;
[0230] Afterwards, the conference management client sends a notification of starting a conference to the conference management server, and the conference management server sends a notification of starting a conference to the fusion control unit of the A node to which it is connected, so that the fusion control unit of the A node sends a notification of starting a conference to the participating terminals respectively; it should be noted here that the interaction between the fusion control unit of the A node and the participating terminals is realized by the fusion control unit of the A node and the fusion control unit of the node to which the participating terminals are connected;
[0231] After receiving the notification of starting a conference, the A node fusion control unit returns a start conference response to the conference management server;
[0232] The participating terminal returns a joining success response to the A node fusion control unit, so that the A node fusion control unit reports the member status (i.e., whether the member has successfully joined the meeting) to the meeting management server, and then the meeting management server reports the member status to the meeting management client;
[0233] (2) Switching process:
[0234] The conference management client switches the terminal A1 in the A node to be the speaker (that is, the user of the terminal A1 is the speaker), and sends a request to the conference management server to switch the terminal A1 to be the speaker, so that the conference management server starts a flow adjustment thread and performs business logic flow adjustment on the terminal, so that the C node terminals C1 and C2 can watch the video and sound of the terminal A1, and the terminal C2 can hear the sound of the C1 terminal. The specific flow adjustment process here is:
[0235] The meeting management server saves the information of the party currently watching and the party expected to watch, and sends a data stream switching request to the A node fusion control unit (the request indicates to listen to the voice of the chairman, listen to the voice of the sender and watch the video of the sender). After receiving the data stream switching request, the A node fusion control unit requests the audio and video stream and stream address from the terminal A1, and the A node fusion control unit decides which link to use. For example, if the link A→C is used, the audio and video stream and stream address of the terminal A1 are sent to the C node fusion control unit, and the C node fusion control unit sends the audio and video stream and stream address of the terminal A1 to the terminals C1 and C2; and the A node fusion control unit also needs to request the audio stream and stream address of the terminal C1, and then send it to the terminal C2; wherein, after the terminals C1 and C2 receive the audio and video streams of the terminal A1, the stream addresses of the decoding channel for the audio and the decoding channel for the video of the receiving terminal A1 are the address of the terminal A1, and after the terminal C2 receives the audio stream of the terminal C1, the stream address of the decoding channel for the audio of the receiving terminal C1 is the address of the terminal C1;
[0236] Among them, the terminal that successfully receives the data stream can return a successful viewing of the data stream to the A node fusion control unit, so that the A node fusion control unit returns a data stream switching response to the conference management server, and then the conference management server returns a response to the conference management client when the A node terminal A1 is switched.
[0237] At this point, the process of switching terminal A1 is completed.
[0238] (3) Traffic adjustment process when the link is disconnected:
[0239] Each node in the ring network can sense whether the link between it and other nodes is available. For example, node C can sense whether the link between it and node A is available. If it is not available (for example, the network cable is disconnected or the link traffic reaches the maximum value), node C cannot receive the data stream of terminal A1, and thus the decoding channel for terminals C1 and C2 to receive terminal A1 will be cleared.
[0240] Among them, when the service is in progress, the link through which the data flow from node A to node C passes is A→C. At this time, the link from A→C is disconnected, and the leader (leader fusion control unit) will reselect the link and automatically switch the data flow link from A→C to A→B→C, thereby notifying nodes A and C of the reselected link;
[0241] It can be understood that the leader fusion control unit reselects the link, and the terminal C1 and the end point C2 in the C node will not withdraw from the meeting. The link that the C node fusion control unit is using is suddenly disconnected, and the C node cannot receive the data stream of the terminal A1. The fusion control unit of the C node will report to the A node fusion control unit through the new link (i.e. A→B→C): the decoding channel of the C node terminal C1 and the terminal C2 receiving the terminal A1 is cleared, and then the A node fusion control unit reports the situation to the meeting management server.
[0242] When the conference management server receives the above content reported by the fusion control unit of node C, the conference management server updates the viewing party (that is, terminals C1 and C2 are removed from the viewing party), thereby comparing the expected viewing and the current viewing. The flow adjustment thread specifically handles the inconsistency between the expected viewing and the current viewing. The thread adjusts the flow three times in total, with an interval of 3 seconds each. If the call fails after three times, it will not continue to adjust the flow, thereby reducing unnecessary interaction instructions with the fusion control unit and not affecting the performance of each service.
[0243] It should be noted that after the conference management server compares the expected viewing party and the viewing party and obtains the terminal that needs to be adjusted, the conference management server can directly send a flow adjustment instruction to the A node fusion control unit, and the A node fusion control unit performs the flow adjustment;
[0244] Alternatively, after the conference management server compares the expected viewing party and the viewing party and obtains the terminal that needs to be adjusted, it can notify the conference management client of the terminal that needs to be adjusted, so that the conference management client sends a flow adjustment instruction to the conference management server, and then the conference management server sends the flow adjustment instruction to the A node fusion control unit, and the A node fusion control unit performs the flow adjustment;
[0245] Alternatively, the conference management server may report to the conference management client that the decoding channels of the C-node terminals C1 and C2 receiving terminal A1 have been cleared, and the conference management client also stores the party currently viewing and the party expected to view. The conference management client compares the party expected to view and the party currently viewing to obtain the terminal that needs to be adjusted, and then the conference management client sends a flow adjustment instruction to the conference management server, and the conference management server sends the flow adjustment instruction to the A-node fusion control unit, which performs the flow adjustment.
[0246] The following describes the specific flow adjustment process after the A node fusion control unit receives the flow adjustment instruction:
[0247] The A node fusion control unit requests the audio and video stream and stream address from the terminal A1, and then sends the obtained audio and video stream and stream address of the terminal A1 to the fusion control unit of the C node through the path of the A node fusion control unit → the B node fusion control unit → the C node fusion control unit, and then the C node fusion control unit sends the audio and video stream and stream address of the terminal A1 to the terminals C1 and C2.
[0248] Afterwards, the terminal that successfully receives the data stream can also return a successful viewing of the data stream to the A node fusion control unit, so that the A node fusion control unit reports the member decoding status response to the conference management server, informing the conference management server that the stream retrieval is successful.
[0249] It should also be noted that the leader fusion control unit may be the fusion control unit of the node to which the server is connected by default, or may be selected from the fusion control units in each node in the ring network through a certain strategy.
[0250] (4) After switching a certain terminal to be the first sender (i.e., the terminal serving as the speaker), if there are other terminals participating in the conference that fail to successfully receive the data stream of the terminal, the fusion control unit to which these terminals are connected will also clear the decoding channels for these terminals to receive the data stream of the terminal serving as the first sender, thereby reporting the situation to the conference management client, thereby triggering the flow adjustment process for these terminals; it should be noted that the flow adjustment process in this case is the same as the flow adjustment process when the link is disconnected, and will not be repeated here.
[0251] To sum up, in the embodiments of the present invention, after the link is reselected, the service can be quickly restored without the user noticing, and the conference management server can support re-adjusting the flow, so that the audio and video flow can be restored without the terminal exiting the network. This is not only applicable to the ring network scenario, but also to other link scenarios. When the terminal fails to watch the flow without exiting the network, the audio and video flow can be restored.
[0252] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0253] An embodiment of the present invention discloses a data stream retrieving system, the data stream retrieving system comprising a first device and a third device, the third device being a server in a network domain accessed by the first device;
[0254] The first device is used to: determine whether a flow adjustment triggering event occurs, and in the case of the flow adjustment triggering event, obtain first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0255] Determine, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0256] In the case where the second terminal exists, sending a first instruction to the third device, wherein the first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal;
[0257] The third device is used to: receive the first instruction, obtain the data stream of the first terminal based on the first instruction, and send the data stream of the first terminal to the second terminal.
[0258] In an optional embodiment of the present invention, the system further includes a fourth device, configured to:
[0259] When the third terminal fails to successfully receive the data stream of the first terminal, sending third information to the first device, so that the first device determines whether a flow adjustment triggering event occurs according to the third information;
[0260] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared.
[0261] An embodiment of the present invention discloses a data stream retrieving system, the data stream retrieving system comprising a first device, a second device and a third device, the third device being a server in a network domain accessed by the first device;
[0262] The first device is used to: determine whether a flow adjustment triggering event occurs, and in the case of the flow adjustment triggering event, obtain first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0263] Determine, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0264] In a case where the second terminal exists, sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0265] The second device is used to: receive the fourth information; send a first instruction to the first device, wherein the first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal;
[0266] The first device is further configured to: receive the first instruction sent by the second device, and send the first instruction to the third device;
[0267] The third device is used to: receive the first instruction, obtain the data stream of the first terminal based on the first instruction, and send the data stream of the first terminal to the second terminal.
[0268] In an optional embodiment of the present invention, the system further includes a fourth device, configured to:
[0269] When the third terminal fails to successfully receive the data stream of the first terminal, sending third information to the first device, so that the first device determines whether a flow adjustment triggering event occurs according to the third information;
[0270] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared.
[0271] It should be noted that each device in the data stream calling system of the embodiment of the present invention can execute the data stream calling method applied to the first device described above, and the corresponding steps in the data stream calling method applied to the first device. For details, please refer to the above description and will not be repeated here.
[0272] Reference Figure 8 The embodiment of the present invention provides a data stream calling device, which is applied to a first device, and the device includes the following modules:
[0273] The judgment module 801 is used to execute step 1: determine whether a flow adjustment triggering event occurs, and in the case of the flow adjustment triggering event, trigger the acquisition module 802, the determination module 803, and the data flow retrieval module 804 to respectively execute the following steps 2 to 4:
[0274] The acquisition module 802 is used to execute step 2: acquire first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0275] The determination module 803 is used to execute step three: determine, according to the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received it, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal;
[0276] The data stream retrieving module 804 is used to execute step 4: when the determining module determines that the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
[0277] In an optional embodiment of the present invention, the flow regulation triggering event includes at least one of the following:
[0278] A first trigger event is used to indicate that a first link is disconnected, where the first link is a link between a network domain accessed by the first device and another network domain, where the other network domain is a network domain accessed by a terminal that expects to receive a data stream of the first terminal;
[0279] A second trigger event, used to indicate that the traffic of the first link reaches a first threshold;
[0280] A third trigger event is used to instruct setting the first terminal to send a data stream;
[0281] The fourth trigger event is used to indicate that after the first terminal is set to send the data stream, there is a terminal that fails to successfully receive the data stream of the first terminal.
[0282] In an optional embodiment of the present invention, the determining module 801 is specifically configured to:
[0283] In the case of receiving third information, determining that at least one of the first trigger event, the second trigger event, and the fourth trigger event occurs, the third information being used to indicate that a decoding channel for at least one third terminal to receive the data stream of the first terminal is cleared;
[0284] In a case where the third information is not received, it is determined that the first trigger event, the second trigger event, and the fourth trigger event have not occurred.
[0285] In an optional embodiment of the present invention, the data stream retrieving module 804 is specifically used for:
[0286] When the second terminal exists and the first trigger event or the second trigger event occurs, the link between the network domain accessed by the first device and the network domain accessed by the second terminal is reselected, and the data stream of the first terminal is called for the second terminal through the reselected link.
[0287] In an optional embodiment of the present invention, the device further comprises:
[0288] The control module is used to control the data stream retrieval module 804 to execute step 4 after completion, control the acquisition module 802 to execute step 2 at a predetermined interval, and sequentially control the determination module 803 to execute step 3 and the data stream retrieval module 804 to execute step 4, until the execution times of steps 2 to 4 reach the target threshold, and then stop executing steps 2 to 4.
[0289] In an optional embodiment of the present invention, when the data stream retrieving module 804 retrieves the data stream of the first terminal for the second terminal, it is specifically used to:
[0290] Sending a first instruction to a third device;
[0291] The third device is a server in the network domain accessed by the first device;
[0292] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0293] In an optional embodiment of the present invention, when the data stream retrieving module 804 retrieves the data stream of the first terminal for the second terminal, it is specifically used to:
[0294] Sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal;
[0295] receiving a first instruction sent by the second device;
[0296] Sending the first instruction to a third device;
[0297] The third device is a server in the network domain accessed by the first device;
[0298] The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
[0299] Reference Fig. 9 , an embodiment of the present invention provides a data stream calling device, which is applied to a second device, and the device includes the following modules:
[0300] The receiving module 901 is configured to receive fourth information sent by the first device, wherein the fourth information is used to indicate that the second terminal currently expects to receive a data stream of the first terminal but fails to do so;
[0301] The first sending module 902 is used to send a first instruction to the first device, wherein the first instruction is used to instruct a third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal, and the third device is a server in the network domain accessed by the first device.
[0302] Reference Fig.10 , an embodiment of the present invention provides a data stream retrieving device, which is applied to a fourth device, and the device includes the following modules:
[0303] The second sending module 1001 is used to send third information to the first device when the third terminal fails to successfully receive the data stream of the first terminal, so that the first device obtains the first information according to the third information, and determines the second terminal that currently expects to receive the data stream of the first terminal but fails to receive it successfully according to the first information and the stored second information, and retrieves the data stream of the first terminal for the second terminal when the second terminal exists;
[0304] The third information is used to indicate that a decoding channel for the third terminal to receive the data stream of the first terminal is cleared;
[0305] The first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal;
[0306] The second information includes identification information of a terminal that desires to receive the data stream of the first terminal.
[0307] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0308] In addition, an embodiment of the present invention further discloses an electronic device, including:
[0309] one or more processors; and
[0310] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the above-mentioned method for retrieving the data stream.
[0311] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the above-mentioned method for retrieving data streams.
[0312] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0313] It should be understood by those skilled in the art that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, read-only memory (Compact Disc Read-Only Memory) CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0314] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0315] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0316] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0317] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0318] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0319] The above is a detailed introduction to a data stream retrieval method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for retrieving a data stream, characterized in that: Applied to a first device, the method includes: Step 1: Determine whether a flow regulation triggering event occurs, and if the flow regulation triggering event occurs, execute the following steps 2 to 4: Step 2: Acquire first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal; Step 3: determining, based on the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal; Step 4: If the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
2. The method according to claim 1, characterized in that The flow regulation triggering event includes at least one of the following: A first trigger event is used to indicate that a first link is disconnected, where the first link is a link between a network domain accessed by the first device and another network domain, where the other network domain is a network domain accessed by a terminal that expects to receive a data stream of the first terminal; A second trigger event, used to indicate that the traffic of the first link reaches a first threshold; A third trigger event is used to instruct setting the first terminal to send a data stream; The fourth trigger event is used to indicate that after the first terminal is set to send the data stream, there is a terminal that fails to successfully receive the data stream of the first terminal.
3. The method according to claim 2, characterized in that The determining whether the first trigger event, the second trigger event, and the fourth trigger event occur includes: In the case of receiving third information, determining that at least one of the first trigger event, the second trigger event, and the fourth trigger event occurs, the third information being used to indicate that a decoding channel for at least one third terminal to receive the data stream of the first terminal is cleared; In a case where the third information is not received, it is determined that the first trigger event, the second trigger event, and the fourth trigger event have not occurred.
4. The method according to claim 2 or 3, characterized in that: The step of, in the case where the second terminal exists, calling the data stream of the first terminal for the second terminal includes: When the second terminal exists and the first trigger event or the second trigger event occurs, the link between the network domain accessed by the first device and the network domain accessed by the second terminal is reselected, and the data stream of the first terminal is called for the second terminal through the reselected link.
5. The method for retrieving data stream according to claim 1, characterized in that: The method further comprises: After the execution of step 4 is completed, the method returns to step 2 at a predetermined interval, and executes step 3 and step 4 in sequence until the execution times of step 2 to step 4 reach a second threshold, and then stops executing step 2 to step 4.
6. The method for retrieving data stream according to claim 1, characterized in that: The step of calling the data stream of the first terminal for the second terminal includes: Sending a first instruction to a third device; The third device is a server in the network domain accessed by the first device; The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
7. The method for retrieving data stream according to claim 1, characterized in that: The step of calling the data stream of the first terminal for the second terminal includes: Sending fourth information to the second device, wherein the fourth information is used to indicate that the second terminal has not successfully received the data stream of the first terminal; receiving a first instruction sent by the second device; Sending the first instruction to a third device; The third device is a server in the network domain accessed by the first device; The first instruction is used to instruct the third device to obtain the data stream of the first terminal and send the data stream of the first terminal to the second terminal.
8. A data stream scheduling device, characterized in that: Applied to a first device, the apparatus comprises: The judgment module is used to execute step 1: judge whether a flow adjustment triggering event occurs, and in the case of the flow adjustment triggering event, trigger the acquisition module, the determination module, and the data flow calling module to respectively execute the following steps 2 to 4: The acquisition module is used to execute step 2: acquire first information, wherein the first information includes identification information of a terminal that currently successfully receives a data stream of the first terminal; The determination module is configured to execute step three: determining, based on the first information and the stored second information, a second terminal that currently expects to receive the data stream of the first terminal but has not successfully received the data stream, wherein the second information includes identification information of the terminal that expects to receive the data stream of the first terminal; The data stream retrieving module is used to execute step 4: when the determining module determines that the second terminal exists, retrieve the data stream of the first terminal for the second terminal.
9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to execute the method for retrieving a data stream as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the method for retrieving a data stream as described in any one of claims 1 to 7.
