Equipment and system for realizing audio-visual service scheduling of heterogeneous network

Abstract

The embodiment of the invention provides equipment and a system for realizing audio-visual service scheduling of a heterogeneous network, and back-end equipment in the system at least comprises a link management module, a link quality monitoring module and a link scheduling module, the front-end equipment at least comprises a link switching control module, a link quality reporting module, a link detection module and a video playing module, and through interaction between the front-end equipment and the rear-end equipment, the scheme of the embodiment of the invention can realize dynamic scheduling in a heterogeneous network environment, and can realize dynamic scheduling under the condition that the transmission quality is not ideal. Therefore, the service quality and the user experience of the audio-visual service are greatly improved.

Description

Technical Field

[0001] This application relates to the field of information technology, and particularly to a device and system for implementing heterogeneous network audio-visual service scheduling. Background Art

[0002] With the rapid development of information technology and the continuous popularization of domestic network transmission technology, people have put forward new requirements for traditional audio-visual services. From high-definition to ultra-high-definition, from live broadcast to on-demand and then to real-time audio-video communication, the demand for audio-visual services by users has become more diverse, and at the same time, higher requirements have been put forward for network performance.

[0003] However, the current real network environment is complex and changeable. With the continuous development of technology, at the front end of the system, the networks that terminal devices can access may include various different network forms such as wired networks, wireless WIFI (Wireless Fidelity) networks, 5G (5th Generation Mobile Communication Technology) networks, etc. These different networks have their own unique transmission characteristics and limitations during transmission. At the same time, at the back end of the system, according to the actual needs of different audio-visual services, multiple transmission links meeting different standards can also be provided. These transmission links cover different bitrate selections, clarity levels, and transmission protocols to adapt to various complex application scenarios and performance requirements.

[0004] Currently, there is no relatively mature solution in the industry that can achieve dynamic scheduling in a heterogeneous network environment and can perform seamless switching in the case of unsatisfactory transmission quality. Summary of the Invention

[0005] An object of this application is to provide a device and system for implementing heterogeneous network audio-visual service scheduling.

[0006] To achieve the above object, an embodiment of this application provides a back-end device for implementing heterogeneous network audio-visual service scheduling, and the back-end device includes:

[0007] A link management module, configured to create and manage transmission links for different audio-visual services;

[0008] A link quality monitoring module, configured to obtain transmission quality data reported by a front-end device, and determine status information of the transmission link and network currently used by the front-end device according to the transmission quality data;

[0009] A link scheduling module, which is used to allocate a transmission link for a terminal device requesting to execute a network audio-visual service, and send a switching instruction to a front-end device according to the status information, so that the front-end device can obtain the media stream of the audio-visual service based on the transmission link and the accessed network, and switch the currently used transmission link and / or network according to the switching instruction.

[0010] Further, the transmission links created and managed by the link management module conform to different standards and are used to transmit media streams with different bitrates, resolutions, and transmission protocols.

[0011] Further, the link scheduling module is also used to extract the characteristic information of the transmission quality data according to the historical data of the collected transmission quality data, and combine the current time and the link load condition to predict the congestion trend of the transmission link, and generate a switching instruction according to the prediction result and the status information.

[0012] Further, the link scheduling module is also used to allocate a transmission link for a newly accessed front-end device according to the prediction result and the status information.

[0013] The embodiment of the present application also provides a front-end device for implementing heterogeneous network audio-visual service scheduling. The front-end device includes:

[0014] A link switching control module, which is used to obtain the media stream of the audio-visual service according to the transmission link allocated by the back-end device and the accessed network when requesting to execute a network audio-visual service, and switch the currently used transmission link and / or network according to the switching instruction, where the switching instruction is determined by the back-end device according to the status information of the transmission link and network currently used by the front-end device;

[0015] A link detection module, which is used to detect the currently available transmission links and networks of the front-end device and detect the transmission quality data of the currently available transmission links and networks;

[0016] A link quality reporting module, which is used to report the transmission quality data to the back-end device, so that the back-end device can determine the status information of the transmission link and network currently used by the front-end device according to the transmission quality data;

[0017] A video playback module, which is used to play audio and video content according to the obtained media stream of the audio-visual service.

[0018] Further, the front-end device further includes:

[0019] An adaptive conversion module, which is used to convert the media streams with different bitrates, resolutions, and transmission protocols transmitted by different standard transmission links into a unified standard adaptive bitstream for providing to the video playback module for playback.

[0020] Further, the adaptive conversion module is further configured to, after the link switching control module switches the currently used transmission link and / or network according to the switching instruction, complete seamless switching at the key frame position of the new media stream according to the timestamp information inserted in the media stream.

[0021] Further, the link switching control module is further configured to implement fault diagnosis, and monitor the quality of the media stream in real time, and discover and report abnormal or interrupted situations.

[0022] Further, the link detection module is configured to continuously detect the transmission link and the network to identify the currently available transmission link and network of the front-end device therein, and detect the transmission quality data of the currently available transmission link and network;

[0023] The link quality reporting module is configured to send the transmission quality data to the back-end device through a message in a preset format, so that the back-end device determines the status information of the transmission link and network currently used by the front-end device according to the transmission quality data.

[0024] The solution of the embodiment of the present application further provides a system for implementing heterogeneous network audio-visual service scheduling, including the back-end device and the front-end device described above.

[0025] Compared with the prior art, in an apparatus and a system for implementing heterogeneous network audio-visual service scheduling provided by the embodiment of the present application, the back-end device at least includes a link management module, a link quality monitoring module, and a link scheduling module, and is capable of creating and managing transmission links for different audio-visual services; obtaining the transmission quality data reported by the front-end device, and determining the status information of the transmission link and network currently used by the front-end device according to the transmission quality data; and allocating a transmission link for a terminal device requesting to execute a network audio-visual service, and sending a switching instruction to the front-end device according to the status information. The front-end device at least includes a link switching control module, a link quality reporting module, a link detection module, and a video playback module, and is capable of obtaining a media stream of an audio-visual service according to the transmission link allocated by the back-end device and the accessed network when requesting to execute a network audio-visual service, and switching the currently used transmission link and / or network according to the switching instruction; detecting the currently available transmission link and network of the front-end device, and detecting the transmission quality data of the currently available transmission link and network; reporting the transmission quality data to the back-end device; and playing audio-visual content according to the obtained media stream of the audio-visual service. Therefore, the solution of the present application can achieve dynamic scheduling in a heterogeneous network environment, and can perform seamless switching in the case of unsatisfactory transmission quality, greatly improving the service quality and user experience of audio-visual services. Description of the Drawings

[0026] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings:

[0027] Figure 1 Schematic diagram of the structure of a backend device for implementing heterogeneous network audio-visual service scheduling provided by an embodiment of the present application;

[0028] Figure 2 Schematic diagram of the structure of a frontend device for implementing heterogeneous network audio-visual service scheduling provided by an embodiment of the present application;

[0029] Figure 3 Schematic diagram of the structure of a system for implementing heterogeneous network audio-visual service scheduling provided by an embodiment of the present application.

[0030] The same or similar reference numerals in the drawings represent the same or similar components. Detailed implementation

[0031] The present application will be further described in detail below with reference to the accompanying drawings.

[0032] In a typical configuration of the present application, devices of the terminal and the service network each include one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0033] The memory may include non-permanent memory in the computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0034] Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer program instructions, data structures, program devices, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette tapes, magnetic disk storage, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0035] In the embodiments of the present application, a device and a system for implementing heterogeneous network audio-visual service scheduling are provided. The backend device at least includes a link management module, a link quality monitoring module, and a link scheduling module, which can create and manage transmission links for different audio-visual services; obtain the transmission quality data reported by the frontend device, and determine the status information of the transmission link and network currently used by the frontend device according to the transmission quality data; and allocate a transmission link for a terminal device requesting to execute a network audio-visual service, and send a switching instruction to the frontend device according to the status information. The frontend device at least includes a link switching control module, a link quality reporting module, a link detection module, and a video playback module, which can obtain the media stream of the audio-visual service according to the transmission link allocated by the backend device and the accessed network when requesting to execute a network audio-visual service, and switch the currently used transmission link and / or network according to the switching instruction; detect the currently available transmission links and networks of the frontend device, and detect the transmission quality data of the currently available transmission links and networks; report the transmission quality data to the backend device; and play audio and video content according to the obtained media stream of the audio-visual service. Thus, the solution of the present application can achieve dynamic scheduling in a heterogeneous network environment, and can perform seamless switching in the case of unsatisfactory transmission quality, greatly improving the service quality and user experience of audio-visual services.

[0036] In an actual scenario, the frontend device is various user devices on the user side, and the user devices include but are not limited to various terminal devices such as computers, mobile phones, and tablets, or can also be application programs executed on the above user devices. The backend device is various network devices on the service side, and the network devices include but are not limited to being implemented by a network host, a single network server, a set of multiple network servers, or a computer set based on cloud computing, or can also be application programs executed on the above network devices. Here, the cloud consists of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing and consists of a virtual computer composed of a group of loosely coupled computers.

[0037] The embodiments of the present application provide a system for implementing heterogeneous network audio-visual service scheduling. The system includes a backend device and a frontend device. The heterogeneous network in the solution of the present application refers to an architecture including multiple different types of networks. For example, the solution of this embodiment can include a wired network, a wireless WIFI network, a 5G network, etc. These networks have different characteristics, so their applicability to different audio-visual services is also different. In the radio and television industry, the current mainstream audio-visual services can be mainly classified into the following three types: live broadcast services, on-demand services, and audio and video communication services. These different types of audio-visual services have different requirements for the network. Therefore, in a heterogeneous network environment, in order to meet the requirements of different audio-visual services, appropriate scheduling methods are needed.

[0038] In the solution of the embodiment of this application, the structure of the backend device is as Figure 1 shown, and at least includes a link management module 110, a link quality monitoring module 120, and a link scheduling module 130.

[0039] The link management module 110 is used to create and manage transmission links for different audio-visual services, so as to ensure that in the context of various different audio-visual service requirements such as live broadcast, on-demand, and instant audio-visual communication, and in a diverse heterogeneous network environment, multiple transmission links meeting different standards are constructed. These transmission links cover different bitrate selections, clarity levels, and transmission protocols, and can be used to transmit media streams with different bitrates, clarity, and transmission protocols to adapt to various complex application scenarios and performance requirements.

[0040] The link quality monitoring module 120 is used to obtain the transmission quality data reported by the front-end device, and determine the status information of the transmission link and network currently used by the front-end device according to the transmission quality data. In the solution of the embodiment of this application, link quality monitoring is an important network management means, which connects various different heterogeneous networks and collects and analyzes the transmission quality data reported by the front-end device in real time. These transmission quality data cover multiple key performance indicators, including but not limited to bandwidth, delay, jitter, packet loss rate, and network load, etc. By continuously monitoring and analyzing these performance indicators, the link quality monitoring module can accurately evaluate the current status of the transmission link and network, and thus provide an accurate basis for the decision-making of the link scheduling module.

[0041] The link scheduling module 130 is used to allocate a transmission link for a terminal device requesting to execute a network audio-visual service, and send a switching instruction to the front-end device according to the status information, so that the front-end device can obtain the media stream of the audio-visual service based on the transmission link and the accessed network, and switch the currently used transmission link and / or network according to the switching instruction.

[0042] In the solution of the embodiment of this application, the link scheduling module not only functions to allocate a starting transmission link for newly added front-end devices, so that they can successfully access the network and start transmitting media stream data, but also undertakes the effective switching work of the transmission links and networks used by the front-end devices according to the preset scheduling strategy.

[0043] This scheduling process supports automation and seamless operation, meaning that adjustments and optimizations to the transmission link and network can be carried out automatically without manual intervention, thus bringing a smoother and more stable user experience to users. In addition, the link scheduling module also has adaptive adjustment capabilities. It can extract the characteristic information of the transmission quality data based on the historical data of the collected transmission quality data, and combine the current time and the link load situation to predict the congestion trend of the transmission link. According to the prediction results and the status information, a switching instruction is generated to update the scheduling decision. When new front-end devices are subsequently connected and transmission links need to be allocated, the transmission links for the newly connected front-end devices can also be allocated based on the previous prediction results and the status information. In addition, the link scheduling module can continuously optimize the scheduling algorithm to adapt to the ever-changing network environment and business requirements. This intelligent scheduling mechanism not only improves the utilization rate of network resources but also greatly enhances the user experience.

[0044] In the solution of the embodiment of this application, the structure of the front-end device is as Figure 2 shown, and at least includes a link switching control module 210, a link detection module 220, a link quality reporting module 230, and a video playback module 240.

[0045] The link switching control module 210 obtains the media stream of the audio-visual service according to the transmission link allocated by the back-end device and the accessed network when requesting to execute the network audio-visual service, and switches the currently used transmission link and / or network according to the switching instruction. Among them, the switching instruction is determined by the back-end device according to the status information of the transmission link and network currently used by the front-end device.

[0046] In the solution of the embodiment of this application, the main responsibilities of the link switching control module include being responsible for obtaining the media stream through the used transmission link and network, ensuring a smooth connection with the back-end device, and following the switching instruction issued by the back-end device to perform the switching of the transmission link and network.

[0047] The link switching control module can also be used to implement fault diagnosis and monitor the quality of the media stream in real time, discover and report abnormal or interrupted situations. When performing tasks, the link switching control module can monitor the quality of the media stream in real time, promptly discover and report any possible abnormal or interrupted situations. In addition, the link switching control module also has certain fault diagnosis capabilities so that it can quickly locate and take corresponding solutions when encountering problems, ensuring the stability and reliability of the media stream transmission.

[0048] The link detection module 220 is used to detect the currently available transmission links and networks of the front-end device, and detect the transmission quality data of the currently available transmission links and networks. When there are multiple potential networks and transmission links available for the front-end device to choose from, the link detection module can continuously detect the transmission links and networks to identify the currently available transmission links and networks of the front-end device, detect the transmission quality data of the currently available transmission links and networks, and thus closely monitor the performance indicators of these available transmission links and networks. All the collected transmission quality data will be sent to the link quality detection module of the back-end device through a message in a preset format by the link quality reporting module. In this way, once a failure occurs in the line where the currently used transmission link and network are located or their performance degrades, the scheduling system can quickly and smoothly switch according to the established switching strategy, ensuring the stability and reliability of the audio-visual service data transmission, and thus achieving a line switching experience that is almost imperceptible to users.

[0049] The link quality reporting module 230 is used to report the transmission quality data to the back-end device, so that the back-end device can determine the status information of the transmission link and network currently used by the front-end device according to the transmission quality data. In the solution of the embodiment of the present application, the main responsibility of the link quality reporting module is to collect and monitor the transmission quality data of the media stream on the front-end device. The transmission quality data includes some key performance indicators, covering multiple aspects such as the highest downstream rate, average rate, network jitter, round-trip delay, etc. In addition, it also includes packing the collected transmission quality data into a message in a preset format and sending it to the link quality detection module of the back-end device through the network for further analysis and processing.

[0050] The video playback module 240 is used to play audio-visual content according to the obtained media stream of the audio-visual service. In an actual scenario, in-depth customized development work can be carried out for the video playback module, so that this module can not only be compatible with and support different coding formats and encapsulation formats of multiple streaming media services, but also provide strong support for instant audio-visual communication services. During the process of the user switching the transmission link and network, the scheduling system can ensure the coherence and smoothness of the user experience and achieve a seamless and imperceptible switching effect.

[0051] In some other embodiments of the present application, the front-end device may further include an adaptive conversion module. The adaptive conversion module converts media streams with different bitrates, resolutions, and transmission protocols transmitted from different standard transmission links into a unified standard adaptive bitstream for the video playback module to play. In addition, after the link switching control module switches the currently used transmission link and / or network according to the switching instruction, seamless switching can be completed at the key frame position of the new media stream according to the timestamp information inserted in the media stream.

[0052] In this embodiment, the adaptive conversion module is responsible for receiving media streams from different transmission links. These media streams from different transmission links may adopt various different transmission protocols, coding standards, and characteristics of different delays. The adaptive conversion module can process and convert them into a standard adaptive bitstream that can be processed by a player through a buffer control mechanism and format conversion processing. This processing process ensures that when playing audio-visual content using the front-end device, a seamless and imperceptible switching experience can be achieved, thereby providing users with a smoother and higher-quality multimedia content experience.

[0053] Thus, the solution of the embodiment of the present application can achieve dynamic scheduling in a heterogeneous network environment, and can perform imperceptible switching of the transmission link and the network in the case of unsatisfactory transmission quality, greatly improving the service quality and user experience of the audio-visual service. For example, in an actual scenario, two video service sources can be set for the backend device, one is deployed in the public network environment and the other is deployed in the local area network environment to simulate two different transmission links. The front-end device is a mobile phone, which can connect to the backend device through a 5G network and a wireless WIFI network.

[0054] In this scenario, if only the video service source deployed in the public network environment is started first, when the mobile phone requests an audio-visual service, since there is only the video service source in the public network environment at this time, the transmission link 1 corresponding to this video service source will be defaulted as the starting transmission link. When the other video service source deployed in the local area network environment is started at this time, since the transmission quality in the local area network environment is better, the system will generate a corresponding switching instruction at this time, switching from transmission link 1 to transmission link 2 corresponding to the video service source in the local area network environment, so as to achieve imperceptible switching of the transmission link and improve the transmission efficiency.

[0055] After the mobile phone requests the audio-visual service through the wireless WIFI network, if another front-end device (such as a tablet computer) requests the audio-visual service through the wireless WIFI network at this time, it will cause the load of the wireless WIFI network to increase. In order to improve the transmission quality, the system will make a scheduling decision based on the transmission quality data at this time, generate a corresponding switching instruction, and notify the mobile phone to access other available networks (such as the 5G network), switching from the wireless WIFI network to the 5G network, so as to achieve a seamless switch of the access network and improve the transmission efficiency. When the tablet computer finishes requesting the audio-visual service, the system can notify the mobile phone to switch back to the original wireless WIFI network, thus realizing dynamic scheduling.

[0056] Here, those skilled in the art should understand that the above specific method for dynamic scheduling in the heterogeneous network environment of this time is only an example. Other forms based on similar principles that exist now or in the future, if applicable to this application, should also be included in the protection scope of this application and be included herein by reference. For example, the seamless switching of the transmission link and the network can be performed simultaneously to obtain a more efficient data transmission path.

[0057] In addition, in the solutions of some embodiments of this application, when determining the optimal network to be switched, the following method can be used to determine the target network to be switched this time: First, calculate the objective score of each network according to the index data of each network. Among them, the index data comes from the transmission quality data uploaded by the front-end device, such as bandwidth, delay, jitter, packet loss rate, and load, etc., so as to quantitatively represent the characteristics of different networks. Among the above index data, the bandwidth is a positive index, that is, the larger the value of the bandwidth index data, the better the performance of the network in this aspect. While the delay, jitter, packet loss rate, and load are negative indexes, that is, the larger the values of these index data such as delay, jitter, packet loss rate, and load, the worse the performance of the network in these aspects.

[0058] When calculating the objective score, first obtain the index data of each network, perform standardization processing on the values of the index data, then calculate the objective weight of each index data using the entropy weight method based on the standardized index data in each network, and finally calculate the objective score of each network based on the objective weight. Among them, the entropy weight method is an objective evaluation method that can be used for multiple objects and multiple indexes. When dealing with the problem of multi-index weighting, it can effectively eliminate the result deviation caused by subjective human assignment, avoid the influence of subjective factors, and improve the objectivity and accuracy of the evaluation results. The specific calculation steps are as follows:

[0059] Step S101, data standardization processing.

[0060] The entropy weight method requires data standardization to ensure fair and effective comparison between different indicators. Since different indicators may have different dimensions and value ranges, directly comparing them will lead to inaccurate results. Therefore, it is necessary to standardize the indicator data.

[0061] First, an indicator data matrix can be constructed based on the indicator data of each network. Taking the 3 networks and 5 indicator data in the above scenario as an example, a 3×5 indicator data matrix X can be constructed as follows:

[0062]

[0063] In this embodiment, m = 3, n = 5, and the element X ij in the indicator data matrix X represents the value of the jth indicator data of the ith network. For example, if the order of the 3 networks in this embodiment is wired network, wireless WIFI network, and 5G network in sequence, and the order of the 5 indicator data is bandwidth, delay, jitter, packet loss rate, and load in sequence, then the element X 23 in the indicator data matrix X represents the value of the jitter indicator of the wireless WIFI network.

[0064] When performing standardization processing, since bandwidth is a positive indicator, the following formula can be used for standardization processing:

[0065]

[0066] where Z ij is the standardized value of the element X ij in the indicator data matrix X after standardization processing, max(X 1j ,…, X mj ) represents taking the maximum value in the jth column of elements, and min(X 1j ,…, X mj ) represents the minimum value in the jth column of elements.

[0067] For negative indicators such as delay, jitter, packet loss rate, and load, the following formula can be used for standardization processing:

[0068]

[0069] For the 3×5 indicator data matrix X in the embodiment of the present application, the elements in the first column are subjected to positive standardization processing, and the elements in the second to fifth columns are subjected to positive standardization processing.

[0070] After the indicator data matrix X undergoes the above standardization processing, a standard data matrix Z can be obtained:

[0071]

[0072] Step S102, calculate the numerical proportion.

[0073] For the numerical proportion P corresponding to the element in the i-th row and j-th column of the matrix ij It can be calculated by the following formula:

[0074]

[0075] Step S103, calculate the entropy value.

[0076] Taking the elements in the j-th column of the matrix as an example, the entropy value e of the corresponding index data j It can be calculated by the following formula:

[0077]

[0078] Step S104, calculate the difference coefficient.

[0079] The difference coefficient is D j It can be calculated by subtracting the entropy value of the corresponding index data from 1. The specific calculation formula is as follows:

[0080] D j = 1 - e j

[0081] Step S105, calculate the objective weights of each index data.

[0082] Taking the weight w of the j-th index data as an example j Its calculation formula is as follows:

[0083]

[0084] Step S106, calculate the objective scores of the objective weight calculation network of each index data.

[0085] Taking the i-th network as an example, its objective score S i The calculation formula is as follows:

[0086]

[0087] Thus, the objective scores of each network are accurately calculated by the entropy weight method for subsequent processing.

[0088] Step S2, calculate the subjective scores of each network for the target audio-visual service according to the index data and the subjective weights of each network for the target audio-visual service set in advance.

[0089] In an actual scenario, considering the differences in the requirements of various audio-visual services for various network metrics during the actual deployment of audio-visual services, subjective weights can be assigned to the metric data of the network, and appropriate subjective weights can be set in advance to calculate the subjective scores of each network for the target audio-visual service, so as to compensate for the one-sidedness of the objective scores and make the comprehensive scores used for evaluation more comprehensive and accurate.

[0090] In some embodiments of the present application, in order to obtain more accurate subjective weights, the subjective weights of each network for the target audio-visual service can be obtained through the expert method, and then according to the metric data and the subjective weights after standardized processing in each network, the subjective scores of each network for the target audio-visual service are calculated.

[0091] Among them, when calculating the subjective weights, in addition to the metric data used in the calculation of the objective scores, the metric data of cost can be further added. For example, the metric data used for the calculation of the objective scores includes 5 items: bandwidth, delay, jitter, packet loss rate, and load, and the metric data used for the calculation of the objective scores includes 6 items: bandwidth, delay, jitter, packet loss rate, load, and cost. Thus, for this scenario, when calculating the subjective scores, both the metric data matrix X and the standard data matrix Z will be expanded into 3×6 matrices.

[0092] Taking the aforementioned three networks as an example, the cost of the 5G network is the highest, the cost of the wired network is the second, and the cost of the wireless WIFI network is the lowest. In this embodiment, the cost metric data of the above three networks can be set as the following values according to this principle: the cost of the 5G network is 10, the cost of the wired network is 5, and the cost of the wireless WIFI network is 1. Since the cost is a negative indicator, the standardized value of the cost can be calculated by the aforementioned method of standardizing negative indicators, and combined with the subjective weights, the subjective scores of each network for the target audio-visual service are calculated.

[0093] In this embodiment, a calculation formula for the subjective score is given, which is specifically as follows:

[0094]

[0095] Among them, represents the subjective score of the i-th network for the target audio-visual service N, and Z ij is the standardized value of the element X ij in the metric data matrix X after standardized processing, represents the subjective weight of the j-th metric data of the i-th network for the audio-visual service N.

[0096] Step S3: Calculate the comprehensive score of each network for the target audiovisual service based on the objective score of each network and the subjective score for the target audiovisual service.

[0097] When calculating the comprehensive score of each network for the target audiovisual service based on the objective score of each network and the subjective score for the target audiovisual service, the following formula can be specifically used for calculation:

[0098]

[0099] Among them, represents the comprehensive score of the i-th network for the target audiovisual service N, R i represents the subjective score of the i-th network for the target audiovisual service N, S i represents the objective score of the i-th network, and t is a constant greater than 0 and less than 1, such that t can be adjusted between 0 and 1 according to the needs of the actual scenario to facilitate optimizing the results of the algorithm. When t is set larger, it means that the subjective score has a greater impact on the comprehensive evaluation. On the contrary, if t is set smaller, it means that the objective score has a greater impact on the comprehensive evaluation.

[0100] Furthermore, after substituting the calculation formulas of and S i that have been publicly disclosed in the previous content, the following comprehensive score calculation formula can be obtained:

[0101]

[0102] Step S4: Determine the target network to which the front-end device is connected as the network with the highest comprehensive score. In order to obtain a better transmission effect and improve the user experience, the comprehensive scores of each available network of the front-end device for the target audiovisual service can be calculated in real time, and the network with the highest comprehensive score can be selected. The network with the highest comprehensive score is determined as the target network to which the front-end device is connected. Thus, it can provide a basis for generating a switching instruction for the back-end device, enabling the front-end device to execute the target audiovisual service using the network with the highest comprehensive score, and ensuring that the target network to which the front-end device is connected is a network suitable for executing the target audiovisual service.

[0103] Based on a similar principle, the optimal transmission link to be switched can also be determined based on the transmission quality data, thereby enabling the system to use the most suitable transmission link and network when executing the audiovisual service and maximizing the user experience.

[0104] Figure 3The design framework of a system for implementing heterogeneous network audio-visual service scheduling provided in an embodiment of the present application is shown. The overall system design is based on multi-link and multi-standard distribution of different services, and uses intelligent detection technology to implement the dynamic allocation and switching function of media flow at the backend of the system, enabling front-end users to achieve seamless switching in a heterogeneous network environment when the network quality is not ideal. During the video playback process of a certain audio-visual service, the scheduling process of the entire system for the transmission link is as follows:

[0105] 1. In the initial stage of starting playback, the backend device of the system will automatically notify the front-end device of the default start-up transmission link information. Subsequently, the front-end device will access the corresponding transmission link using the network connected by the user according to the received information and start the video playback process;

[0106] 2. During the video playback process, the front-end device will periodically report the current transmission quality data to the backend device. These data may include, but are not limited to, key parameters such as the highest downlink rate, average rate, network jitter, and round-trip delay;

[0107] 3. The backend device will continuously collect and analyze the transmission quality data of all transmission links, and judge the load situation of each transmission link and whether there is a congestion situation through these transmission quality data, and then select a suitable start-up transmission link for the newly connected front-end device to transmit the video stream;

[0108] 4. Once it is found that a certain transmission link has congestion or a failure occurs, the backend device will comprehensively consider factors such as the priority of the front-end device, the cost of the transmission line, and load balancing, and send a link switching instruction to the front-end device on this link;

[0109] 5. After receiving the link switching instruction sent by the backend device, the front-end device will quickly switch to the new transmission link and start the streaming media transmission work. At the same time, the backend device needs to ensure that the delay difference between the two transmission links before and after the switching is controlled within the range of the media stream adaptive cache of the front-end device to avoid playback interruption;

[0110] 6. The adaptive conversion module in the front-end device will complete the seamless switching at the key frame position of the new media stream, ensuring that the audio and video of the video player can be kept synchronized during the switching process, so as to achieve a smooth playback experience and avoid any stuttering or discontinuity.

[0111] In particular, the methods and / or embodiments in the embodiments of the present application can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. When the computer program is executed by a processing unit, the above functions defined in the solutions of the present application are executed.

[0112] It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device.

[0113] In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0114] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0115] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0116] As another aspect, this application also provides a computer-readable medium. The computer-readable medium can be included in the devices described in the above embodiments; or it can exist separately and not be assembled into the device. The above computer-readable medium carries one or more computer program instructions, and the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of the multiple embodiments of this application described above.

[0117] It should be noted that the present application can be implemented in software and / or a combination of software and hardware. For example, it can be implemented using an application specific integrated circuit (ASIC), a general purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and the like. Additionally, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with the processor to execute each step or function.

[0118] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present application, the present application can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be construed as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the apparatus claims can also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are used to denote names and do not denote any particular order.

Claims

1. A back-end device for implementing heterogeneous network audio-visual service scheduling, characterized in that: The back-end equipment includes: Link management module, used to create and manage transmission links for different audio-visual services; A link quality monitoring module is used to obtain transmission quality data reported by the front-end device and determine the status information of the transmission link and network currently used by the front-end device based on the transmission quality data; The link scheduling module is used to allocate a transmission link to a terminal device requesting to perform a network audio-visual service, and to send a switching instruction to a front-end device based on the status information, so that the front-end device obtains the media stream of the audio-visual service based on the transmission link and the accessed network, and switches the currently used transmission link and / or network according to the switching instruction.

2. The back-end device according to claim 1, characterized in that: The transmission links created and managed by the link management module comply with different standards and are used to transmit media streams with different bit rates, resolutions and transmission protocols.

3. The back-end device according to claim 1, characterized in that: The link scheduling module is also used to extract characteristic information of the transmission quality data based on the collected historical data of the transmission quality data, and predict the congestion trend of the transmission link in combination with the current time and link load conditions, and generate a switching instruction based on the prediction result and the status information.

4. The back-end device according to claim 3, characterized in that: The link scheduling module is further configured to allocate a transmission link to a newly connected front-end device according to the prediction result and the status information.

5. A front-end device for implementing heterogeneous network audio-visual service scheduling, characterized in that: The front-end device includes: a link switching control module, configured to, when requesting to execute a network audiovisual service, obtain the media stream of the audiovisual service based on the transmission link and network accessed by the backend device, and switch the currently used transmission link and / or network according to the switching instruction, wherein the switching instruction is determined by the backend device based on status information of the transmission link and network currently used by the front-end device; The link detection module is used to detect the transmission links and networks currently available to the front-end device and detect the transmission quality data of the currently available transmission links and networks; A link quality reporting module, configured to report transmission quality data to the back-end device, so that the back-end device determines the status information of the transmission link and network currently used by the front-end device based on the transmission quality data; The video playback module is used to play audio and video content according to the media stream of the acquired audio and video service.

6. The front-end device according to claim 5, characterized in that: The front-end device also includes: The adaptive conversion module is used to convert media streams of different bit rates, resolutions and transmission protocols transmitted from transmission links of different standards into a unified standard adaptive bit stream to provide it to the video playback module for playback.

7. The front-end device according to claim 6, characterized in that: The adaptive conversion module is further configured to complete seamless switching at a key frame position of the new media stream according to timestamp information inserted in the media stream after the link switching control module switches the currently used transmission link and / or network according to the switching instruction.

8. The front-end device according to claim 5, characterized in that: The link switching control module is also used to implement fault diagnosis, monitor the quality of media streams in real time, and detect and report abnormalities or interruptions.

9. The front-end device according to claim 5, characterized in that: The link detection module is used to continuously detect transmission links and networks to identify transmission links and networks currently available to the front-end device, and to detect transmission quality data of the currently available transmission links and networks; The link quality reporting module is used to send the transmission quality data to the back-end device through a message in a preset format, so that the back-end device can determine the status information of the transmission link and network currently used by the front-end device based on the transmission quality data.

10. A system for implementing heterogeneous network audio-visual service scheduling, characterized in that: The device comprises the back-end device according to any one of claims 1 to 4 and the front-end device according to any one of claims 5 to 9.

Images