Multimedia data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a multimedia data processing method and device, equipment and a storage medium. The method comprises the following steps: in response to a received transmission request of a client device for multimedia data, transmitting the multimedia data to the client device based on a selected transmission control requirement; acquiring a first record for transmission of the multimedia data, wherein the first record at least comprises a first transmission identifier for indicating transmission and a transmission control requirement; acquiring a second record of the client device for the transmission feedback, wherein the second record at least comprises a second transmission identifier indicating the transmission and a quality of experience (QoE) feedback for the transmission; and in response to determining that the first transmission identifier and the second transmission identifier are matched, determining a QoE evaluation result for the transmission control requirement based on the first record and the second record.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the computer field, and in particular, to a method, an apparatus, a device, a computer-readable storage medium, and a computer program product for multimedia data processing. Background Art

[0002] With the development of computer technology, more and more applications are configured to provide multimedia content to users to meet their interactive needs, such as video, audio, image, document, etc. Users usually expect to obtain smooth and high-quality multimedia content. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for processing multimedia data is provided. The method includes: in response to receiving a transmission request for multimedia data from a client device, transmitting the multimedia data to the client device based on a selected transmission control requirement; obtaining a first record for the transmission of the multimedia data, the first record at least including a first transmission identifier indicating the transmission and a transmission control requirement; obtaining a second record for transmission feedback from the client device, the second record at least including a second transmission identifier indicating the transmission and a quality of experience (QoE) feedback for the transmission; and in response to determining that the first transmission identifier and the second transmission identifier match, determining a QoE evaluation result for the transmission control requirement based on the first record and the second record.

[0004] In a second aspect of the present disclosure, a device for multimedia data processing is provided. The device includes: a transmission module, configured to transmit multimedia data to a client device based on a selected transmission control requirement in response to receiving a transmission request for multimedia data from a client device; a first acquisition module, configured to acquire a first record of the transmission of multimedia data, the first record at least including a first transmission identifier indicating the transmission and a transmission control requirement; a second acquisition module, configured to acquire a second record of the transmission feedback of the client device, the second record at least including a second transmission identifier indicating the transmission and a quality of experience (QoE) feedback for the transmission; and a determination module, configured to determine a QoE evaluation result for the transmission control requirement based on the first record and the second record in response to determining that the first transmission identifier and the second transmission identifier match.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure may be implemented;

[0011] Figure 2 A flowchart showing a signaling flow of multimedia data processing according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing an example architecture of multimedia data processing according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic diagram showing an example architecture of multimedia data processing according to some embodiments of the present disclosure;

[0014] Figure 5 A flowchart showing a partial process of multimedia data processing according to some embodiments of the present disclosure;

[0015] Figure 6 A flowchart showing a process of multimedia data processing according to some embodiments of the present disclosure;

[0016] Figure 7 A schematic structural block diagram showing an example apparatus for multimedia data processing according to some embodiments of the present disclosure; and

[0017] Figure 8 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0020] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0021] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0022] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0023] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.

[0024] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0025] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0026] As mentioned above, with the development of computer technology, more and more applications are configured to provide multimedia content to users to meet their interactive needs, such as video, audio, images, documents, etc. In order to avoid network congestion and improve the smoothness of multimedia data transmission, more and more content providers use congestion control algorithms to control the transmission of multimedia data, and the types of congestion control algorithms are increasing. However, due to the diversity of networks in actual application scenarios, the network quality may be different in different regions, types, and time periods. Moreover, different congestion control algorithms may have different optimization goals and design limitations, resulting in different types of congestion control algorithms bringing different user experiences. If the quality of experience (Quality of Experience, abbreviated as QoE) brought to users by the congestion control algorithm can be accurately determined, it will be beneficial to the subsequent adjustment and optimization of the congestion control algorithm.

[0027] In view of this, an embodiment of the present disclosure proposes an improved solution for multimedia data processing. In this solution, if a transmission request for multimedia data is received from a client device, the multimedia data is transmitted to the client device based on the selected transmission control requirement. A first record for the transmission of the multimedia data is obtained, and the first record at least includes a first transmission identifier indicating the transmission and a transmission control requirement. A second record for transmission feedback from the client device is obtained, and the second record at least includes a second transmission identifier indicating the transmission and QoE feedback for the transmission. In response to determining that the first transmission identifier and the second transmission identifier match, a QoE evaluation result for the transmission control requirement is determined based on the first record and the second record.

[0028] In this way, the embodiments of the present disclosure can combine the first record of the server device and the second record of feedback from the client device to determine the QoE evaluation results of the transmission control requirements (such as congestion control algorithms), and can provide parameters for the subsequent selection, adjustment and optimization of the transmission control requirements, which is conducive to improving the transmission quality of multimedia data.

[0029] Various example implementations of the solution are described in detail below in conjunction with the accompanying drawings.

[0030] Example Environment

[0031] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the example environment 100, an application 120 is installed in a client device 110. A user 140 can interact with the application 120 via the client device 110 and / or an attached device of the client device 110.

[0032] In the embodiments of the present disclosure, the application 120 may be any appropriate application associated with providing multimedia data. For example, the application 120 may be an application that provides multimedia data such as audio, video, or audio and video to users. Figure 1 In the environment, if the application 120 is in an active state, the client device 110 can present an interface 150 of the application 120. The interface 150 can include various interfaces that the application 120 can provide, such as an interface for playing videos, an interface for playing audio, and the like.

[0033] In some embodiments, the client device 110 communicates with the server device 130 to provide services for the application 120. The client device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the client device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The server device 130 can be various types of computing systems / servers that can provide computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0034] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0035] Example Interaction

[0036] Figure 2 1 is a flowchart of a signaling flow 200 of multimedia data processing according to some embodiments of the present disclosure. The signaling flow 200 involves the client device 110 and the server device 130. Figure 1 The signaling flow 200 is described with reference to the environment 100 in FIG. 1 , but this is merely exemplary.

[0037] In an embodiment of the present disclosure, as shown in the signaling flow 200, the client device 110 sends (202) a transmission request for multimedia data to the server device 130, and the server device 130 receives (204) the transmission request from the client device 110. In response to receiving the transmission request, the server device 130 selects (206) a transmission control requirement. Thereafter, the server device 130 sends (208) multimedia data to the client device 110 based on the selected transmission control requirement. The multimedia data may include, but is not limited to, audio, video, text, images, and the like. After receiving (210) the multimedia data from the server device 130, the client device 110 may output the multimedia data, for example, the client device 110 may play audio, or present video, text, images, and the like.

[0038] The transmission control requirements are used to control the transmission process of multimedia data. The transmission control requirements may include control strategies, requirements, and the like for the transmission process of multimedia data. Generally speaking, the server device 130 controls the transmission process of multimedia data based on the transmission control requirements, hoping to efficiently and stably transmit the multimedia data to the client device 110. The performance of the transmission control requirements will directly affect the transmission of multimedia data, and thus can affect the user's quality of experience. In the case of different transmission control requirements, the control strategies and requirements of the transmission control requirements may be different. In some examples, the transmission control requirements may include a congestion control algorithm, which may include requirements for the startup process of the server device 130 transmitting multimedia data, requirements for the transmission rate, requirements for retransmission of lost data segments, recovery mechanisms when congestion occurs, and the like. In some examples, the transmission control requirements may also include any other appropriate algorithms, control strategies, or requirements, and the embodiments of the present disclosure do not limit the specific content of the transmission control requirements.

[0039] In some embodiments, the server device 130 may be implemented as a single server or a server system composed of multiple servers. Figure 3 As shown, Figure 3A schematic diagram of an example architecture 300 for multimedia data processing according to some embodiments of the present disclosure is shown. In the example architecture 300, the server device 130 includes a multimedia server 310, a transmission control engine 320, and a log server 330. The client device 110 can send a transmission request for multimedia data (e.g., video) to the multimedia server 310. In response to receiving the transmission request, the multimedia server 320 selects a transmission control requirement (e.g., a congestion control algorithm). Thereafter, the multimedia server 320 transmits the video to the client device 110 via a network connection between the multimedia server 320 and the client device 110. Furthermore, the multimedia server 320 controls the transmission process of the video using the selected congestion control algorithm to avoid network congestion during the video transmission process.

[0040] Regarding the selection of the transmission control requirement, in some embodiments, the server device 130 may pre-build a set of transmission control requirements. If a transmission request is received, the server device 130 may generate a random number. The server device 130 may select a transmission control requirement from the set of transmission control requirements based on the random number. In this way, random selection of the transmission control requirement can be achieved.

[0041] As an example, each transmission control requirement in a set of transmission control requirements may have a corresponding requirement number. For example, there may be K transmission control requirements in the set, and the requirement numbers of the K transmission control requirements may be 0, 1, 2, ..., K-1, respectively. The following is a correspondence table of requirement numbers and requirement names of an example set.

[0042] Table 1

[0043] Request Number Required Name 0 Transmission Control Request 0 1 Transmission Control Requirement 1 2 Transmission Control Requirement 2 … … K-1 Transmission Control Requirements K-1

[0044] The transmission request may include configuration information of the client device, and the configuration information may include at least one of the following configuration parameters: client identifier, transmission identifier, time information, network information, etc. The client identifier is used to indicate the client device 110. The transmission identifier is used to indicate the transmission operation of the multimedia data, and the transmission identifier may be a randomly assigned transmission number (ID) that can indicate the transmission operation. The network information is used to indicate the network type of the communication network between the server device 130 and the client device 110.

[0045] The server device 130 can generate a random number based on at least one configuration parameter in the configuration information of the client device. For example, the server device 130 can perform a hash calculation on the transmission identifier, perform a modulo operation on the calculation result with the total number of transmission control requirements in the set (for example, K), and use the result of the modulo operation as the random number. Afterwards, the server device 130 can select a transmission control requirement whose requirement number matches the random number from the set of transmission control requirements. Assuming the random number is "2", the server device 130 can select "Transmission Control Requirement 2" with a requirement number of "2".

[0046] As another example, Figure 4 and Figure 5 As shown, Figure 4 A schematic diagram showing an example architecture 400 for multimedia data processing according to some embodiments of the present disclosure is shown. Figure 5 A flowchart of an example process 500 of multimedia data processing according to some embodiments of the present disclosure is shown. The example architecture 400 shows a first set 410 and a second set 420. The first set 410 can be understood as a set consisting of transmission control requirements A1, A2, ..., AN to be evaluated. The second set 420 can be understood as a set consisting of evaluated transmission control requirements B1, B2, ..., BM. The evaluated transmission control requirements B1, B2, ..., BM here can be transmission control requirements with better performance (such as congestion control algorithms with better performance) selected according to the QoE evaluation results. The server device 110 can predetermine a predetermined numerical range, and the predetermined numerical range can be divided into a first sub-range and a second sub-range that do not overlap with each other. The first sub-range can be matched to the first set, and the second sub-range can be matched to the second set.

[0047] In box 510 of process 500, the server device 130 receives a transmission request for multimedia data from a client device. In box 520, in response to receiving the transmission request, the server device 130 generates a random number within a predetermined numerical range, for example, a random number within the range of (0, 1). In box 530, the server device 130 determines whether the random number falls within a first sub-range. If it is determined in box 530 that the random number falls within the first sub-range, the process 500 proceeds to box 540. In box 540, the server device 130 selects a transmission control requirement to be evaluated from the first set 410. If it is determined in box 530 that the random number does not fall within the first sub-range, indicating that the random number falls within the second sub-range, the process 500 proceeds to box 550. In box 550, the server device 130 selects an evaluated transmission control requirement from the second set 420.

[0048] For example, the predetermined numerical range may be (0, 1), the first sub-range may be (0, X), and the second sub-range may be (X, 1). If the random number falls within the range of (0, X), the server device 130 selects the transmission control requirement to be evaluated. If the random number falls within the range of (X, 1), the server device 130 selects the transmission control requirement that has been evaluated. In this way, the number of transmission times of multimedia data transmitted using the transmission control requirement to be evaluated accounts for approximately X*100% of the total number of transmission times of multimedia data, and the number of transmission times of multimedia data transmitted using the evaluated transmission control requirement accounts for approximately (1-X)*100% of the total number of transmission times of multimedia data. By adjusting the specific value of X, the number of times the transmission control requirement to be evaluated and the evaluated transmission control requirement are used to transmit multimedia data and the ratio between the two can be adjusted, which can not only verify the performance of the transmission control requirement to be evaluated, but also ensure the user experience.

[0049] In some examples, the difference between the upper limit and the lower limit of the first sub-range is less than the difference between the upper limit and the lower limit of the second sub-range. Assuming that the first sub-range is (0, X) and the second sub-range is (X, 1), the specific value of X can be configured to be less than or equal to 0.05 (for example, 0.04). In this way, it can be ensured that the number of times the transmission control requirement to be evaluated is selected is much less than the number of times the evaluated transmission control requirement is selected, and it can be ensured that most users are not affected by the evaluation process of the transmission control requirement to be evaluated, and the experience quality of most users can be ensured.

[0050] It is understandable that the above-mentioned predetermined numerical range, the first sub-range and the second sub-range are all exemplary, and any appropriate range can be selected according to actual needs, and the embodiments of the present disclosure do not specifically limit this. It should also be noted that the above-mentioned method of selecting a transmission control requirement from a set of transmission control requirements using a random number is only exemplary. In actual application, the server device 130 can select any appropriate method to select a transmission control requirement from a set of transmission control requirements, and the embodiments of the present disclosure do not specifically limit this.

[0051] In an embodiment of the present disclosure, as shown in the signaling flow 200, the server device 130 obtains (212) a first record for the transmission of multimedia data, the first record at least including a first transmission identifier and a transmission control requirement, the first transmission identifier is used to indicate the transmission operation of the multimedia data this time. In some embodiments, the first record may also include configuration information of the server device 130, such as location information of the server device 130, etc. It is understandable that the first record is not limited to including the above information, but may also include any other appropriate information, which is not limited in the embodiment of the present disclosure.

[0052] As an example, the multimedia data may include video data. In response to receiving the video playback request, the client device 110 may randomly generate a session number (ID) for identifying the current video session. Figure 3 As shown, the client device 110 may send a transmission request carrying the session number to the multimedia server 310. In response to the transmission request, the multimedia server 310 transmits the video data to the client device 110 based on the selected transmission control requirement. After the video data transmission is completed, the multimedia server 310 may upload a first record including the session number and the transmission control requirement to the log server 330.

[0053] In an embodiment of the present disclosure, as shown in the signaling flow 200, the client device 110 feeds back (214) a second record to the server device 130, and the server device 130 receives (216) the second record. The second record includes at least a second transmission identifier indicating the transmission and experience quality feedback for the transmission. The second transmission identifier is also used to indicate the transmission operation of the multimedia data this time, and the second transmission identifier and the first transmission identifier may be the same or different. In some examples, when the multimedia data includes video data, the first transmission identifier and the second transmission identifier may both include the session number of the current video session.

[0054] The QoE feedback may include at least one QoE indicator that can directly or indirectly indicate the quality of experience of the transmission of multimedia data. In some examples, the at least one QoE indicator may include, for example, the viewing time of the user viewing the multimedia data, the start time from when the client device 110 receives a request to play the multimedia data to when the client device 110 starts to present the multimedia data, network delay, packet loss rate, clarity of the multimedia data, etc.

[0055] In some embodiments, the second record may also include configuration information of the client device 110, and the configuration information of the client device may include at least one configuration parameter. In some examples, the at least one configuration parameter may include a client identifier, which is used to indicate the client device or the client on the client device. For example, the client may be configured to be able to play a video (e.g., a video player), and the client identifier may include, for example, a player number that can indicate the video player. Alternatively or additionally, the at least one configuration parameter may also include network information, which may indicate the network type of the communication network to which the client device 110 is connected (e.g., a WIFI network, a 4G network, a 5G network, etc.). Alternatively or additionally, the second record may also include time information (e.g., a timestamp), which may indicate the time when the client device 110 outputs the multimedia data.

[0056] In some examples, such as Figure 3As shown, the client device 110 can receive video data from the multimedia server 310. After determining that the video data has been received, the client device 110 can upload a second record to the log server 330. The second record may include a player number of the video player on the client device 110, a session number indicating this video session, time information, network information, and QoE feedback.

[0057] In an embodiment of the present disclosure, as shown in signaling flow 200, in response to determining that the first transmission identifier and the second transmission identifier match, the server device 130 determines (220) a QoE evaluation result for the transmission control requirement based on the first record and the second record. The QoE evaluation result may indicate the QoE of transmitting multimedia data to the client device 110 based on the transmission control requirement, where the QoE may also be referred to as the QoE of the transmission control requirement. In some examples, the QoE evaluation result may include a QoE level or a QoE metric that can indicate the QoE of the transmission control requirement.

[0058] In some embodiments, as shown in the signaling flow 200, after the server device 130 determines that the multimedia data transmission is completed, the server device 130 can determine the first record and the second record for the transmission of the multimedia data based on the matching of the first transmission identifier and the second transmission identifier. The server device 130 can generate (218) a target transmission record for the transmission based on the first record and the second record of the transmission of the multimedia data. Afterwards, the server device 130 can determine (220) a QoE evaluation result based at least on the target transmission record.

[0059] The target transmission record includes at least transmission control requirements and transmission QoE feedback. In some examples, the target transmission record may include configuration information of the corresponding client device 110, and the configuration information of the client device 110 includes at least one configuration parameter, for example, the at least one configuration parameter may include a client identifier (e.g., a player number of a video player), network information, and the like. Alternatively or additionally, the target transmission record may also include configuration information of the server device 130, such as location information of the server device 130, and the like. Alternatively or additionally, the target transmission record may also include a transmission identifier indicating the transmission of multimedia data (e.g., a session number of a video session). Alternatively or additionally, the target transmission record may also include time information (e.g., a timestamp), which may indicate the time when the client device 110 outputs the multimedia data (e.g., the playback time of the video).

[0060] In some embodiments, the server device 130 may save the target transmission record to a database. Figure 3As shown, the log server 330 can generate a transmission record for the transmission of multimedia data based on the first record and the second record. Afterwards, the log server 330 can save the transmission record in a database at the log server 330. It should be noted that it is not necessary to save the target transmission record to the database. In some examples, after the server device 130 generates the target transmission record, it can also directly determine the QoE evaluation result based on the target transmission record.

[0061] In some embodiments, the server device 130 may select at least one transmission record including a target transmission record from a database. The at least one transmission record may include the same transmission control requirement, and the at least one transmission record may also include at least one target configuration parameter of the client device 110. The server device 130 may determine a QoE evaluation result for the transmission control requirement based on the at least one transmission record. The QoE evaluation result can indicate the QoE of transmitting multimedia data to a client device having at least one target configuration parameter.

[0062] In some examples, at least one target configuration parameter of the client device 110 may include network information. The server device 130 may obtain multiple transmission records within a predetermined time range (e.g., the previous day, the previous week, etc.). The multiple transmission records may include the same transmission control requirement, and the network information of the multiple transmission records indicates that the network type of the communication network to which the corresponding client device 110 is connected is the same. On this basis, the determined QoE evaluation result can indicate the transmission control requirement for the QoE of the client device 110 connected to the specific communication network.

[0063] Alternatively or additionally, the server device 130 may obtain multiple transmission records within a predetermined time range. The multiple transmission records may include the same transmission control requirement, at least one target configuration parameter of the client device 110, and at least one target configuration parameter of the server device 130. For example, the multiple transmission records may include the same transmission control requirement. The network information of the multiple transmission records may indicate that the network type of the communication network to which the corresponding client device 110 is connected is the same. Moreover, the multiple transmission records may also include the location information of the same server device 130, that is, the multiple transmission records correspond to the same server device 130. In this way, the determined QoE evaluation result can indicate the QoE of the transmission control requirement for the client device 110 within a specific type of communication network and the server device 130 at a specific location.

[0064] In some examples, such as Figure 3As shown, the server device 130 may be deployed with a transmission control engine 320. The transmission control engine 320 may obtain multiple transmission records within a predetermined time range from the log server 330. The multiple transmission records may be transmission records formed by the same multimedia server 310 transmitting video data to a client device 110 in a specific type of communication network based on the same transmission control requirement. In this way, the determined QoE evaluation result can indicate the QoE of the multimedia server 310 at a specific location transmitting video data to a client device 110 in a specific type of communication network based on a specific transmission control requirement.

[0065] Regarding the determination process of the QoE evaluation result, in actual application, any appropriate QoE evaluation method can be selected according to actual needs to determine the QoE evaluation result for the transmission control requirement based on the transmission record. The embodiment of the present disclosure does not specifically limit the QoE evaluation method. As an example, the transmission control engine 320 can obtain multiple transmission records within a predetermined time range from the log server 330. The transmission control engine 320 can use at least one QoE indicator (such as viewing time, start time, network delay, packet loss rate, etc.) in the QoE feedback of each transmission record as an input parameter of the QoE evaluation method. The QoE evaluation method is used to determine the QoE metric corresponding to each transmission record based on the QoE indicator in each transmission record. Afterwards, the transmission control engine 320 can determine the average QoE metric of the transmission control requirement within the predetermined time range based on the QoE metric corresponding to the multiple transmission records, and use the average QoE metric as the QoE evaluation result of the transmission control requirement within the predetermined time range.

[0066] In some embodiments, the server device 130 may determine whether the QoE indicated by the QoE evaluation result of each transmission control requirement meets the predetermined quality requirement. If it is determined that the QoE indicated by the QoE evaluation result of the transmission control requirement meets the predetermined quality requirement, the transmission control requirement is added to the third set. The transmission control requirements in the third set are selected for transmitting multimedia data to the client device in a subsequent time period.

[0067] The specific requirements for the predetermined quality requirements can be set according to actual needs. In some examples, the predetermined quality requirements may include requirements for the QoE level or QoE metric in the QoE evaluation result. The server device 130 can determine that the QoE level or QoE metric in the QoE evaluation result meets the predetermined quality requirements. If the QoE level or QoE metric meets the predetermined quality requirements, the corresponding transmission control requirements are added to the third set.

[0068] In other examples, the predetermined quality requirements may include a requirement for the ranking of QoE metrics in the QoE evaluation results. The server device 130 may determine the ranking of QoE metrics in the QoE evaluation results of multiple transmission control requirements, and may select a predetermined number of transmission control requirements with the highest ranking to add to the third set. For example, the transmission control engine 320 may obtain different transmission control requirements c in each time interval t (e.g., each day, each hour) in a predetermined time period ω (e.g., one month). i These QoE evaluation results include different transmission control requirements c i Average QoE metric for a server device 130 at a specific location l and a specific type of communication network a The transport control engine 320 may determine the average QoE metric by, for example, the following formula: Highest transmission control requirements * .

[0069]

[0070] The transmission control engine 320 can transmit the transmission control request c * Added to the third set of, for example, the multimedia server 310. If the multimedia server 310 receives a transmission request at time interval t+1, it may select a transmission control requirement c from the third set. * . Then, based on the transmission control requirement c * The multimedia data is sent to the client device 110. It should be noted that the above-mentioned methods for selecting the predetermined quality requirement and the transmission control requirement are exemplary, and any appropriate predetermined quality requirement may be selected to select the transmission control requirement according to actual needs. The embodiments of the present disclosure do not specifically limit this aspect.

[0071] In some examples, the third set and the second set can be implemented as the same set. Of course, the third set can also be different from the second set. For example, the third set can be a set of transmission control requirements constructed for a specific server device, a client device in a specific type of communication network, or a specific client device. On this basis, the specific server device can use the transmission control requirements in the third set to transmit multimedia data to the client device. The transmission control requirements in the third set can be used to control the transmission of multimedia data to a client device in a specific type of communication network, or can be used to control the transmission of multimedia data to a specific client device.

[0072] In this way, the embodiments of the present disclosure can combine the first record of the server device and the second record of feedback from the client device to determine the QoE evaluation results of the transmission control requirements (such as congestion control algorithms), and can provide parameters for the subsequent selection, adjustment and optimization of the transmission control requirements, which is conducive to improving the transmission quality of multimedia data.

[0073] Example procedures, devices and equipment

[0074] Figure 6 A flowchart of a process 600 of multimedia data processing according to some embodiments of the present disclosure is shown. The process 600 may be implemented at the server device 130, or may be implemented by the server device 130 and the client device 110 in collaboration.

[0075] In block 610 , in response to receiving a transmission request for multimedia data from a client device, the server device 130 transmits the multimedia data to the client device based on the selected transmission control requirement.

[0076] In block 620 , the server device 130 obtains a first record for transmission of multimedia data, where the first record at least includes a first transmission identifier indicating the transmission and a transmission control requirement.

[0077] In block 630 , the server device 130 obtains a second record of transmission feedback from the client device, where the second record at least includes a second transmission identifier indicating the transmission and quality of experience (QoE) feedback on the transmission.

[0078] In block 640 , in response to determining that the first transmission identifier and the second transmission identifier match, the server device 130 determines a QoE evaluation result for the transmission control requirement based on the first record and the second record.

[0079] In some embodiments, transmitting multimedia data to a client device based on a selected transmission control requirement includes: generating a random number in response to receiving a transmission request; selecting a transmission control requirement from a set of transmission control requirements based on the random number; and transmitting the multimedia data to the client device based on the selected transmission control requirement.

[0080] In some embodiments, selecting a transmission control requirement based on a random number includes: in response to a random number generated within a predetermined numerical range falling into a first sub-range within the predetermined numerical range, selecting a first transmission control requirement from a first set including transmission control requirements to be evaluated; and in response to a random number generated within a predetermined numerical range falling into a second sub-range within the predetermined numerical range, selecting a second transmission control requirement from a second set including evaluated transmission control requirements, the first sub-range and the second sub-range not overlapping with each other.

[0081] In some embodiments, the difference between the upper limit and the lower limit of the first sub-range is less than the difference between the upper limit and the lower limit of the second sub-range.

[0082] In some embodiments, determining a QoE evaluation result based on a first record and a second record includes: in response to determining that a first transmission identifier in the first record and a second transmission identifier in the second record match, generating a target transmission record for the transmission based on the first record and the second record, the target transmission record including at least a transmission control requirement and QoE feedback of the transmission; and determining a QoE evaluation result for the transmission control requirement based at least on the generated target transmission record.

[0083] In some embodiments, the generated transmission records are maintained in a database, each transmission record in the database also includes configuration information of the corresponding client device, the configuration information includes at least one configuration parameter, and wherein at least based on the generated target transmission record, determining the QoE evaluation result for the transmission control requirement includes: selecting at least one transmission record including the transmission control requirement and including at least one target configuration parameter from the database, at least one transmission record including the target transmission record; and based on the at least one transmission record, determining the QoE evaluation result for the transmission control requirement, the QoE evaluation result indicating the QoE of transmitting multimedia data to a client device having at least one target configuration parameter.

[0084] In some embodiments, process 600 further includes: in response to determining that the QoE indicated by the QoE evaluation result of the transmission control requirement meets the predetermined quality requirement, adding the transmission control requirement to a third set, and the transmission control requirements in the third set are selected for transmitting multimedia data to the client device in a subsequent time period.

[0085] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 7 A schematic structural block diagram of an example apparatus 700 for multimedia data processing according to some embodiments of the present disclosure is shown. The apparatus 700 may be implemented as or included in the server device 130. Each module / component in the apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0086] like Figure 7As shown, the apparatus 700 includes: a transmission module 710, a first acquisition module 720, a second acquisition module 730 and a determination module 740. The transmission module 710 is configured to transmit multimedia data to the client device based on the selected transmission control requirement in response to receiving a transmission request for multimedia data from the client device. The first acquisition module 720 is configured to obtain a first record for the transmission of multimedia data, the first record at least including a first transmission identifier indicating the transmission and a transmission control requirement. The second acquisition module 730 is configured to obtain a second record for the transmission feedback of the client device, the second record at least including a second transmission identifier indicating the transmission and a quality of experience QoE feedback for the transmission. The determination module 740 is configured to determine the QoE evaluation result for the transmission control requirement based on the first record and the second record in response to determining that the first transmission identifier and the second transmission identifier match.

[0087] In some embodiments, the transmission module 710 is further configured to: generate a random number in response to receiving a transmission request; select a transmission control requirement from a set of transmission control requirements based on the random number; and transmit multimedia data to the client device based on the selected transmission control requirement.

[0088] In some embodiments, the transmission module 710 is further configured to: in response to a random number generated within a predetermined numerical range falling into a first sub-range within the predetermined numerical range, select a first transmission control requirement from a first set including transmission control requirements to be evaluated; and in response to a random number generated within a predetermined numerical range falling into a second sub-range within the predetermined numerical range, select a second transmission control requirement from a second set including evaluated transmission control requirements, the first sub-range and the second sub-range do not overlap with each other.

[0089] In some embodiments, the difference between the upper limit and the lower limit of the first sub-range is less than the difference between the upper limit and the lower limit of the second sub-range.

[0090] In some embodiments, the determination module 740 is further configured to: in response to determining that the first transmission identifier in the first record and the second transmission identifier in the second record match, generate a target transmission record for the transmission based on the first record and the second record, the target transmission record including at least the transmission control requirements and the QoE feedback of the transmission; and determine the QoE evaluation result for the transmission control requirements based at least on the generated target transmission record.

[0091] In some embodiments, the generated transmission records are maintained in a database, each transmission record in the database also includes configuration information of the corresponding client device, the configuration information includes at least one configuration parameter, and the determination module 740 is further configured to: select at least one transmission record including a transmission control requirement and including at least one target configuration parameter from the database, at least one transmission record including a target transmission record; and based on the at least one transmission record, determine a QoE evaluation result for the transmission control requirement, the QoE evaluation result indicating the QoE of transmitting multimedia data to a client device having at least one target configuration parameter.

[0092] In some embodiments, the device 700 also includes: an adding module, configured to add the transmission control requirement to a third set in response to determining that the QoE indicated by the QoE evaluation result of the transmission control requirement meets the predetermined quality requirement, and the transmission control requirements in the third set are selected for transmitting multimedia data to the client device in a subsequent time period.

[0093] The units and / or modules included in the device 700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 700 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] Figure 8 8 is a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 8 The electronic device 800 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 8 The electronic device 800 shown may include or be implemented as Figure 1 The terminal device 120, or Figure 7 Device 700.

[0095] like Figure 8As shown, the electronic device 800 is in the form of a general electronic device. The components of the electronic device 800 may include, but are not limited to, one or more processors or processing units 810, a memory 820, a storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 820. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 800.

[0096] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 800.

[0097] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 8 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0098] The communication unit 840 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 800 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0099] The input device 850 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 860 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 800 may also communicate with one or more external devices (not shown) through the communication unit 840 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 800, or communicate with any device that allows the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0100] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0101] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0105] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing multimedia data, applied to a server device, comprising: In response to receiving a transmission request for multimedia data from a client device, transmitting the multimedia data to the client device based on the selected transmission control requirement; Acquire a first record of the transmission of the multimedia data, the first record comprising at least a first transmission identifier indicating the transmission and the transmission control requirement; Acquire a second record of the transmission feedback by the client device, where the second record at least includes a second transmission identifier indicating the transmission and a quality of experience (QoE) feedback for the transmission; as well as In response to determining that the first transmission identifier and the second transmission identifier match, a QoE evaluation result for the transmission control requirement is determined based on the first record and the second record.

2. The method of claim 1 , wherein transmitting the multimedia data to the client device based on the selected transmission control requirement comprises: In response to receiving the transmission request, generating a random number; Selecting a transmission control requirement from a set of transmission control requirements based on the random number; as well as Based on the selected transmission control requirement, the multimedia data is transmitted to the client device.

3. The method of claim 2, wherein selecting a transmission control requirement based on the random number comprises: In response to the random number generated within a predetermined value range falling within a first sub-range within the predetermined value range, selecting a first transmission control requirement from a first set of transmission control requirements to be evaluated; as well as In response to the random number generated within the predetermined numerical range falling into a second sub-range within the predetermined numerical range, a second transmission control requirement is selected from a second set of evaluated transmission control requirements, the first sub-range and the second sub-range not overlapping each other. 4 . The method of claim 3 , wherein a difference between an upper limit and a lower limit of the first subrange is smaller than a difference between an upper limit and a lower limit of the second subrange.

5. The method according to claim 1, wherein determining the QoE evaluation result based on the first record and the second record comprises: In response to determining that the first transmission identifier in the first record matches the second transmission identifier in the second record, generating a target transmission record for the transmission based on the first record and the second record, the target transmission record including at least the transmission control requirement and QoE feedback of the transmission; as well as The QoE evaluation result for the transmission control requirement is determined based at least on the generated target transmission record.

6. The method of claim 5, wherein the generated transmission records are maintained in a database, each transmission record in the database further comprising configuration information of a corresponding client device, the configuration information comprising at least one configuration parameter, and wherein based at least on the generated target transmission record, determining the QoE evaluation result for the transmission control requirement comprises: selecting from the database at least one transmission record including the transmission control requirement and including at least one target configuration parameter, the at least one transmission record including the target transmission record; as well as Based on the at least one transmission record, the QoE evaluation result for the transmission control requirement is determined, the QoE evaluation result indicating a QoE of transmitting multimedia data to a client device having the at least one target configuration parameter.

7. The method according to claim 1, further comprising: In response to determining that the QoE indicated by the QoE evaluation result of the transmission control requirement meets a predetermined quality requirement, the transmission control requirement is added to a third set, and the transmission control requirements in the third set are selected for transmitting multimedia data to the client device in a subsequent time period.

8. A device for multimedia data processing, comprising: a transmission module, configured to, in response to receiving a transmission request for multimedia data from a client device, transmit the multimedia data to the client device based on a selected transmission control requirement; A first acquisition module is configured to acquire a first record of the transmission of the multimedia data, wherein the first record at least includes a first transmission identifier indicating the transmission and the transmission control requirement; A second acquisition module is configured to obtain a second record of the transmission feedback by the client device, wherein the second record at least includes a second transmission identifier indicating the transmission and a quality of experience (QoE) feedback for the transmission; as well as A determination module is configured to determine a QoE evaluation result for the transmission control requirement based on the first record and the second record in response to determining that the first transmission identifier and the second transmission identifier match.

9. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images