Multimedia data processing method, device, client, server, storage medium and program product

Abstract

The embodiment of the present application relates to the field of the Internet, and discloses a method for processing multimedia data. The method includes: in the process of playing the first multimedia data of the target multimedia data to the target object, collecting the operation data of the target object; calling the sensitivity prediction model on the end, predicting the sensitivity of the target object based on the operation data, and obtaining the current sensitivity of the target object; if it is determined to adjust the gear parameters of the multimedia data based on the current sensitivity of the target object, then sending the gear parameter adjustment information to the server, the gear parameter adjustment information includes the target gear parameters that match the current sensitivity of the target object; receiving the second multimedia data whose gear parameters are the target gear parameters sent by the server, and playing the second multimedia data to the target object. The use of the embodiment of the present application can ensure that the played multimedia data meets the needs of the target object.

Description

Technical Field

[0001] The present application relates to the field of Internet, and in particular to a method, device, client, server, storage medium and program product for processing multimedia data. Background Art

[0002] The publisher can publish multimedia data to the server. For example, if the multimedia data is a video, the server directly sends the video published by the publisher to one or more clients, and the client objects can only view the video published by the publisher. However, the above method does not take into account the different needs of different objects for the video. Based on this, how to ensure that the multimedia data sent by the server to the client meets the needs of the client objects is a problem that needs to be solved at present. Summary of the invention

[0003] The embodiments of the present application provide a method, device, client, server, storage medium and program product for processing multimedia data, which can ensure that the played multimedia data meets the needs of the target object.

[0004] On the one hand, an embodiment of the present application provides a method for processing multimedia data, the method comprising:

[0005] In the process of playing the first multimedia data to the target object, collecting the operation data of the target object;

[0006] The sensitivity prediction model on the calling end is used to predict the sensitivity of the target object based on the operation data to obtain the current sensitivity of the target object;

[0007] If it is determined based on the current sensitivity of the target object that the gear parameter of the multimedia data is adjusted, gear parameter adjustment information is sent to the server, where the gear parameter adjustment information includes a target gear parameter that matches the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the first multimedia data;

[0008] receiving second multimedia data sent by the server and having a gear parameter that is the target gear parameter, and playing the second multimedia data to the target object.

[0009] In one embodiment, the method further includes: estimating the activity of the target object based on the operation data to obtain the current activity of the target object;

[0010] If it is determined based on the current sensitivity of the target object that the gear parameter of the multimedia data is adjusted, then gear parameter adjustment information is sent to the server, including:

[0011] If it is determined to adjust the gear parameter of the multimedia data based on the historical activity, current activity and current sensitivity of the target object, the gear parameter adjustment information is sent to the server.

[0012] In one embodiment, the method of determining the manner of adjusting the gear parameter of the multimedia data based on the historical activity, current activity, and current sensitivity of the target object includes:

[0013] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0014] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, and the difference between the historical activity of the target object and the current activity is greater than the preset activity threshold, it is determined to adjust the gear parameter of the multimedia data.

[0015] In one embodiment, the method further includes: before playing the target multimedia data, obtaining a gear parameter of the target multimedia data, wherein the target multimedia unit data refers to any multimedia data;

[0016] If the gear parameter of the target multimedia data is less than a preset parameter threshold, and it is determined based on the target multimedia data and the remaining load to perform super-resolution processing on the target multimedia data, super-resolution processing is performed on the target multimedia data to obtain target multimedia data after super-resolution processing;

[0017] The target multimedia data after the super-resolution processing is played to the target object.

[0018] In one embodiment, the method of determining the super-resolution processing method for the target multimedia data based on the target multimedia unit data and the remaining load includes:

[0019] estimating a load required for super-resolution processing of the target multimedia data based on data parameters of the target multimedia data;

[0020] If the remaining load is greater than or equal to the estimated load, it is determined to perform super-resolution processing on the target multimedia data.

[0021] In one embodiment, the step of adjusting the gear parameter of the multimedia data based on the current sensitivity of the target object includes:

[0022] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0023] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, it is determined to adjust the gear parameter of the multimedia data.

[0024] On the other hand, an embodiment of the present application provides another method for processing multimedia data, the method comprising:

[0025] receiving gear parameter adjustment information sent by a client, the gear parameter adjustment information including a target gear parameter that matches a current sensitivity of a target object, the current sensitivity of the target object being obtained by predicting the sensitivity of the target object based on operation data of the target object by calling a sensitivity prediction model on the client, the operation data being collected by the client in a process of playing first multimedia data of target multimedia data to the target object, the target gear parameter being different from a gear parameter of the first multimedia data;

[0026] Acquire second multimedia data whose gear parameter is the target gear parameter;

[0027] The second multimedia data is sent to the client.

[0028] In one embodiment, the method further includes: calling a cloud sensitivity prediction model to perform sensitivity prediction on the target object based on the object information and historical operation data of the target object to obtain the sensitivity of the target object;

[0029] The first multimedia data is sent to the client, wherein a gear parameter of the first multimedia data matches the sensitivity of the target object.

[0030] In one embodiment, the method further includes: receiving the current sensitivity of the target object and the operation data sent by the client, wherein the current sensitivity of the target object and the operation data are sent by the client after determining to adjust the gear parameter of the target multimedia data;

[0031] Based on the current sensitivity of the target object, the operation data and the object information of the target object, the cloud-based sensitivity prediction model is optimized to obtain an optimized cloud-based sensitivity prediction model.

[0032] In one embodiment, it also includes: sending the optimized cloud-side sensitivity prediction model to the client, so that the client updates the client's on-end sensitivity prediction model based on the optimized cloud-side sensitivity prediction model to obtain an updated on-end sensitivity prediction model.

[0033] On the other hand, an embodiment of the present application provides a multimedia data processing device, the multimedia data processing device comprising:

[0034] An input unit, used to collect operation data of the target object during the process of playing the first multimedia data to the target object;

[0035] A processing unit, configured to call a sensitivity prediction model on the terminal, perform sensitivity prediction on the target object based on the operation data, and obtain a current sensitivity of the target object;

[0036] an output unit, configured to send gear parameter adjustment information to a server if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, the gear parameter adjustment information including a target gear parameter that matches the current sensitivity of the target object, the target gear parameter being different from the gear parameter of the first multimedia data;

[0037] The input unit is further used to receive the second multimedia data sent by the server, the gear parameter of which is the target gear parameter;

[0038] The output unit is further used to play the second multimedia data to the target object.

[0039] On the other hand, an embodiment of the present application provides another multimedia data processing device, the multimedia data processing device comprising:

[0040] a receiving unit, configured to receive gear parameter adjustment information sent by a client, the gear parameter adjustment information including a target gear parameter that matches a current sensitivity of a target object, the current sensitivity of the target object being obtained by predicting the sensitivity of the target object based on operation data of the target object by calling a sensitivity prediction model on the client, the operation data being collected by the client in a process of playing first multimedia data to the target object, and the target gear parameter being different from a gear parameter of the first multimedia data;

[0041] A processing unit, configured to obtain second multimedia data whose gear parameter is the target gear parameter;

[0042] A sending unit is used to send the second multimedia data to the client.

[0043] On the other hand, an embodiment of the present application provides a client, including a processor, a storage device, and a communication interface, wherein the processor, the storage device, and the communication interface are connected to each other, wherein the storage device is used to store a computer program that supports the terminal to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the following steps:

[0044] In the process of playing the first multimedia data to the target object, collecting the operation data of the target object;

[0045] The sensitivity prediction model on the calling end is used to predict the sensitivity of the target object based on the operation data to obtain the current sensitivity of the target object;

[0046] If it is determined based on the current sensitivity of the target object that the gear parameter of the multimedia data is adjusted, gear parameter adjustment information is sent to the server, where the gear parameter adjustment information includes a target gear parameter that matches the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the first multimedia data;

[0047] receiving second multimedia data sent by the server and having a gear parameter that is the target gear parameter, and playing the second multimedia data to the target object.

[0048] On the other hand, an embodiment of the present application provides a server, including a processor, a storage device, and a communication interface, wherein the processor, the storage device, and the communication interface are connected to each other, wherein the storage device is used to store a computer program that supports a terminal to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the following steps:

[0049] receiving gear parameter adjustment information sent by a client, the gear parameter adjustment information including a target gear parameter that matches a current sensitivity of a target object, the current sensitivity of the target object being obtained by predicting the sensitivity of the target object based on operation data of the target object by calling a sensitivity prediction model on the client, the operation data being collected by the client in a process of playing first multimedia data to the target object, and the target gear parameter being different from a gear parameter of the first multimedia data;

[0050] Acquire second multimedia data whose gear parameter is the target gear parameter;

[0051] The second multimedia data is sent to the client.

[0052] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, enable the processor to execute the multimedia data processing method described in any of the above aspects.

[0053] On the other hand, an embodiment of the present application provides a computer product, which includes a computer program, and the computer program is suitable for being loaded by a processor and executing the method for processing multimedia data described in any one of the above aspects.

[0054] In the embodiment of the present application, in the process of playing the first multimedia data to the target object, the operation data of the target object is collected, the sensitivity prediction model on the call end is called, the sensitivity prediction of the target object is performed based on the operation data, and the current sensitivity of the target object is obtained. If the gear parameter of the multimedia data is adjusted based on the current sensitivity of the target object, the gear parameter adjustment information is sent to the server, and the gear parameter adjustment information includes a target gear parameter matching the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the first multimedia data. The second multimedia data with the gear parameter as the target gear parameter sent by the server is received, and the second multimedia data is played to the target object. For example, different objects have different sensitivities, or the sensitivity of the same object in different scenes is different, then the object has different requirements for multimedia data. For example, assuming that the current sensitivity of the target object is low based on the prediction of the operation data of the target object, then the multimedia data with the gear parameter of the lower gear can meet the needs of the target object. Therefore, playing the multimedia data with the gear parameter of the lower gear to the target object can improve the playback fluency of the media data while meeting the needs of the target object, and save the bandwidth cost of the server and reduce the traffic of the client. Assuming that the current sensitivity of the target object is higher based on the prediction of the target object's operation data, the multimedia data of the gear parameters of the higher gear can meet the needs of the target object, so the multimedia data of the gear parameters of the higher gear is played to the target object. Therefore, the current sensitivity of the target object is predicted based on the operation data, and then the multimedia data of the gear parameters matching the current sensitivity is played to the target object, which can ensure that the played multimedia data meets the needs of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a flow chart of a communication system provided by an embodiment of the present application;

[0057] Figure 2 It is a flowchart of a method for processing multimedia data provided by an embodiment of the present application;

[0058] Figure 3 is a flowchart of another method for processing multimedia data provided by an embodiment of the present application;

[0059] Figure 4is a structural schematic diagram of a multimedia data processing device provided in an embodiment of the present application;

[0060] Figure 5 It is a schematic diagram of the structure of a client provided in an embodiment of the present application;

[0061] Figure 6 is a structural schematic diagram of another multimedia data processing device provided in an embodiment of the present application;

[0062] Figure 7 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] The higher the data quality of multimedia data, the better viewing experience it can bring to users. However, multimedia data with higher data quality requires more storage space and higher bandwidth cost under the same encoding algorithm, which will consume more traffic for users and easily cause more jams during viewing. Studies have shown that different users have different requirements for data quality. For example, some users are not sensitive to the clarity of multimedia data. In addition, the same user has different requirements for data quality in different scenarios. For example, when users watch multimedia data under a 4G network, they can reduce the clarity to bring a smoother viewing experience, which can offset the damage to the experience caused by the clarity. That is, users have lower requirements for data quality under a 4G network, that is, users have higher requirements for real-time performance than data quality under a 4G network. However, when users watch multimedia data under a wifi network, they have higher requirements for data quality, that is, users have lower requirements for real-time performance than data quality under a wifi network. Based on this, a technical solution of an embodiment of the present application is proposed. The current sensitivity of the target object is predicted based on the operation data, and then multimedia data with gear parameters matching the current sensitivity is played to the target object, which can ensure that the played multimedia data meets the requirements of the target object.

[0065] The multimedia data processing method provided in the embodiment of the present application can be applied to Figure 1In the communication system shown, the communication system may include a server 101 and at least one client 102, and the server 101 establishes a communication connection with any client 102 through any communication mode, so that the server 101 exchanges data with the client 102 through the communication connection. Specifically, the client in the embodiment of the present application may be any client that establishes a communication connection with the server 101, and the client collects the operation data of the target object in the process of playing the first multimedia data to the target object; the sensitivity prediction model on the call end is used to predict the sensitivity of the target object based on the operation data to obtain the current sensitivity of the target object; if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, the gear parameter adjustment information is sent to the server 101, and the gear parameter adjustment information includes the target gear parameter that matches the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the first multimedia data; the second multimedia data with the gear parameter being the target gear parameter sent by the server 101 is received, and the second multimedia data is played to the target object.

[0066] In an exemplary scenario, if the server has a need to push target multimedia data to the client, the server can send first multimedia data to the client. In the process of playing the first multimedia data of the target multimedia data to the target object, the client collects the operation data of the target object, calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, obtains the current sensitivity of the target object, and if it is determined to adjust the gear parameters of the multimedia data based on the current sensitivity of the target object, then the gear parameter adjustment information is sent to the server, the gear parameters sent by the server are received as the second multimedia data of the target gear parameters, and the second multimedia data is played to the target object.

[0067] In another exemplary scenario, the client can display the playback interface of the first multimedia data. If the client obtains the playback instruction of the target object for the first multimedia data, the client can generate a download request for the first multimedia data and send the download request to the server. The server can respond to the download request and send the first multimedia data to the client. In the process of playing the first multimedia data to the target object, the client collects the operation data of the target object, calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, and obtains the current sensitivity of the target object. If it is determined to adjust the gear parameters of the multimedia data based on the current sensitivity of the target object, the gear parameter adjustment information is sent to the server, the second multimedia data with the gear parameters as the target gear parameters sent by the server is received, and the second multimedia data is played to the target object.

[0068] Among them, the playback instruction for the first multimedia data can be generated by the client when the display time of the playback interface reaches a preset time threshold. Optionally, it can also be generated by the client in response to the target object's playback operation on the first multimedia data. The playback operation can, for example, be the target object clicking the play control in the playback interface, or the target object inputting a voice to instruct playback (voice such as "start playing"), etc., which is not limited to the embodiments of the present application.

[0069] To facilitate understanding, the relevant terms involved in the embodiments of the present application are explained and illustrated below.

[0070] Multimedia data: Multimedia data refers to certain multimedia data or any multimedia data, that is, multimedia data that the server needs to send to the client. It can be multimedia data pushed by the server to the client, or multimedia data that the client requests the server to send. Multimedia data refers to data presented in the form of graphics, images, sounds, texts, animations or videos. For example, the server is a video server, and the client that establishes a communication connection with the server is a video playback client. The target multimedia data may refer to the video data uploaded to the server by the publisher, that is, the video data that the server wants to send to the client. For another example, the server is an audio server, and the client that establishes a communication connection with the server is an audio playback client. The target multimedia data may refer to the audio data uploaded to the server by the publisher, that is, the audio data that the server wants to send to the client.

[0071] In one embodiment, the first multimedia data and the second multimedia data can be two independent multimedia data. For example, the first multimedia data is the multimedia data currently being played on the display screen, and the second multimedia data is the next multimedia data played to the target object by the client in response to a swipe-up operation of the target object.

[0072] In another embodiment, the first multimedia data and the second multimedia data may be partial data in the target multimedia data, that is, the first multimedia data may be the first multimedia unit data in the target multimedia data, and the second multimedia data may be the second multimedia unit data in the target multimedia data. Taking the target multimedia data as a video as an example, assuming that the duration of the target multimedia data is 30 seconds (s), and the duration of each multimedia unit data is 5s, the target multimedia data includes 6 multimedia unit data. Among them, the multiple multimedia unit data included in the target multimedia data are continuous in the time dimension, for example, the first multimedia unit data is adjacent to the second multimedia unit data in the multiple multimedia unit data, and the first multimedia unit data is located before the second multimedia unit data, that is, the end time of the first multimedia unit data in the target multimedia data is the start time of the second multimedia unit data.

[0073] Gear parameters: Gear parameters refer to parameters such as resolution, frame rate, bit rate or clarity, where the data quality of each multimedia data can be reflected by parameters such as resolution, frame rate, bit rate or clarity. For example, the server can adjust the resolution of any multimedia data to obtain multimedia data with multiple resolution gears. For example, the resolution of a certain multimedia data uploaded by the publisher to the server is 480P (the letter P represents line-by-line scanning, and the number 480 represents vertical resolution, that is, there are 480 horizontal scanning lines in the vertical direction). The server can adjust the resolution of the multimedia data to obtain multimedia data with four resolution gear parameters of 1080P, 720P, 480P and 270P, where the difference between the above four multimedia data is only the resolution, and the data content of the above four multimedia data is the same as the data content of the multimedia data with a resolution of 480P. Optionally, for any multimedia unit data, the server can adjust the resolution of the multimedia unit data to obtain multimedia unit data with four resolution level parameters of 1080P, 720P, 480P and 270P, wherein the difference between the above four multimedia unit data is only the resolution, and the data content of the above four multimedia unit data is the same as the data content of any of the above multimedia data.

[0074] Operational data: Operational data refers to information used to predict object needs. Object needs refer to the needs of the object to watch or listen to multimedia data. For example, the object hopes to know the data content of the multimedia data while the multimedia data is played smoothly. For example, the object hopes that the data quality of the played multimedia data is high, etc.

[0075] Optionally, the operation data may be used to characterize the behavior of the object when watching or listening to multimedia data, and the operation data may include one or more of the following: whether the object performs operations such as like, comment, favorite, follow, cache, appreciate, send gift or share, whether the object performs operations such as cancel like, cancel favorite, cancel follow or cancel appreciate, and the operation behavior generated during the process of watching at least one multimedia data within a preset time period, such as like, comment, favorite, follow, cache, appreciate, send gift or share. Optionally, the operation data may also include one or more of the following: the running time of the client, the time spent watching multimedia data during the running process of the client, the average consumption of multimedia data in each running process, the viewing time of each multimedia data, the operation frequency, etc. The operation frequency may include the frequency of like, comment, favorite, follow, cache, appreciate, send gift or share, etc.

[0076] In the specific implementation of this application, the objects involved may refer to users, and the user-related data involved, such as object information, etc. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with local laws, regulations and standards.

[0077] It can be understood that the embodiment of the present application introduces the processing scheme of multimedia data mentioned in the embodiment of the present application by taking the scenario in which the multimedia data is video data as an example, and it does not limit the embodiment of the present application. The processing scheme of multimedia data mentioned in the embodiment of the present application can also be applied to the scenario of playing other multimedia data, such as audio data or text data, etc. The embodiment of the present application does not limit this.

[0078] Combination Figure 1 The communication system shown is shown in Figure 2 , Figure 2 is a flowchart of a method for processing multimedia data provided by an embodiment of the present application; Figure 2 The multimedia data processing scheme shown includes but is not limited to steps S201 to S206, wherein:

[0079] S201, the client collects operation data of the target object while playing first multimedia data to the target object.

[0080] In a specific implementation, the client may obtain the operation data of the target object during the process of playing the first multimedia data through an input device of a terminal device running the client. The input device may exemplarily include a touch panel, a microphone, or a button.

[0081] In the case where the first multimedia data is the first multimedia unit data in the target multimedia data, the first multimedia unit data played by the client to the target object may be any multimedia unit data in the target multimedia data.

[0082] In one example, the first multimedia unit data may refer to the first multimedia unit data in the target multimedia data, that is, the start time of the first multimedia unit data is the start time of the target multimedia data. The server may perform sensitivity prediction on the target object based on the object information and historical operation data of the target object to obtain the sensitivity of the target object. Then, the server may obtain the first multimedia unit data of the gear parameter matching the sensitivity of the target object, and send the first multimedia unit data to the client, so that the client plays the first multimedia unit data to the target object.

[0083] In another example, the first multimedia unit data may refer to the nth multimedia unit data in the target multimedia data, where n is greater than 1, that is, the start time of the first multimedia unit data is different from the start time of the target multimedia data. In the process of playing the previous multimedia unit data of the first multimedia unit data to the target object, the client collects the operation data of the target object, calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, and obtains the current sensitivity of the target object. If it is determined to adjust the gear parameter of the target multimedia data based on the current sensitivity of the target object, the client sends gear parameter adjustment information to the server, and the gear parameter adjustment information includes a target gear parameter that matches the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the previous multimedia unit data. Then, the client receives the first multimedia unit data whose gear parameter is the target gear parameter sent by the server, and plays the first multimedia unit data to the target object.

[0084] In another example, the first multimedia unit data may refer to the nth multimedia unit data in the target multimedia data, where n is greater than 1, that is, the start time of the first multimedia unit data is different from the start time of the target multimedia data. In the process of playing the previous multimedia unit data of the first multimedia unit data to the target object, the client collects the operation data of the target object, calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, and obtains the current sensitivity of the target object. If it is determined not to adjust the gear parameters of the target multimedia data based on the current sensitivity of the target object, the client sends the gear parameter retention information to the server. Then, the client receives the first multimedia unit data sent by the server, and the gear parameters of the first multimedia unit data are the same as the gear parameters of the previous multimedia unit data, and then the client plays the first multimedia unit data to the target object.

[0085] S202, the client calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, and obtains the current sensitivity of the target object.

[0086] For example, assuming that during the process of playing the first multimedia data to the target object, the target object's dwelling time on a single multimedia data is significantly shortened, or the target object generates behaviors such as unfollowing or uncollecting, then the client can determine that the target object has a high quality requirement for multimedia data, that is, the current sensitivity of the target object is high. Assuming that during the process of playing the first multimedia data to the target object, the gear parameter of the first multimedia data is small, but the target object's dwelling time on a single multimedia data remains unchanged, shortens slightly, or increases significantly, or the target object generates behaviors such as following or collecting, then the client can determine that the target object has a low quality requirement for multimedia data, that is, the current sensitivity of the target object is low.

[0087] In an embodiment of the present application, the on-end sensitivity prediction model is deployed in the client. Compared with deploying the sensitivity prediction model on the server, the server calls the sensitivity prediction model to perform sensitivity prediction based on the operation data sent by the client. The embodiment of the present application can reduce the sensitivity prediction delay, thereby improving the processing efficiency of multimedia data.

[0088] In one embodiment, the on-end sensitivity prediction model may be sent by the server to the client. For example, after the target object registers a client, the client establishes a communication connection with the server, the server may send the sensitivity prediction model to the client, and the client may use the sensitivity prediction model as the on-end sensitivity prediction model and store the on-end sensitivity prediction model.

[0089] In another embodiment, the server or the client may pre-build a sensitivity prediction model, and then use the operation data of the client's target object when viewing multimedia data in a historical time period and the gear parameters of the multimedia data viewed by the target object as samples to train the sensitivity prediction model to obtain a trained sensitivity prediction model, so that the gear parameters determined by the sensitivity predicted based on the trained sensitivity prediction model meet the needs of the client's target object.

[0090] S203, if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, the client sends gear parameter adjustment information to the server, the gear parameter adjustment information includes target gear parameters matching the current sensitivity of the target object, and the target gear parameters are different from the gear parameters of the first multimedia data.

[0091] In one implementation, the client can obtain the gear parameter corresponding to the current sensitivity of the target object according to the preset correspondence between the sensitivity and the gear parameter, and if the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, it is determined to adjust the gear parameter of the multimedia data. If the gear parameter corresponding to the current sensitivity of the target object is the same as the gear parameter of the first multimedia data, it is determined not to adjust the gear parameter of the multimedia data.

[0092] In one example, the server can pre-establish a correspondence between sensitivity and gear parameters, and send each sensitivity and its corresponding gear parameter to the client, which stores each sensitivity and its corresponding gear parameter. After obtaining the current sensitivity of the target object, the client can search for the gear parameter corresponding to the current sensitivity of the target object.

[0093] In another example, the client can pre-establish a correspondence between sensitivity and gear parameters, and after obtaining the current sensitivity of the target object, can search for the gear parameters corresponding to the current sensitivity of the target object. Optionally, the correspondence between sensitivity and gear parameters can be set by the target object or by the client.

[0094] In one implementation, the client can estimate the activity of the target object based on the operation data to obtain the current activity of the target object. If it is determined to adjust the gear parameters of the multimedia data based on the historical activity, current activity and current sensitivity of the target object, the gear parameter adjustment information is sent to the server.

[0095] Specifically, if the difference between the current activity and the historical activity of the target object is large, it indicates that the activity of the target object may be affected by the adjustment of the gear parameters. Therefore, the client can determine whether to adjust the gear parameters of the multimedia data based on the historical activity and current activity of the target object, as well as the current sensitivity. If the gear parameters of the multimedia data are adjusted, the gear parameter adjustment information is sent to the server. If the gear parameters of the multimedia data are not adjusted, the gear parameter retention information is sent to the server. Optionally, if the gear parameters of the multimedia data are not adjusted, the client may not send the gear parameter adjustment information to the server. If the server does not receive the gear parameter adjustment information sent by the client within a preset time, it is determined that the client does not adjust the gear parameters of the multimedia data. Then, the server can send the second multimedia data with the same gear parameters as the first multimedia data to the client.

[0096] In one implementation, the client can obtain the gear parameter corresponding to the current sensitivity of the target object based on the correspondence between the preset sensitivity and the gear parameter. If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, and the difference between the historical activity and the current activity of the target object is greater than the preset activity threshold, it is determined to adjust the gear parameter of the multimedia data.

[0097] In one implementation, the client can obtain the gear parameter corresponding to the current sensitivity of the target object based on the correspondence between the preset sensitivity and the gear parameter. If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, or the difference between the historical activity and the current activity of the target object is greater than the preset activity threshold, it is determined to adjust the gear parameter of the multimedia data.

[0098] S204: The server obtains second multimedia data whose gear parameters are target gear parameters.

[0099] In one example, the server may pre-generate second multimedia data with different gear parameters, and after receiving the gear parameter adjustment information sent by the client, search for second multimedia data with gear parameters being the target gear parameters.

[0100] In another example, after receiving the gear parameter adjustment information sent by the client, the server may adjust the gear parameters of the second multimedia data in the memory to obtain the second multimedia data whose gear parameters are the target gear parameters.

[0101] S205: The server sends second multimedia data whose gear parameters are target gear parameters to the client.

[0102] S206: The client plays the second multimedia unit data to the target object.

[0103] Before playing the target multimedia data, obtaining the gear parameter of the target multimedia data, wherein the target multimedia unit data refers to any multimedia data;

[0104] In one implementation, if the gear parameter of the target multimedia data is less than a preset parameter threshold, and it is determined based on the target multimedia data and the remaining load to perform super-resolution processing on the target multimedia data, the client performs super-resolution processing on the target multimedia data to obtain the target multimedia data after super-resolution processing, and then plays the target multimedia data after super-resolution processing to the target object. The target multimedia data refers to any multimedia data, such as the first multimedia data or the second multimedia data.

[0105] Among them, super-resolution is to improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction.

[0106] In this embodiment, when the gear parameter of the multimedia data is less than the preset parameter threshold, the client performs super-resolution processing on the multimedia data locally using a super-resolution algorithm, which can improve the data quality of the multimedia data while ensuring the smoothness of multimedia data playback.

[0107] In one implementation, the client may estimate the load required for super-resolution processing of the target multimedia data based on data parameters of the target multimedia data, and determine to perform super-resolution processing on the target multimedia data if the remaining load is greater than or equal to the estimated load.

[0108] The load may include one or more of the following: CPU resources, GPU resources, memory resources, or input / output (I / O) resources, etc. Since the super-resolution algorithm requires a certain amount of memory and computing power, the load required for super-resolution processing of the multimedia data can be estimated first, and then it is determined whether the remaining load of the terminal device running the client is sufficient. If sufficient, the super-resolution function on the terminal is enabled.

[0109] In an embodiment of the present application, the client predicts the current sensitivity of the target object based on the operation data, and then plays multimedia data with gear parameters matching the current sensitivity to the target object, thereby ensuring that the played multimedia data meets the needs of the target object.

[0110] Combination Figure 1 The communication system shown is shown in Figure 3 , Figure 3 is a flowchart of another method for processing multimedia data provided by an embodiment of the present application; Figure 3 The multimedia data processing scheme shown includes but is not limited to steps S301 to S313, wherein:

[0111] S301, the server obtains object information and historical operation data of the first client.

[0112] In a specific implementation, the server can collect data by embedding points on the client. After obtaining the raw data, the server processes the features into standard input feature data required by the cloud sensitivity prediction model through analysis, data cleaning, sampling, feature extraction, feature selection and other steps through feature engineering on the end. Exemplarily, feature analysis can be performed through the Pearson correlation coefficient method in the feature engineering of the embodiment of the present application. Exemplarily, the specific process of data cleaning can be to delete dirty data such as negative viewing time or integer overflow.

[0113] The object information may include gender, age, network status of the terminal device running the first client, device information of the terminal device, the region where the terminal device is located, the central processing unit (CPU) usage rate of the terminal device, the graphics processing unit (GPU) usage rate of the terminal device, etc. The network status may include the network type or network speed, etc. The network type may include a wifi network, a 4G network, a 5G network, etc. The device information may include the operating system running on the terminal device, the model of the terminal device, etc.

[0114] The historical operation data refers to the operation data of the first object of the first client during the historical time period, for example, the operation data of the first object during the process of watching multimedia data in the last month.

[0115] S302, the server calls a cloud-based sensitivity prediction model to predict the sensitivity of a first object of the first client based on the object information and historical operation data of the first client to obtain the sensitivity of the first object.

[0116] In an embodiment of the present application, the cloud sensitivity prediction model can be obtained by pre-training the server through training samples. For example, the server can obtain the object information, historical operation data and sensitivity labels of at least one client, and the sensitivity labels are used to indicate the sensitivity of the object of the corresponding client. The server can call the initial cloud sensitivity prediction model, predict the sensitivity of the object of each client based on the object information and historical operation data of each client, obtain the sensitivity of each object, and train the initial cloud sensitivity prediction model in the direction of reducing the difference between the sensitivity of each object and the sensitivity indicated by the sensitivity label of the corresponding client to obtain the cloud sensitivity prediction model.

[0117] In one implementation, the server may estimate the activity of the first object based on the object information and historical operation data of the first client to obtain the activity of the first object.

[0118] Optionally, the activity may be predicted by an activity prediction algorithm or a neural network model. The training process of the neural network model may refer to the training process of the cloud sensitivity prediction model.

[0119] S303: The server sends the cloud sensitivity prediction model to the first client.

[0120] S304: The first client uses the cloud sensitivity prediction model as the on-end sensitivity prediction model and deploys the on-end sensitivity prediction model.

[0121] S305: The server sends first multimedia data to the first client, and the gear parameter of the first multimedia data matches the sensitivity of the first object.

[0122] S306: The first client collects operation data of the first object while playing the first multimedia data to the first object.

[0123] S307: The first client calls the sensitivity prediction model on the terminal to perform sensitivity prediction on the first object based on the operation data to obtain the current sensitivity of the first object.

[0124] S308, if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the first object, the first client sends gear parameter adjustment information to the server, the gear parameter adjustment information includes a target gear parameter matching the current sensitivity of the first object, and the target gear parameter is different from the gear parameter of the first multimedia data.

[0125] For details of steps S306 to S308 in the embodiment of the present application, reference may be made to the description of steps S201 to S203 in the above embodiment, which will not be described in detail in the embodiment of the present application.

[0126] S309: The server sends second multimedia data whose gear parameters are target gear parameters to the first client.

[0127] S310: The first client plays second multimedia data whose gear parameters are target gear parameters to the first object.

[0128] S311, the first client sends the current sensitivity and operation data of the first object to the server.

[0129] S312: The server optimizes the cloud-based sensitivity prediction model based on the current sensitivity and operation data of the first object to obtain an optimized cloud-based sensitivity prediction model.

[0130] In an embodiment of the present application, if the first client determines to adjust the gear parameters of the multimedia data based on the current sensitivity of the first object, then it indicates that the cloud-based sensitivity prediction model predicts the sensitivity of the first object of the first client based on the object information and historical operation data of the first client, and the prediction result is inaccurate. Therefore, the cloud-based sensitivity prediction model can be optimized based on the current sensitivity and operation data of the first object.

[0131] For example, when the gear parameter is lowered, the server sends a low-gear video to the client. After the client plays the video, if the user's current active value obtained based on the real-time collected operation data is lower than the historical active value, it indicates that the user's active value is affected by the downshift, that is, the clear sensitivity predicted by the cloud sensitivity prediction model is not accurate enough, so the cloud sensitivity prediction model needs to be optimized. Specifically, after the first client determines to adjust the gear parameter of the multimedia data based on the current sensitivity of the first object, it can send the real-time collected operation data and the predicted current sensitivity to the server so that the server can optimize the cloud sensitivity prediction model.

[0132] The embodiment of the present application optimizes the cloud-based sensitivity prediction model through operation data and sensitivity fed back by at least one client, thereby improving the accuracy of the cloud-based sensitivity prediction model.

[0133] S313, the server sends the optimized cloud sensitivity prediction model to the second client.

[0134] After the second client is registered, the server can obtain the object information and historical operation data of the second client, call the optimized cloud sensitivity prediction model, and predict the sensitivity of the second object of the second client based on the object information and historical operation data of the second client to obtain the sensitivity of the second object. The server sends the optimized cloud sensitivity prediction model to the second client. The second client uses the optimized cloud sensitivity prediction model as the terminal sensitivity prediction model and deploys the terminal sensitivity prediction model. The server sends the third multimedia data to the second client, and the gear parameter of the third multimedia data matches the sensitivity of the second object. In the process of playing the third multimedia data to the second object, the second client collects the operation data of the second object. The second client calls the terminal sensitivity prediction model, predicts the sensitivity of the second object based on the operation data, and obtains the current sensitivity of the second object. If it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the second object, the second client sends the gear parameter adjustment information to the server, and the gear parameter adjustment information includes the target gear parameter matching the current sensitivity of the second object, and the target gear parameter is different from the gear parameter of the third multimedia data. The server sends the fourth multimedia data whose gear parameter is the target gear parameter to the second client. The second client plays the fourth multimedia data whose gear parameter is the target gear parameter to the second object.

[0135] In the embodiment of the present application, the server predicts the sensitivity of the first object of the first client based on the object information and historical operation data of the first client, obtains the sensitivity of the first object, and sends the first multimedia data with the gear parameters matching the sensitivity of the first object to the first client, so as to mine potential objects with lower sensitivity, and downshift the parameters in the multimedia data consumption process for the objects with lower sensitivity, thereby saving the object traffic and bandwidth costs, and improving the playing smoothness of the multimedia data. In addition, the client can collect the operation data of the object in real time, and update the sensitivity of the object based on the operation data, so as to play the media data with the gear parameters matching the current sensitivity of the object to the object, so as to ensure that the played multimedia data meets the needs of the target object.

[0136] An embodiment of the present application further provides a computer storage medium, in which program instructions are stored. When the program instructions are executed, they are used to implement the corresponding methods described in the above embodiments.

[0137] See also Figure 4 , Figure 4 It is a structural schematic diagram of a multimedia data processing device provided in an embodiment of the present application.

[0138] In one implementation of the multimedia data processing device of the embodiment of the present application, the multimedia data processing device includes the following structure.

[0139] An input unit 401 is used to collect operation data of a target object during the process of playing the first multimedia data to the target object;

[0140] The processing unit 402 is used to call the sensitivity prediction model on the terminal, perform sensitivity prediction on the target object based on the operation data, and obtain the current sensitivity of the target object;

[0141] an output unit 403, configured to send gear parameter adjustment information to a server if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, the gear parameter adjustment information including a target gear parameter that matches the current sensitivity of the target object, the target gear parameter being different from the gear parameter of the first multimedia data;

[0142] The input unit 401 is further used to receive the second multimedia data sent by the server, the gear parameter of which is the target gear parameter;

[0143] The output unit 403 is further configured to play the second multimedia data to the target object.

[0144] In one embodiment, the processing unit 402 is further configured to: estimate the activity of the target object based on the operation data to obtain the current activity of the target object;

[0145] When the output unit 403 determines to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, and sends the gear parameter adjustment information to the server, it is used to:

[0146] If it is determined to adjust the gear parameter of the multimedia data based on the historical activity, current activity and current sensitivity of the target object, the gear parameter adjustment information is sent to the server.

[0147] In one embodiment, when determining the manner of adjusting the gear parameter of the multimedia data based on the historical activity, current activity, and current sensitivity of the target object, the processing unit 402 is configured to:

[0148] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0149] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, and the difference between the historical activity of the target object and the current activity is greater than the preset activity threshold, it is determined to adjust the gear parameter of the multimedia data.

[0150] In one embodiment, the processing unit 402 is further configured to:

[0151] Before playing the target multimedia data, obtaining the gear parameter of the target multimedia data, wherein the target multimedia unit data refers to any multimedia data;

[0152] If the gear parameter of the target multimedia data is less than a preset parameter threshold, and it is determined based on the target multimedia data and the remaining load to perform super-resolution processing on the target multimedia data, super-resolution processing is performed on the target multimedia data to obtain target multimedia data after super-resolution processing;

[0153] The target multimedia data after the super-resolution processing is played to the target object.

[0154] In one embodiment, the processing unit 402 determines a method for performing super-resolution processing on the target multimedia data based on the target multimedia unit data and the remaining load, including:

[0155] estimating a load required for super-resolution processing of the target multimedia data based on data parameters of the target multimedia data;

[0156] If the remaining load is greater than or equal to the estimated load, it is determined to perform super-resolution processing on the target multimedia data.

[0157] In one embodiment, when determining to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, the processing unit 402 is configured to:

[0158] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0159] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, it is determined to adjust the gear parameter of the multimedia data.

[0160] In the embodiment of the present application, the processing unit 402 predicts the current sensitivity of the target object based on the operation data, and then the output unit 403 plays multimedia data with gear parameters matching the current sensitivity to the target object, thereby ensuring that the played multimedia data meets the needs of the target object.

[0161] See also Figure 5 , Figure 51 is a schematic diagram of the structure of a client provided in an embodiment of the present application. The client in the embodiment of the present application includes a power supply module and other structures, and includes a processor 501, a storage device 502 and a communication interface 503. The processor 501, the storage device 502 and the communication interface 503 can exchange data, and the processor 501 implements the corresponding multimedia data processing method.

[0162] The storage device 502 may include a volatile memory, such as a random-access memory (RAM); the storage device 502 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the storage device 502 may also include a combination of the above-mentioned types of memory.

[0163] The processor 501 may be a central processing unit (CPU). The processor 501 may also be a combination of a CPU and a GPU. In the client, multiple CPUs and GPUs may be included as needed to perform corresponding data processing. In one embodiment, the storage device 502 is used to store program instructions. The processor 501 may call program instructions to implement various methods involved in the embodiments of the present application.

[0164] In a first possible implementation, the processor 501 of the client calls the program instructions stored in the storage device 502, and is used to collect the operation data of the target object during the process of playing the first multimedia data to the target object; calls the sensitivity prediction model on the end, predicts the sensitivity of the target object based on the operation data, and obtains the current sensitivity of the target object; if it is determined to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, then sends gear parameter adjustment information to the server, and the gear parameter adjustment information includes a target gear parameter that matches the current sensitivity of the target object, and the target gear parameter is different from the gear parameter of the first multimedia data; receives the second multimedia data sent by the server, whose gear parameter is the target gear parameter; and plays the second multimedia data to the target object.

[0165] In one embodiment, the processor 501 is further configured to: estimate the activity of the target object based on the operation data to obtain the current activity of the target object;

[0166] If the processor 501 determines to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, then when sending gear parameter adjustment information to the server, it is configured to:

[0167] If it is determined to adjust the gear parameter of the multimedia data based on the historical activity, current activity and current sensitivity of the target object, the gear parameter adjustment information is sent to the server.

[0168] In one embodiment, when the processor 501 determines the way to adjust the gear parameter of the multimedia data based on the historical activity, current activity, and current sensitivity of the target object, it is configured to:

[0169] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0170] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, and the difference between the historical activity of the target object and the current activity is greater than the preset activity threshold, it is determined to adjust the gear parameter of the multimedia data.

[0171] In one embodiment, the processor 501 is further configured to:

[0172] Before playing the target multimedia data, obtaining the gear parameter of the target multimedia data, wherein the target multimedia unit data refers to any multimedia data;

[0173] If the gear parameter of the target multimedia data is less than a preset parameter threshold, and it is determined based on the target multimedia data and the remaining load to perform super-resolution processing on the target multimedia data, super-resolution processing is performed on the target multimedia data to obtain target multimedia data after super-resolution processing;

[0174] The target multimedia data after the super-resolution processing is played to the target object.

[0175] In one embodiment, the processor 501 determines a method of performing super-resolution processing on the target multimedia data based on the target multimedia unit data and the remaining load, including:

[0176] estimating a load required for super-resolution processing of the target multimedia data based on data parameters of the target multimedia data;

[0177] If the remaining load is greater than or equal to the estimated load, it is determined to perform super-resolution processing on the target multimedia data.

[0178] In one embodiment, when the processor 501 determines to adjust the gear parameter of the multimedia data based on the current sensitivity of the target object, it is configured to:

[0179] According to a preset correspondence between sensitivity and gear parameters, obtaining a gear parameter corresponding to the current sensitivity of the target object;

[0180] If the gear parameter corresponding to the current sensitivity of the target object is different from the gear parameter of the first multimedia data, it is determined to adjust the gear parameter of the multimedia data.

[0181] In the embodiment of the present application, the processor 501 predicts the current sensitivity of the target object based on the operation data, and then plays multimedia data with gear parameters matching the current sensitivity to the target object through the communication interface 503, thereby ensuring that the played multimedia data meets the needs of the target object.

[0182] See also Figure 6 , Figure 6 It is a structural diagram of another multimedia data processing device provided in an embodiment of the present application.

[0183] In one implementation of the multimedia data processing device of the embodiment of the present application, the multimedia data processing device includes the following structure.

[0184] A receiving unit 601 is used to receive gear parameter adjustment information sent by a client, where the gear parameter adjustment information includes a target gear parameter that matches a current sensitivity of a target object, where the current sensitivity of the target object is obtained by predicting the sensitivity of the target object based on operation data of the target object by calling a sensitivity prediction model on the client, where the operation data is collected by the client in the process of playing first multimedia data to the target object, and the target gear parameter is different from the gear parameter of the first multimedia data;

[0185] The processing unit 602 is used to obtain second multimedia data whose gear parameter is the target gear parameter;

[0186] The sending unit 603 is configured to send the second multimedia data to the client.

[0187] In one embodiment, the processing unit 602 is further configured to call a cloud-based sensitivity prediction model to perform sensitivity prediction on the target object based on the object information and historical operation data of the target object to obtain the sensitivity of the target object;

[0188] The sending unit 603 is further configured to send the first multimedia data to the client, wherein the gear parameter of the first multimedia data matches the sensitivity of the target object.

[0189] In one embodiment, the receiving unit 601 is further configured to receive the current sensitivity of the target object and the operation data sent by the client, where the current sensitivity of the target object and the operation data are sent by the client after determining to adjust the gear parameter of the target multimedia data;

[0190] The processing unit 602 is further used to optimize the cloud-based sensitivity prediction model based on the current sensitivity of the target object, the operation data and the object information of the target object to obtain an optimized cloud-based sensitivity prediction model.

[0191] In one embodiment, the sending unit 603 is further used to send the optimized cloud-side sensitivity prediction model to the client, so that the client updates the client's on-end sensitivity prediction model based on the optimized cloud-side sensitivity prediction model to obtain an updated on-end sensitivity prediction model.

[0192] In the embodiment of the present application, after the receiving unit 601 receives the gear parameter adjustment information sent by the client, the sending unit 603 sends the second multimedia data with the gear parameter as the target gear parameter to the client, which can ensure that the multimedia data played by the client meets the needs of the target object.

[0193] See also Figure 7 , Figure 7 7 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server in the embodiment of the present application includes a power supply module and other structures, and includes a processor 701, a storage device 702, and a communication interface 703. The processor 701, the storage device 702, and the communication interface 703 can exchange data, and the processor 701 implements the corresponding multimedia data processing method.

[0194] The storage device 702 may include a volatile memory, such as a random-access memory (RAM); the storage device 702 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the storage device 702 may also include a combination of the above-mentioned types of memory.

[0195] The processor 701 may be a central processing unit (CPU). The processor 701 may also be a combination of a CPU and a GPU. In the server, multiple CPUs and GPUs may be included as needed to perform corresponding data processing. In one embodiment, the storage device 702 is used to store program instructions. The processor 701 may call program instructions to implement various methods involved in the embodiments of the present application.

[0196] In a first possible implementation, the processor 701 of the server calls the program instructions stored in the storage device 702 to receive gear parameter adjustment information sent by the client, the gear parameter adjustment information including target gear parameters matching the current sensitivity of the target object, the current sensitivity of the target object is obtained by predicting the sensitivity of the target object based on the operation data of the target object using a sensitivity prediction model on the client calling end, the operation data is collected by the client in the process of playing first multimedia data to the target object, and the target gear parameters are different from the gear parameters of the first multimedia data; obtain second multimedia data whose gear parameters are the target gear parameters; and send the second multimedia data to the client.

[0197] In one embodiment, the processor 701 is further configured to call a cloud sensitivity prediction model to perform sensitivity prediction on the target object based on the object information and historical operation data of the target object to obtain the sensitivity of the target object;

[0198] The first multimedia data is sent to the client, wherein a gear parameter of the first multimedia data matches the sensitivity of the target object.

[0199] In one embodiment, the processor 701 is further configured to receive the current sensitivity of the target object and the operation data sent by the client, where the current sensitivity of the target object and the operation data are sent by the client after determining to adjust the gear parameter of the target multimedia data;

[0200] Based on the current sensitivity of the target object, the operation data and the object information of the target object, the cloud-based sensitivity prediction model is optimized to obtain an optimized cloud-based sensitivity prediction model.

[0201] In one embodiment, the processor 701 is also used to send the optimized cloud-side sensitivity prediction model to the client, so that the client updates the client's on-end sensitivity prediction model based on the optimized cloud-side sensitivity prediction model to obtain an updated on-end sensitivity prediction model.

[0202] In the embodiment of the present application, after receiving the gear parameter adjustment information sent by the client, the processor 701 sends the second multimedia data whose gear parameters are the target gear parameters to the client, which can ensure that the multimedia data played by the client meets the needs of the target object.

[0203] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). The computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the blockchain node, etc.

[0204] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of this application are still within the scope of the present invention.

Claims

1. A method for processing multimedia data, characterized in that: The method is applied to a client, and the method comprises: In the process of playing the first multimedia data to the target object, collecting the operation data of the target object; The sensitivity prediction model on the calling end is used to predict the sensitivity of the target object based on the operation data to obtain the current sensitivity of the target object; According to a preset correspondence between sensitivity and gear parameters, a target gear parameter matching the current sensitivity of the target object is obtained; estimating the activity of the target object based on the operation data to obtain the current activity of the target object; If the target gear parameter is different from the gear parameter of the first multimedia data, and the difference between the historical activity of the target object and the current activity is greater than a preset activity threshold, determining to adjust the gear parameter of the multimedia data; Sending gear parameter adjustment information to a server, the gear parameter adjustment information including a target gear parameter that matches the current sensitivity of the target object, the target gear parameter being different from the gear parameter of the first multimedia data; receiving second multimedia data sent by the server and having a gear parameter that is the target gear parameter, and playing the second multimedia data to the target object.

2. The method according to claim 1, characterized in that The method further comprises: Before playing the target multimedia data, obtaining the gear parameter of the target multimedia data, wherein the target multimedia data refers to any multimedia data; If the gear parameter of the target multimedia data is less than a preset parameter threshold, and it is determined based on the target multimedia data and the remaining load to perform super-resolution processing on the target multimedia data, super-resolution processing is performed on the target multimedia data to obtain target multimedia data after super-resolution processing; The target multimedia data after the super-resolution processing is played to the target object.

3. The method according to claim 2, characterized in that The method of determining the super-resolution processing method for the target multimedia data based on the target multimedia data and the residual load includes: estimating a load required for super-resolution processing of the target multimedia data based on data parameters of the target multimedia data; If the remaining load is greater than or equal to the estimated load, it is determined to perform super-resolution processing on the target multimedia data.

4. A method for processing multimedia data, characterized in that: include: Receive gear parameter adjustment information sent by a client, the gear parameter adjustment information including a target gear parameter that matches a current sensitivity of a target object, the current sensitivity of the target object being obtained by predicting the sensitivity of the target object based on operation data of the target object by using a sensitivity prediction model on a calling end of the client, the operation data being collected by the client in the process of playing first multimedia data of target multimedia data to the target object, and the target gear parameter being different from the gear parameter of the first multimedia data; the gear parameter adjustment information is sent when the client determines that the target gear parameter is different from the gear parameter of the first multimedia data, and a difference between a historical activity of the target object and a current activity of the target object is greater than a preset activity threshold, and the current activity of the target object is obtained by estimating the activity of the target object based on the operation data; Acquire second multimedia data whose gear parameter is the target gear parameter; The second multimedia data is sent to the client.

5. The method according to claim 4, characterized in that The method further comprises: Calling a cloud-based sensitivity prediction model to perform sensitivity prediction on the target object based on the object information and historical operation data of the target object to obtain the sensitivity of the target object; The first multimedia data is sent to the client, wherein a gear parameter of the first multimedia data matches the sensitivity of the target object.

6. The method according to claim 5, characterized in that The method further comprises: receiving the current sensitivity of the target object and the operation data sent by the client, wherein the current sensitivity of the target object and the operation data are sent by the client after determining to adjust the gear parameter of the target multimedia data; Based on the current sensitivity of the target object, the operation data and the object information of the target object, the cloud-based sensitivity prediction model is optimized to obtain an optimized cloud-based sensitivity prediction model.

7. The method according to claim 6, characterized in that The method further comprises: The optimized cloud-side sensitivity prediction model is sent to the client, so that the client updates the client's on-end sensitivity prediction model based on the optimized cloud-side sensitivity prediction model to obtain an updated on-end sensitivity prediction model.

8. A multimedia data processing device, characterized in that: The device is applied to a client, and comprises: An input unit, used to collect operation data of the target object during the process of playing the first multimedia data to the target object; A processing unit, configured to call a sensitivity prediction model on the terminal, perform sensitivity prediction on the target object based on the operation data, and obtain a current sensitivity of the target object; obtain a target gear parameter that matches the current sensitivity of the target object according to a preset correspondence between sensitivity and gear parameter; estimate the activity of the target object based on the operation data to obtain the current activity of the target object; if the target gear parameter is different from the gear parameter of the first multimedia data, and the difference between the historical activity of the target object and the current activity is greater than a preset activity threshold, determine to adjust the gear parameter of the multimedia data; an output unit, configured to send gear parameter adjustment information to a server, wherein the gear parameter adjustment information includes a target gear parameter that matches a current sensitivity of the target object, and the target gear parameter is different from a gear parameter of the first multimedia data; The input unit is further used to receive second multimedia data sent by the server, the gear parameter of which is the target gear parameter; The output unit is further used to play the second multimedia data to the target object.

9. A multimedia data processing device, characterized in that: The device comprises: A receiving unit, configured to receive gear parameter adjustment information sent by a client, the gear parameter adjustment information including a target gear parameter that matches a current sensitivity of a target object, the current sensitivity of the target object being obtained by predicting the sensitivity of the target object based on operation data of the target object by calling a sensitivity prediction model on the client, the operation data being collected by the client in the process of playing first multimedia data of target multimedia data to the target object, and the target gear parameter being different from the gear parameter of the first multimedia data; the gear parameter adjustment information is sent when the client determines that the target gear parameter is different from the gear parameter of the first multimedia data, and a difference between the historical activity of the target object and the current activity of the target object is greater than a preset activity threshold, and the current activity of the target object is obtained by estimating the activity of the target object based on the operation data; A processing unit, configured to obtain second multimedia data whose gear parameter is the target gear parameter; A sending unit is used to send the second multimedia data to the client.

10. A client, characterized in that: The client comprises a processor, a storage device and a communication interface, wherein the processor, the storage device and the communication interface are connected to each other, wherein: The storage device is used to store a computer program, wherein the computer program includes program instructions; The processor is used to call the program instructions to execute the multimedia data processing method according to any one of claims 1 to 3.

11. A server, characterized in that: The server includes a processor, a storage device and a communication interface, and the processor, the storage device and the communication interface are connected to each other, wherein: The storage device is used to store a computer program, wherein the computer program includes program instructions; The processor is used to call the program instructions to execute the multimedia data processing method according to any one of claims 4 to 7.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the method for processing multimedia data according to any one of claims 1 to 7 is executed.

13. A computer product, characterized in that: The computer product comprises a computer program, and the computer program is suitable for being loaded by a processor and executing the method for processing multimedia data according to any one of claims 1 to 7.

Images