Multimedia transmission method based on resolution, server, medium and product
By segmented encoding of multimedia content based on the maximum support resolution of the user equipment and the current network bandwidth, and combined with user preferences, the problem of fixed resolution in the existing multimedia transmission methods is solved, intelligent and personalized multimedia transmission is realized, and user experience and network resource utilization efficiency are improved.
Patent Information
- Application Number
- CN202510366983.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-26
- Publication Date
- 2025-06-13
AI Technical Summary
In the existing multimedia transmission methods, the fixed resolution cannot effectively adapt to the changes in the display capability and network bandwidth of the user equipment, resulting in poor picture quality or overload of data transmission.
By determining the segmented resolution of multimedia content based on the maximum support resolution of the user equipment and the current network bandwidth, and encoding it in combination with user preferences, the intelligent and personalized transmission of multimedia content is achieved.
Provide better user experience under limited bandwidth, realize efficient utilization of network resources, meet the personalized needs of different users, and dynamically adjust the resolution when network conditions change to ensure transmission quality.
Smart Images

Figure CN120151565A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data transmission, and in particular, to a multimedia transmission method, server, medium, and product based on resolution. Background Art
[0002] With the rapid development of Internet technology, the online transmission of multimedia content has become an important part of daily life. How to achieve the transmission of multimedia content while providing the best user experience has become an urgent problem to be solved.
[0003] In related technologies, the multimedia transmission method usually encodes multimedia content by the server at a fixed resolution and then transmits it to the user device.
[0004] However, this method has some obvious defects. For example, when the fixed resolution is too low, the picture quality of the multimedia content is blurred, often making it difficult for users to clearly view the multimedia content; when the fixed resolution is high, due to the limitations of the screen size and battery life of the user device, the high-resolution multimedia content increases the burden of data transmission and may also cause the user device to overheat and consume power too quickly. Summary of the Invention
[0005] This application provides a multimedia transmission method, server, medium, and product based on resolution, which is used to improve the quality of multimedia transmission content.
[0006] In a first aspect, this application provides a multimedia transmission method based on resolution, which is applied to a server. The method includes: determining a first resolution according to the maximum supported resolution of the user device; determining a second resolution according to the current network bandwidth and a preset network bandwidth-resolution correspondence table, where the network bandwidth-resolution correspondence table includes the correspondence between network bandwidth intervals and resolutions; determining user preferences according to the user's viewing records within a historical preset duration; determining a first data segment and a second data segment in the multimedia content based on the user preferences, where the first data segment is used to represent the part where the user's interest exceeds a preset interest threshold, and the second data segment is used to represent the part where the user's interest is lower than the preset interest threshold; if the first resolution is less than the second resolution, encoding the first data segment at the second resolution and encoding the second data segment at the first resolution to obtain an encoded multimedia data stream; if the second resolution is less than the first resolution, encoding the first data segment at the first resolution and encoding the second data segment at the second resolution to obtain an encoded multimedia data stream; and transmitting the multimedia data stream to the user device.
[0007] By adopting the above technical solution, the server adjusts the resolution and encodes the multimedia content in segments according to the maximum supported resolution of the user device, the current network bandwidth, and the user preferences, achieving intelligent and personalized transmission of the multimedia content, ensuring that the parts the user is interested in can be presented in a higher resolution, while the parts the user is less interested in are in a lower resolution to save bandwidth. This differential transmission strategy can not only provide a better user experience under limited bandwidth, but also achieve efficient utilization of network resources, and at the same time meet the personalized needs of different users.
[0008] Combined with some embodiments of the first aspect, in some embodiments, determining the user preferences according to the user's viewing records within a historical preset duration specifically includes: establishing a user model according to the user information; determining the videos the user is interested in according to the user's viewing records within the historical preset duration; combining the videos the user is interested in with the user model to obtain a user interest model, and the user preference is included in the user interest model.
[0009] By adopting the above technical solution, first, the server establishes a user model to capture the user's basic characteristics and behavior patterns. Then, based on the user's viewing records within the historical preset duration, the server identifies the videos the user is interested in. Finally, the server combines these videos the user is interested in with the user model to form a more comprehensive and dynamic user interest model. This method can comprehensively and accurately depict the user preferences, not only considering the user's static characteristics, but also incorporating the dynamic viewing behavior, providing a reliable basis for subsequent content segmentation and differential transmission, and thus enabling more accurate resource allocation and transmission optimization.
[0010] Combined with some embodiments of the first aspect, in some embodiments, determining the first data segment and the second data segment in the multimedia content based on the user preference, where the first data segment is used to represent the part where the user interest degree exceeds a preset interest degree threshold, and the second data segment is used to represent the part where the user interest degree is lower than the preset interest degree threshold, specifically includes: determining the content characteristics in the multimedia data; predicting the user interest degree of each data segment in the multimedia data through the user interest model based on the user preference and the content characteristics; marking the data segment whose user interest degree exceeds the preset interest degree threshold as the first data segment, and marking the data segment whose user interest degree is lower than the preset interest degree threshold as the second data segment.
[0011] By adopting the above technical solution, refined analysis and processing of the multimedia content are realized, so as to accurately identify the content segments that the user is most concerned about. By combining the content characteristics in the multimedia data and the user preference for prediction, the accuracy and personalization degree of segmentation are improved.
[0012] In some embodiments in combination with some embodiments of the first aspect, before the step of transmitting the multimedia data stream to the user equipment, the method further includes: determining the transmission protocol of the multimedia data stream according to a preset transmission type - transmission protocol correspondence table and the type of the multimedia data stream; encapsulating the multimedia data stream according to the format characteristics of the transmission protocol.
[0013] By adopting the above technical solution, before transmitting the multimedia data stream, the server performs transmission protocol selection and encapsulation processing, ensuring the efficiency and reliability of data transmission. Different types of multimedia content (such as videos, audios, images, etc.) may require different transmission strategies, and selecting the appropriate transmission protocol can optimize the transmission performance. Encapsulating the multimedia data stream according to the format characteristics of the selected transmission protocol ensures the integrity and security of the multimedia data stream during transmission.
[0014] In some embodiments in combination with some embodiments of the first aspect, the step of transmitting the multimedia data stream to the user equipment specifically includes: detecting the packet loss rate and latency of the multimedia data stream during transmission; if the packet loss rate exceeds a preset packet loss rate threshold or the latency exceeds a preset duration threshold, reducing the first resolution and the second resolution according to a preset resolution adjustment level table and re - encoding the multimedia content.
[0015] By adopting the above technical solution, first, the server monitors the packet loss rate and latency of the multimedia data stream during transmission in real time, so as to be able to detect changes in the network condition in a timely manner for subsequent adjustment. Second, when it is detected that the network condition deteriorates (the packet loss rate exceeds the preset packet loss rate threshold or the latency exceeds the preset duration threshold), the server reduces the first resolution and the second resolution according to the preset resolution adjustment level table and re - encodes the multimedia content. This dynamic adjustment strategy can effectively cope with network fluctuations, ensuring that the basic transmission quality can still be maintained when the network condition deteriorates. By re - encoding the multimedia content, while ensuring the accessibility of the content, the data transmission volume can be reduced, thus alleviating the network pressure and greatly improving the user's viewing experience, especially in the case of unstable network conditions.
[0016] In some embodiments in combination with some embodiments of the first aspect, after the step of transmitting the multimedia data stream to the user equipment, the method further includes: receiving the user's quality feedback on the multimedia data stream; if the quality feedback shows that the user satisfaction is lower than a preset satisfaction threshold, determining a third resolution according to the real - time network bandwidth and a preset network bandwidth - resolution correspondence table; taking the minimum of the first resolution and the third resolution as the adapted resolution of the multimedia content; re - encoding the multimedia content based on the adapted resolution to obtain an optimized multimedia data stream; and transmitting the optimized multimedia data stream to the user equipment.
[0017] By adopting the above technical solution, it is ensured that the transmission quality will not exceed the bearing capacity of the device or network, which can not only improve the user's viewing experience, but also guarantee the quality of the multimedia transmission content.
[0018] In combination with some embodiments of the first aspect, in some embodiments, determining the first resolution according to the maximum supported resolution of the user device specifically includes: obtaining the model information of the user device; determining the maximum supported resolution of the user device according to the model information and the preset device model-maximum supported resolution correspondence table, and determining the maximum supported resolution as the first resolution; if the device model corresponding to the model information does not exist in the device model-maximum supported resolution correspondence table, then determining the default resolution as the first resolution.
[0019] By adopting the above technical solution, first of all, the server obtains the model information of the user device, so as to quickly and accurately identify different terminal devices, providing a basis for subsequent resolution matching. Secondly, the server uses the preset device model-maximum supported resolution correspondence table to efficiently determine the display capabilities of specific devices, avoiding the cumbersome real-time calculation process. Thirdly, when an unknown device model is encountered, the server will use the default resolution as an alternative solution, and this fault tolerance mechanism ensures the continuity and universality of the service. This intelligent device adaptation method greatly improves the accuracy and efficiency of multimedia transmission, providing the most suitable viewing experience for different terminal users.
[0020] In a second aspect, an embodiment of the present application provides a server, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the server to execute the method described in the first aspect and any possible implementation manner in the first aspect.
[0021] In a third aspect, an embodiment of the present application provides a computer program product containing instructions, when the above computer program product runs on a server, enabling the above server to execute the method described in the first aspect and any possible implementation manner in the first aspect.
[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, when the above instructions run on a server, enabling the above server to execute the method described in the first aspect and any possible implementation manner in the first aspect.
[0023] Understandably, the server provided in the second aspect above, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here.
[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By adopting the above technical solution, the server adjusts the resolution and encodes the multimedia content in segments according to the maximum supported resolution of the user device, the current network bandwidth, and the user preference, realizing the intelligent and personalized transmission of the multimedia content, ensuring that the parts that the user is interested in can be presented at a higher resolution, while the parts that the user is less interested in use a lower resolution to save bandwidth. This differential transmission strategy can not only provide a better user experience under limited bandwidth, but also realize the efficient utilization of network resources, and at the same time meet the personalized needs of different users.
[0025] 2. By adopting the above technical solution, the server selects a transmission protocol and performs encapsulation processing before transmitting the multimedia data stream, ensuring the efficiency and reliability of data transmission. Different types of multimedia content (such as video, audio, image, etc.) may require different transmission strategies, and selecting the appropriate transmission protocol can optimize the transmission performance. Encapsulating the multimedia data stream according to the format characteristics of the selected transmission protocol ensures the integrity and security of the multimedia data stream during transmission.
[0026] 3. By adopting the above technical solution, first, the server monitors the packet loss rate and delay of the multimedia data stream in real time during transmission, so as to be able to detect changes in the network condition in a timely manner for subsequent adjustment. Second, when it is detected that the network condition deteriorates (the packet loss rate exceeds the preset packet loss rate threshold or the delay exceeds the preset duration threshold), the server will lower the first resolution and the second resolution according to the preset resolution adjustment level table and re-encode the multimedia content. This dynamic adjustment strategy can effectively cope with network fluctuations, ensuring that the basic transmission quality can still be maintained when the network condition deteriorates. By re-encoding the multimedia content, while ensuring the accessibility of the content, the data transmission volume can be reduced, thereby alleviating the network pressure and greatly improving the user's viewing experience, especially in the case of unstable network conditions. Description of the Drawings
[0027] Figure 1 is a flowchart of a method for multimedia transmission based on resolution in an embodiment of the present application; Figure 2 is another flowchart of a method for multimedia transmission based on resolution in an embodiment of the present application; Figure 3 It is a schematic structural diagram of an entity device of a server in an embodiment of the present application. Detailed implementation manners
[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification of the present application, the singular forms "a", "an", "the above", "the", and "this" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.
[0029] Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.
[0030] The method provided in this embodiment will be described in terms of its process below. Please refer to Figure 1 , which is a schematic flowchart of a multimedia transmission method based on resolution in an embodiment of the present application.
[0031] S101. Determine a first resolution according to the maximum supported resolution of the user device; Among them, the user device refers to a terminal device for receiving and playing multimedia content, such as a smart phone, a tablet computer, a smart TV, etc. The maximum supported resolution represents the highest picture clarity that the user device can display, usually expressed as the number of horizontal pixels multiplied by the number of vertical pixels, such as 1920x1080. The first resolution refers to the transmission resolution determined according to the user device.
[0032] When the user requests to play multimedia content, the server needs to determine the transmission resolution suitable for the user device, that is, the first resolution. Specifically, the server can determine the maximum supported resolution of the user device according to the model information of the user device, or directly obtain it from the user, so as to ensure that the transmitted multimedia content does not exceed the display ability of the user device and avoid unnecessary bandwidth waste.
[0033] S102. Determine a second resolution according to the current network bandwidth and a preset network bandwidth-resolution correspondence table, where the network bandwidth-resolution correspondence table includes the correspondence between network bandwidth intervals and resolutions; Among them, the current network bandwidth refers to the network transmission rate currently available to the user device, usually measured in bits per second (bps). The network bandwidth-resolution refers to a pre-set correspondence table between network bandwidth and recommended transmission resolution. The network bandwidth range represents the network transmission rate within a certain range, such as 0-1 Mbps, 1-5 Mbps, etc. The second resolution refers to the recommended transmission resolution determined according to the current network bandwidth.
[0034] When the user requests to play multimedia content, the server needs to determine the transmission resolution suitable for the current network bandwidth, that is, the second resolution. Specifically, first, the server can obtain the current network bandwidth through network monitoring software. Then, the server queries the pre-set network bandwidth-resolution correspondence table, finds the network bandwidth range corresponding to the current network bandwidth, and obtains the transmission resolution recommended for this network bandwidth range as the second resolution. For example, if the current network bandwidth is 3 Mbps and the transmission resolution recommended for the 1-5 Mbps range in the network bandwidth-resolution correspondence table is 720p, then the server will set 720p as the second resolution.
[0035] S103. Determine user preferences based on the user's viewing records within a historical preset duration; Among them, the historical preset duration refers to a fixed time period used to analyze user behavior, such as the past 30 days, 60 days, etc. The viewing records include information such as the type, duration, and frequency of the multimedia content the user has watched. User preferences represent the degree of interest and preference tendency of the user for different multimedia contents.
[0036] When the user requests to play multimedia content, the server needs to analyze the user's personal preferences to achieve personalized transmission. Specifically, the server will retrieve the user's viewing records within a historical preset duration (such as the past 30 days). These viewing records may include the type of multimedia content the user has watched, the viewing duration and frequency of each type, the user's ratings or interaction behaviors, etc. The server will use data analysis algorithms, such as collaborative filtering or content-based filtering, to process these viewing records to determine user preferences.
[0037] S104. Determine the first data segment and the second data segment in the multimedia content based on the user preferences. The first data segment is used to represent the part where the user's interest degree exceeds the preset interest degree threshold, and the second data segment is used to represent the part where the user's interest degree is lower than the preset interest degree threshold; Among them, the multimedia content refers to digital media resources such as videos and audios that need to be transmitted. The first data segment represents the content part with a relatively high interest degree of the user. The second data segment represents the content part with a relatively low interest degree of the user. The preset interest degree threshold refers to the standard dividing line for distinguishing high and low interest degrees. The user's interest degree refers to the predicted degree of interest of the user in a specific content segment.
[0038] After obtaining the user preferences, the server needs to perform intelligent segmentation on the multimedia content to be transmitted. Specifically, first, the server analyzes the characteristics of the multimedia content, such as the scene, theme, characters, etc. of the video. Then, the server matches these characteristics with the user preferences and calculates a predicted user interest level for each segment of the multimedia content. Next, the server compares these user interest levels with a preset interest threshold. Segments with a user interest level exceeding the preset interest threshold are marked as the first data segments, indicating that the user may be highly interested; segments with a user interest level lower than the preset interest threshold are marked as the second data segments, indicating that the user may be less interested. This segmentation method lays the foundation for subsequent differential transmission, ensuring the transmission quality of the content that the user is most interested in under limited network resources.
[0039] S105. If the first resolution is less than the second resolution, encode the first data segment at the second resolution and encode the second data segment at the first resolution to obtain an encoded multimedia data stream; Among them, encoding refers to the process of converting the original multimedia content into a digital signal of a specific format and quality. The multimedia data stream represents the continuous digital signal after encoding processing and is used for network transmission.
[0040] After completing the segmentation of the multimedia content, the server needs to perform differential encoding on the multimedia content according to the previously determined first resolution, second resolution, and user interest level. Specifically, when the first resolution (the maximum supported resolution of the user device) is less than the second resolution (the recommended transmission resolution under the current network bandwidth), the server will adopt a more flexible encoding strategy. For the first data segment (the part with high user interest), the server will encode it at the second resolution (higher resolution) to ensure that the content that the user is most interested in can be presented in the best quality. For the second data segment (the part with low user interest), the server will encode it at the first resolution (lower resolution), which can save bandwidth resources while ensuring the basic viewing experience. Through this differential encoding method, the server can maximize the use of available network resources under the limited display capabilities of the user device and provide a personalized viewing experience for the user.
[0041] S106. If the second resolution is less than the first resolution, encode the first data segment at the first resolution and encode the second data segment at the second resolution to obtain an encoded multimedia data stream; Specifically, when the second resolution (the recommended transmission resolution under the current network bandwidth) is less than the first resolution (the maximum supported resolution of the user device), the server will adopt a more flexible encoding strategy. For the first data segment (the part with high user interest), the server will encode it using the first resolution (higher resolution) to ensure the display quality of important content. For the second data segment (the part with low user interest), the server will encode it using the second resolution (lower resolution) to adapt to the current network conditions. This encoding strategy can, under the limited current network bandwidth, prioritize ensuring the quality of the content that the user is most interested in, while ensuring the smoothness of the overall transmission by reducing the resolution of secondary content.
[0042] S107. Transmit the multimedia data stream to the user device.
[0043] After completing the differential encoding of the multimedia content, the server needs to send the processed multimedia data stream to the user. Specifically, the server can establish a communication connection with the user device and send the multimedia data stream from the server to the user device through the network.
[0044] By adopting the above technical solution, the server adjusts the resolution and encodes the multimedia content in a segmented manner according to the maximum supported resolution of the user device, the current network bandwidth, and user preferences, realizing the intelligent and personalized transmission of multimedia content, ensuring that the parts that the user is interested in can be presented at a higher resolution, while the parts that the user is less interested in use a lower resolution to save bandwidth. This differential transmission strategy can not only provide a better user experience under limited bandwidth, but also realize the efficient utilization of network resources, and at the same time meet the personalized needs of different users.
[0045] The following further describes the more specific process of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of the multimedia transmission method based on resolution in the embodiment of the present application.
[0046] S201. Obtain the model information of the user device; When the user requests to play multimedia content, the server needs to determine the model information of the user device. The model information represents the specific model and version of the user device, usually including the manufacturer, product series, and specific model, etc. Specifically, the server can obtain the model information of the user device through various methods. A common method is through the User-Agent field in the HTTP request header. This field usually contains information such as the operating system, browser version, and device model of the user device. Another method is for the client application to actively report the model information of the user device. This method can provide more detailed and accurate device parameters. Other implementation methods are not limited here.
[0047] S202. Determine the maximum supported resolution of the user device according to the model information and the pre-set device model - maximum supported resolution correspondence table, and determine this maximum supported resolution as the first resolution; Among them, the device model - maximum supported resolution correspondence table refers to a pre-established database or list that contains various device models and their corresponding maximum supported resolutions. The maximum supported resolution represents the highest picture clarity that the device can display, usually expressed as the number of horizontal pixels multiplied by the number of vertical pixels, such as 1920x1080 or 3840x2160.
[0048] After obtaining the model information of the user device, the server needs to determine the display capability of the user device to prevent the transmitted multimedia content from exceeding the display capability of the user device. Specifically, first, the server queries the pre-established device model - maximum supported resolution correspondence table. The device model - maximum supported resolution correspondence table is usually compiled by the service provider based on the parameter information of common devices in the market and is updated regularly to include newly released device models. The server will use the obtained model information of the user device to match in this device model - maximum supported resolution correspondence table to find the corresponding maximum supported resolution. For example, if the user device is model A, the device model - maximum supported resolution correspondence table may show that its maximum supported resolution is 2532x1170. The server will set this queried maximum supported resolution as the first resolution as a reference basis for subsequent content encoding and transmission.
[0049] S203. If the device model corresponding to the model information does not exist in the device model - maximum supported resolution correspondence table, determine the default resolution as the first resolution; Among them, the default resolution refers to a pre-set safe and generally applicable resolution value, such as 1280x720 (720p), when the specific maximum supported resolution corresponding to the device model of the user device cannot be determined. The non-existence of the device model corresponding to the model information means that the server cannot find the matching model information in the pre-set device model - maximum supported resolution correspondence table, which may be because the user device is too new or too old, or is an unconventional device.
[0050] In some cases, the server may not be able to find the model information of the user device in the preset device model - maximum supported resolution correspondence table. In this case, an alternative solution is needed to determine the appropriate transmission resolution. Specifically, when the server fails to find in the device model - maximum supported resolution correspondence table, it will retrieve the stored default resolution as the first resolution. This default resolution is selected as a compromise value that can be normally displayed on most devices without causing excessive bandwidth waste. Common default resolutions may be 1280x720 (720p) or 1920x1080 (1080p).
[0051] S204. Determine a second resolution according to the current network bandwidth and the preset network bandwidth - resolution correspondence table, where the network bandwidth - resolution correspondence table includes the correspondence between network bandwidth intervals and resolutions; Specifically, reference can be made to step S102, which will not be elaborated here.
[0052] S205. Establish a user model according to the user information; Among them, user information refers to various data related to the user, including but not limited to personal profiles, behavior records, preference settings, etc. The user model is a mathematical or structured representation of user information, used to predict and analyze the interests and needs of the user.
[0053] Before preparing to push personalized multimedia content, the server needs to first construct a user model that can accurately describe user information. Specifically, first, the server collects various user information, including basic profiles such as the user's age, gender, geographical location, as well as behavior data such as the user's viewing history, search records, likes and comments. Then, the server will use data mining and machine learning algorithms, such as collaborative filtering, clustering analysis or neural networks, etc., to process and analyze these data, so as to identify features such as the user's interest patterns, content preferences, viewing habits, etc. Finally, the server will integrate these features into a structured user model, which may be represented as a multi - dimensional vector or a complex data structure.
[0054] S206. Determine the videos that the user is interested in according to the user's viewing records within the preset historical duration; Among them, the preset historical duration refers to a fixed time period used to analyze user behavior, such as the past 30 days, 60 days, etc. The viewing records include detailed information such as the video content viewed by the user, the viewing duration, and the viewing frequency. The videos that the user is interested in refer to the video content that can be inferred from the user's past behavior and that the user may like or be interested in.
[0055] After establishing a preliminary user model, the server needs to further analyze user preferences. Specifically, first, the server determines a suitable historical preset duration, usually a recent period, such as the past 30 days or 90 days. This historical preset duration should be long enough to capture user preferences but not too long to include outdated user interests. Then, the server retrieves all the viewing records of the user during this period, including the viewing duration, completion rate, and number of repeat viewings of each video. The server uses some weighting algorithms, such as giving different weights according to the viewing duration and completion rate, to evaluate the importance of each video to the user. In addition, the server may also consider the user's explicit feedback, such as likes and favorites. Through these analyses, the server can generate a list of videos that the user is interested in. This list may contain specific video IDs or may be a generalization of video categories or topics.
[0056] S207. Combine the videos that the user is interested in with the user model to obtain a user interest model, which includes the user preferences. Among them, the user interest model refers to a more accurate and dynamic representation of user interests formed by comprehensively considering user information and the videos that the user is interested in. User preferences represent the degree of interest and preference tendency of the user for different multimedia contents, usually represented in a quantitative form, such as interest weights or probability distributions.
[0057] After obtaining the videos that the user is interested in, the server needs to combine these specific videos that the user is interested in with the previously established user model to form a more comprehensive and accurate user interest model. Specifically, first, the server analyzes the characteristics of the videos that the user is interested in, such as the theme, type, and style of the videos. Then, the server uses machine learning algorithms, such as feature extraction or topic models, to extract key interest points or topics from these videos. These extracted features are fused with the information in the user model, and methods such as weighted average, matrix factorization, or deep learning may be used to obtain the user interest model. For example, the user interest model may show that the user's interest level in technology videos is 0.8 and the interest level in sports videos is 0.5. This comprehensive user interest model can more accurately reflect the overall user preferences and provide more precise guidance for subsequent personalized transmission.
[0058] S208. Determine the content characteristics in the multimedia data. Before preparing to personalize multimedia content, the server needs to deeply analyze the content features in the multimedia data. Among them, the content features represent the key attributes and characteristics in the multimedia data, including but not limited to the theme, type, style, sentiment tendency, visual elements, etc. Specifically, first, the server will preprocess the multimedia data, such as video frame segmentation, audio segmentation, etc. Then, the server will use various content analysis techniques, such as computer vision algorithms, natural language processing, audio analysis, etc., to extract content features. For video content, it may analyze the objects, scenes, actions, etc. in the picture; for audio content, it may analyze features such as speech, music, and sentiment. In addition, the server may also utilize metadata information, such as titles, descriptions, tags, etc., to supplement the content features. The content features may be represented in the form of vectors, tag sets, or structured data.
[0059] S209. Based on the user preference and the content feature, predict the user interest degree of each data segment in the multimedia data through the user interest model; Among them, the data segment refers to a time segment or content unit in the multimedia content. The user interest degree refers to the predicted interest degree of the user in a specific content segment.
[0060] After obtaining the user preference and the content feature, the server needs to evaluate the interest degree of the user in each part of the multimedia content. Specifically, first, the server will divide the multimedia content into several data segments, and these data segments may be divided based on time (such as every 30 seconds) or based on the content structure (such as scene changes). Then, the server will use the previously established user interest model, combined with the content features of each data segment, to calculate the user interest degree of each data segment. This calculation process may involve various machine learning algorithms, such as collaborative filtering, content-based filtering, or deep learning models, etc. For example, if the user model shows that the user is interested in the technology theme, and the content features of a certain data segment show that it contains a large amount of technology-related content, then the predicted user interest degree of this data segment may be relatively high. The server will generate a user interest degree for each data segment.
[0061] S210. Mark the data segments with the user interest degree exceeding the preset interest degree threshold as the first data segments, and mark the data segments with the user interest degree lower than the preset interest degree threshold as the second data segments; After predicting the user interest level for each data segment, the server needs to classify the multimedia content based on these prediction results for subsequent differential processing. Specifically, first, the server retrieves a preset interest threshold, which may be obtained through data analysis and user research. For example, it may be set to 0.7 (assuming the user interest level ranges from 0 to 1). Then, the server traverses all data segments and compares the user interest level of each data segment with this preset interest threshold. For those data segments whose user interest level exceeds the preset interest threshold, the server marks them as the first data segments, indicating that these are the contents that users may be highly interested in. On the contrary, for those data segments whose user interest level is lower than the preset interest threshold, the server marks them as the second data segments, indicating that these are the contents with relatively low user interest. This marking process may add flags in the database or generate two separate lists of data segments, which is not limited here.
[0062] S211. If the first resolution is less than the second resolution, encode the first data segment at the second resolution and encode the second data segment at the first resolution to obtain an encoded multimedia data stream; Specifically, refer to step S105, which will not be elaborated here.
[0063] S212. If the second resolution is less than the first resolution, encode the first data segment at the first resolution and encode the second data segment at the second resolution to obtain an encoded multimedia data stream; Specifically, refer to step S106, which will not be elaborated here.
[0064] S213. Determine the transmission protocol of the multimedia data stream according to a preset transmission type - transmission protocol correspondence table and the type of the multimedia data stream; Among them, the transmission type - transmission protocol correspondence table refers to a pre - defined reference table that lists different types of multimedia data streams and their corresponding optimal transmission protocols. The type of the multimedia data stream refers to the nature and characteristics of the multimedia data stream, such as video stream, audio stream, real - time streaming media, etc. The transmission protocol refers to the rules and standards for transmitting data over the network, such as HTTP, RTMP, HLS, etc.
[0065] Before preparing to transmit a multimedia data stream, the server needs to select the most suitable transmission protocol to ensure the efficiency and reliability of data transmission. Specifically, first, the server will identify the type of the multimedia data stream to be transmitted, which may involve analyzing characteristics such as the format, encoding method, and real-time requirements of the multimedia data stream. Then, the server will consult the transmission type - transmission protocol correspondence table, which is usually compiled based on a large amount of practical experience and performance test results and contains the correspondence between various types of multimedia data streams and their optimal transmission protocols. For example, for real-time video, the transmission type - transmission protocol correspondence table may recommend using the RTMP protocol; for on-demand video, it may recommend using the HLS or DASH protocol. The server will look up the identified type of multimedia data stream in the transmission type - transmission protocol correspondence table and select the corresponding transmission protocol. If there are multiple optional transmission protocols, the server may further consider factors such as network conditions and device compatibility to make a final choice.
[0066] S214. Encapsulate the multimedia data stream according to the format characteristics of the transmission protocol; Among them, the format characteristics refer to the data organization and structure methods required by a certain transmission protocol. Encapsulation processing means the process of reorganizing and packing the original multimedia data stream according to the requirements of the selected transmission protocol.
[0067] After determining the suitable transmission protocol, the server needs to adjust the multimedia data stream to conform to the format characteristics required by the transmission protocol. Specifically, first, the server will analyze the specific specifications of the selected transmission protocol, including the structure of data packets, the format of header information, the data segmentation method, etc. Then, the server will use specialized encapsulation tools or libraries to reorganize the original multimedia data stream. This process may include adding the header information required by the transmission protocol, splitting the multimedia data stream into appropriately sized packets, adding timestamps and synchronization information, etc. For example, if the selected protocol is the HLS protocol, the server will split the multimedia data stream into several short-time segments and generate the corresponding playlist file. If it is the RTMP protocol, the multimedia data stream needs to be encapsulated into the RTMP message format. The encapsulation process may also involve additional processing such as encryption and compression to meet the requirements of security and efficiency. Through this encapsulation processing, the server ensures that the multimedia data stream can be correctly recognized and transmitted by the selected transmission protocol, preparing for subsequent network transmission.
[0068] S215. Detect the packet loss rate and latency of the multimedia data stream during transmission; Among them, the packet loss rate refers to the ratio of the number of lost data packets to the total number of sent data packets during transmission. Latency represents the time required for the multimedia data stream to travel from the sender to the receiver.
[0069] After starting the transmission of the multimedia data stream, the server needs to continuously monitor the transmission quality in order to adjust the transmission strategy in a timely manner. Specifically, the server will use network monitoring tools or built-in monitoring modules to track the transmission of data packets. For the detection of the packet loss rate, the server may add sequence numbers to the data packets and calculate the number of lost packets based on the feedback from the receiving end. The measurement of latency may involve sending specific probe packets and recording the round-trip time, or analyzing the timestamp information in the multimedia video stream. The server will regularly collect the packet loss rate and latency during the transmission of the multimedia data stream, perhaps taking a sample every few seconds or dozens of seconds.
[0070] S216. If the packet loss rate exceeds the preset packet loss rate threshold or the latency exceeds the preset duration threshold, lower the first resolution and the second resolution according to the preset resolution adjustment level table, and re-encode the multimedia content; Among them, the preset packet loss rate threshold and the preset duration threshold refer to the pre-set acceptable maximum packet loss rate and latency values. The preset resolution adjustment level table refers to a pre-defined reference standard for adjusting the resolution of multimedia content according to the first resolution and the second resolution when the network conditions deteriorate. Lowering the resolution means reducing the clarity of the multimedia video stream to reduce the data volume. Re-encoding refers to the process of re-compressing and processing the multimedia content using the new resolution.
[0071] When the server detects a decline in the network transmission quality, it needs to adjust the transmission strategy in a timely manner to ensure the user experience. Specifically, first, the server will compare the detected packet loss rate and latency with the preset packet loss rate threshold and the preset duration threshold. The preset packet loss rate threshold and the preset duration threshold are usually determined based on user experience research. For example, the preset packet loss rate threshold may be set to 5%, and the preset duration threshold may be set to 500 milliseconds. If the packet loss rate exceeds the preset packet loss rate threshold or the latency exceeds the preset duration threshold, the server will initiate the adjustment process. The server will consult the preset resolution adjustment level table, which usually contains the recommended resolution levels under different network conditions. For example, it may be reduced from 1080p to 720p, or from 720p to 480p. The server will adjust the resolutions of both the first data segment (the part with high user interest) and the second data segment (the part with low user interest) simultaneously, but may use a relatively higher resolution for the first data segment. Then, the server will re-encode the multimedia content using the new resolution. This process may involve adjusting the bit rate, changing the encoder settings, etc.
[0072] S217. Receive the quality feedback of the user on the multimedia data stream; While continuously transmitting a multimedia data stream, the server also needs to collect the quality feedback of the user on the multimedia data stream. Among them, the quality feedback refers to the evaluation and opinions of the user on the quality of the received multimedia data stream. Specifically, the server may obtain the quality feedback of the user through various channels. A common way is to provide a simple scoring mechanism on the playback interface, such as a five-star rating or a slider, so that the user can directly rate the quality of the multimedia data stream. Another way is to analyze the implicit feedback of the user, such as the viewing duration, whether there is fast-forwarding or repeated viewing, etc. The server may also provide a text feedback entry, allowing the user to describe in detail the problems they encounter.
[0073] S218. If the quality feedback shows that the user satisfaction is lower than the preset satisfaction threshold, then determine the third resolution according to the real-time network bandwidth and the preset network bandwidth-resolution correspondence table; Among them, the preset satisfaction threshold refers to the pre-set acceptable minimum user satisfaction level. The real-time network bandwidth refers to the data transmission capacity of the current network connection. The network bandwidth-resolution refers to the pre-set correspondence table between the network bandwidth and the recommended transmission resolution. The network bandwidth range represents a certain range of network transmission rates, such as 0-1 Mbps, 1-5 Mbps, etc. The third resolution refers to the recommended transmission resolution determined according to the real-time network bandwidth.
[0074] Specifically, reference can be made to step S102, which will not be elaborated here.
[0075] S219. Take the smaller one of the first resolution and the third resolution as the adaptation resolution of the multimedia content; Among them, the adaptation resolution refers to the finally selected resolution for re-encoding the multimedia content. The smaller resolution refers to the one with the fewest pixels among the compared first resolution and the third resolution.
[0076] After obtaining the third resolution, the server needs to make a final resolution selection. Specifically, the server will compare the previously determined first resolution (the maximum supported resolution of the user device) and the just-determined third resolution (based on the network condition and user feedback). The comparison method is usually to directly compare the pixel numbers of the two, for example, 1920x1080 and 1280x720. The server will select the smaller one of these two resolutions as the final adaptation resolution. Selecting the smaller resolution is to ensure smooth transmission and playback under the current network conditions and user device limitations. For example, if the first resolution is 1080p and the third resolution is 720p, then the server will select 720p as the adaptation resolution.
[0077] S220. Re-encode the multimedia content based on the adaptation resolution to obtain an optimized multimedia data stream; Among them, re-encoding refers to the process of re-compressing and processing multimedia content using an adapted resolution. Optimizing the multimedia data stream representation refers to the data stream that has been adjusted and re-processed and is more suitable for the current transmission conditions.
[0078] After determining the adapted resolution, the server needs to re-process the multimedia content to adapt to the new transmission conditions. Specifically, the server will start an encoding engine, which may be a multimedia processing tool like FFmpeg. The re-encoding process first adjusts the original multimedia content to the selected adapted resolution. In addition to changing the resolution, the server may also adjust other encoding parameters, such as bit rate, frame rate, encoder preset, etc. The re-encoding process may also include processing of the audio stream, such as adjusting the audio bit rate or switching to a more efficient audio coding format. The server will perform such processing on the entire multimedia content to obtain a new optimized multimedia data stream.
[0079] S221. Transmit the optimized multimedia data stream to the user device.
[0080] Specifically, reference can be made to step S107, which will not be elaborated here.
[0081] By adopting the above technical solution, before transmitting the multimedia data stream, the server performs transmission protocol selection and encapsulation processing, ensuring the efficiency and reliability of data transmission. Different types of multimedia content (such as videos, audios, images, etc.) may require different transmission strategies, and selecting an appropriate transmission protocol can optimize the transmission performance. Encapsulating the multimedia data stream according to the format characteristics of the selected transmission protocol ensures the integrity and security of the multimedia data stream during transmission.
[0082] By adopting the above technical solution, first, the server real-time monitors the packet loss rate and latency of the multimedia data stream during transmission, so as to be able to promptly detect changes in the network condition for subsequent adjustment. Second, when it detects that the network condition deteriorates (the packet loss rate exceeds the preset packet loss rate threshold or the latency exceeds the preset duration threshold), the server will lower the first resolution and the second resolution according to the preset resolution adjustment level table and re-encode the multimedia content. This dynamic adjustment strategy can effectively cope with network fluctuations, ensuring that the basic transmission quality can still be maintained when the network condition deteriorates. By re-encoding the multimedia content, while ensuring the accessibility of the content, the data transmission volume can be reduced, thereby alleviating the network pressure and greatly enhancing the user's viewing experience, especially in the case of unstable network conditions.
[0083] The server in the embodiment of the present invention application will be described from the perspective of hardware processing. Please refer to Figure 3 , which is a schematic structural diagram of an entity device of the server in the embodiment of the present application.
[0084] It should be noted that Figure 3 The structure of the server shown is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present invention.
[0085] As Figure 3 shown, the server includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage section 308 into the random access memory (RAM) 303, such as executing the method described in the above embodiments. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0086] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD), an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read from it can be installed into the storage section 308 as needed.
[0087] Specifically, according to the embodiments of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are executed.
[0088] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings.
[0090] Specifically, the server in this embodiment includes a processor and a memory, and a computer program is stored on the memory. When the computer program is executed by the processor, the multimedia transmission method based on resolution provided in the above embodiment is implemented.
[0091] On the other hand, the present invention also provides a computer-readable storage medium, which may be included in the server described in the above embodiment; or it may exist separately and not be assembled into the server. The above storage medium carries one or more computer programs. When the above one or more computer programs are executed by a processor of the server, the server implements the multimedia transmission method based on resolution provided in the above embodiment.
[0092] As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.
[0093] As used in the foregoing embodiments, depending on the context, the term "when" may be construed to mean "if", "after", "in response to determining", or "in response to detecting". Similarly, depending on the context, the phrase "upon determining" or "if (the stated condition or event) is detected" may be construed to mean "if determined", "in response to determining", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".
[0094] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by a computer program instructing relevant hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the foregoing method embodiments. The foregoing storage medium includes various media that can store program codes, such as ROM, random access memory (RAM), magnetic disks, or optical discs.
Claims
1. A resolution-based multimedia transmission method, characterized in that: Applied to a server, the method comprises: Determining a first resolution according to a maximum supported resolution of a user device; Determining the second resolution according to the current network bandwidth and a preset network bandwidth-resolution correspondence table, wherein the network bandwidth-resolution includes a correspondence between a network bandwidth interval and a resolution; Determine user preferences based on the user's viewing history within a preset time period; Determine a first data segment and a second data segment in the multimedia content based on the user preference, the first data segment is used to represent a portion where the user's interest exceeds a preset interest threshold, and the second data segment is used to represent a portion where the user's interest is lower than the preset interest threshold; If the first resolution is smaller than the second resolution, encoding the first data segment at the second resolution, and encoding the second data segment at the first resolution to obtain an encoded multimedia data stream; If the second resolution is smaller than the first resolution, encoding the first data segment at the first resolution and encoding the second data segment at the second resolution to obtain an encoded multimedia data stream; The multimedia data stream is transmitted to the user equipment.
2. The method according to claim 1, characterized in that Determining the user preference according to the user's viewing history within a preset time period specifically includes: Establish a user model based on user information; Determine the video that the user is interested in based on the user's viewing history within a preset time period; The user-interested video is combined with the user model to obtain a user-interested model, wherein the user-interested model includes the user preference.
3. The method according to claim 2, characterized in that The determining of a first data segment and a second data segment in the multimedia content based on the user preference, wherein the first data segment is used to represent a portion where the user's interest exceeds a preset interest threshold, and the second data segment is used to represent a portion where the user's interest is lower than the preset interest threshold, specifically includes: Determining content characteristics in the multimedia data; Based on the user preference and the content feature, predicting the user interest level of each data segment in the multimedia data by using the user interest model; The data segment whose user interest exceeds a preset interest threshold is marked as the first data segment, and the data segment whose user interest is lower than the preset interest threshold is marked as the second data segment.
4. The method according to claim 1, characterized in that: Before the step of transmitting the multimedia data stream to the user equipment, the method further includes: Determining the transmission protocol of the multimedia data stream according to a preset transmission type-transmission protocol correspondence table and the type of the multimedia data stream; The multimedia data stream is encapsulated according to the format characteristics of the transmission protocol.
5. The method according to claim 1, characterized in that The transmitting the multimedia data stream to the user equipment specifically includes: Detecting the packet loss rate and delay of the multimedia data stream during transmission; If the packet loss rate exceeds a preset packet loss rate threshold or the delay exceeds a preset duration threshold, the first resolution and the second resolution are lowered according to a preset resolution adjustment level table, and the multimedia content is re-encoded.
6. The method according to claim 1, characterized in that After the step of transmitting the multimedia data stream to the user equipment, the method further comprises: receiving quality feedback from the user on the multimedia data stream; If the quality feedback shows that the user satisfaction is lower than a preset satisfaction threshold, determining a third resolution according to the real-time network bandwidth and a preset network bandwidth-resolution correspondence table; Using the smallest resolution between the first resolution and the third resolution as the adaptation resolution of the multimedia content; Re-encoding the multimedia content based on the adapted resolution to obtain an optimized multimedia data stream; The optimized multimedia data stream is transmitted to the user equipment.
7. The method according to claim 1, characterized in that The determining the first resolution according to the maximum supported resolution of the user equipment specifically includes: Obtaining model information of the user equipment; Determine the maximum supported resolution of the user device according to the model information and a preset device model-maximum supported resolution correspondence table, and determine the maximum supported resolution as the first resolution; If the device model corresponding to the model information does not exist in the device model-maximum supported resolution correspondence table, the default resolution is determined to be the first resolution.
8. A server, characterized in that: The server includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the server to execute the method described in any one of claims 1-7.
9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a server, the server is caused to execute the method according to any one of claims 1 to 7.
10. A computer program product, characterized in that When the computer program product is run on a server, the server is caused to execute the method according to any one of claims 1 to 7.
