Video processing method, electronic device, storage medium and program product

Abstract

The present disclosure relates to a video processing method, an electronic device, a storage medium and a program product. The method comprises: determining that a target object performs a non-private viewing behavior based on behavior data of the target object for a to-be-played video; dividing a plurality of playback devices into master and slave playback devices according to display-related information of the plurality of playback devices associated with the non-private viewing behavior; determining a first transcoding parameter of a master stream based on display-related information of the slave playback device, network information of the master playback device, and to-be-played video resource information; determining self-transcoding possibility information of the slave playback device according to a difference between a visual parameter in the display-related information and a visual parameter in the first transcoding parameter, in combination with a device intelligence level of the slave playback device; and sending the self-transcoding possibility information to the corresponding slave playback device, for determining whether to convert the master stream into an auxiliary stream. The present disclosure can effectively realize adaptive video stream transmission.

Description

Technical Field

[0001] This disclosure relates to the fields of video processing, cloud computing and big data edge computing, and in particular to a video processing method, electronic device, storage medium and program product. Background Technology

[0002] With the rapid development of mobile internet technology and the widespread adoption of smart terminal devices, users' demand for cross-device, multi-screen audio-visual experiences is growing. Video casting technology has emerged to meet this need, becoming a key bridge connecting small-screen smart terminals (such as smartphones and tablets) with large-screen display devices (such as smart TVs and projectors). It aims to seamlessly and efficiently transmit video content from the former to the latter for playback, thereby achieving a better viewing experience. Summary of the Invention

[0003] This disclosure provides a video processing method, electronic device, storage medium, and program product that can effectively realize the transmission of adaptive video streams.

[0004] In a first aspect, embodiments of this disclosure provide a video processing method, including:

[0005] Based on the behavioral data of the target object towards the video to be played, it is determined that the target object performs non-isolated viewing behavior, which includes multi-screen sharing or screen casting behavior.

[0006] Based on the display-related information of the multiple playback devices associated with the non-exclusive viewing behavior, the multiple playback devices are divided into primary playback devices and secondary playback devices;

[0007] Based on the display-related information of the playback device, the network information of the main playback device, and the resource information of the video to be played, the first transcoding parameters of the main bitstream are determined;

[0008] The video to be played is transcoded based on the first transcoding parameters to obtain the main bitstream, and the main bitstream is sent to the main playback device and the slave playback device;

[0009] Based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, and in conjunction with the device intelligence level of the playback device, the possibility information of the playback device's automatic transcoding is determined.

[0010] The self-transcoding probability information is sent to the corresponding slave playback device, and the self-transcoding probability information is used to determine whether the corresponding slave playback device will convert the main bitstream into an auxiliary bitstream.

[0011] Secondly, embodiments of this disclosure provide a video processing method, the method being executed by a playback device, the method comprising:

[0012] The main bitstream is obtained by transcoding the video to be played based on a first transcoding parameter, which is determined based on the display-related information of the secondary playback device, the network information of the primary playback device, and the resource information of the video to be played.

[0013] The self-transcoding probability information of the playback device is obtained; the self-transcoding probability information is determined based on the difference between the visual parameters in the display-related information of the playback device and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device, wherein the playback device is any one of the playback devices, and the playback device is determined by dividing the multiple playback devices according to the display-related information of multiple playback devices associated with the target object when performing non-isolated viewing behavior;

[0014] If the self-transcoding probability information does not meet the self-transcoding conditions, the main bitstream is used for playback; or, if the self-transcoding probability information meets the self-transcoding conditions, the bitstream conversion matrix of the playback device is obtained, and the main bitstream is converted using the bitstream conversion matrix to obtain the auxiliary bitstream of the playback device.

[0015] Thirdly, embodiments of this disclosure provide a video processing apparatus, including:

[0016] The first determining module is used to determine, based on the target object’s behavioral data toward the video to be played, that the target object performs a non-isolated viewing behavior, which includes multi-screen sharing or screen casting behavior.

[0017] The partitioning module is used to partition the multiple playback devices into a main playback device and a secondary playback device based on display-related information of the multiple playback devices associated with the non-exclusive viewing behavior.

[0018] The second determining module is used to determine the first transcoding parameters of the main bitstream based on the display-related information of the playback device, the network information of the main playback device, and the resource information of the video to be played.

[0019] The transcoding processing module is used to transcode the video to be played based on the first transcoding parameters to obtain the main bitstream, and send the main bitstream to the main playback device and the slave playback device;

[0020] The third determining module is used to determine the possibility information of automatic transcoding by the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device.

[0021] The sending module is used to send the self-transcoding possibility information to the corresponding slave playback device. The self-transcoding possibility information is used to determine whether the corresponding slave playback device will convert the main bitstream into an auxiliary bitstream.

[0022] Fourthly, embodiments of this disclosure provide a video processing apparatus, the apparatus being configured on a playback device, the apparatus comprising:

[0023] The first acquisition module is used to acquire the main bitstream, which is obtained by transcoding the video to be played based on the first transcoding parameters. The first transcoding parameters are determined based on the display-related information of the secondary playback device, the network information of the primary playback device, and the resource information of the video to be played.

[0024] The second acquisition module is used to acquire the self-transcoding probability information of the playback device; the self-transcoding probability information is determined based on the difference between the visual parameters in the display-related information of the playback device and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device, wherein the playback device is any one of the playback devices, and the playback device is determined by dividing the multiple playback devices according to the display-related information of multiple playback devices associated with the target object when performing non-isolated viewing behavior;

[0025] The playback module is configured to play the main bitstream when the self-transcoding probability information does not meet the self-transcoding conditions; or, when the self-transcoding probability information meets the self-transcoding conditions, to obtain the bitstream conversion matrix of the playback device and use the bitstream conversion matrix to convert the main bitstream to obtain the auxiliary bitstream of the playback device.

[0026] Fifthly, embodiments of this disclosure provide a video processing system, including:

[0027] The cloud storage device is configured to perform the method described in the first aspect above;

[0028] The playback device is configured to perform the method described in the second aspect above.

[0029] Sixthly, embodiments of this disclosure provide an electronic device, including:

[0030] One or more processors;

[0031] The processor is configured to invoke instructions to cause the electronic device to perform the method described in the first or second aspect.

[0032] In a seventh aspect, embodiments of this disclosure provide a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first or second aspect.

[0033] Eighthly, embodiments of this disclosure provide a program product including at least one of a program and instructions, wherein when the program or instructions are executed by an electronic device, they implement the steps of the methods described in the first or second aspect.

[0034] According to the technical solution disclosed herein, by providing an auxiliary bitstream to the playback device, only one transcoding is required from the backend server. During this process, the transcoding of the main bitstream not only considers the main playback device but also comprehensively considers the optimal transcoding parameters of all playback devices. Furthermore, by calculating the optimal transcoding parameters of the main bitstream during screen casting and sharing, the playback device automatically optimizes the main bitstream based on its own transcoding probability information to obtain an optimized auxiliary bitstream. This improves the playback effect of the playback device with minimal additional server computing power consumption. Through flexible and intelligent management, buffering time and stuttering are reduced, user viewing satisfaction is improved, and adaptive video stream transmission can be effectively achieved.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0037] Figure 1 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0038] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0039] Figure 3 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0040] Figure 4 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0041] Figure 5 This is a block diagram of a video processing apparatus according to an exemplary embodiment.

[0042] Figure 6 This is a block diagram of a video processing apparatus according to an exemplary embodiment.

[0043] Figure 7 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0044] Figure 8 This is an example diagram illustrating one-to-one screen mirroring behavior according to an exemplary embodiment.

[0045] Figure 9 This is an example diagram illustrating one-to-many screen mirroring behavior according to an exemplary embodiment.

[0046] Figure 10 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0048] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0049] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0050] It should be noted that the acquisition, transmission, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0051] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0052] It is worth noting that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solutions disclosed herein. However, this does not mean that the applicant has used or necessarily used such solutions.

[0053] The video processing method, electronic device, storage medium, and program product of this disclosure are described below with reference to the accompanying drawings.

[0054] Figure 1 This is a flowchart illustrating a video processing method according to an exemplary embodiment. It should be noted that the execution entity of the video processing method in this embodiment can be a video processing device, which can be implemented by software and / or hardware, and can be configured in an electronic device. For example, the electronic device can be a cloud storage device. Figure 1 As shown, the video processing method may include, but is not limited to, the following steps.

[0055] In step 101, the non-isolated behavior of the target object is determined based on the target object's behavior data in relation to the video to be played.

[0056] In the embodiments of this disclosure, the non-isolated viewing behavior may include, but is not limited to, multi-screen sharing or screen casting. For example, the cloud storage device's video playback device can capture the behavioral data of the target object (such as a user) in real time. In one possible implementation, a multi-screen sharing and screen casting function detection capability is added, activated when video playback begins, and continuously monitored during video resource playback to determine whether the screen casting function, internal capture card, and external capture card of the video playback device are enabled. By detecting the above, it can be determined whether the user is simultaneously playing the video through multiple terminal devices during the current video playback process. When more than one device is used for simultaneous playback and display, it is considered non-isolated viewing behavior; similarly, when only one device is used for playback and display, it is considered isolated viewing behavior.

[0057] In some embodiments, the video to be played can be understood as a video to be played on an electronic device. For example, if the electronic device is a cloud storage device, then the video to be played can be understood as a video to be played on the cloud storage device.

[0058] In step 102, based on the display-related information of the multiple playback devices associated with the non-isolated viewing behavior, the multiple playback devices are divided into main playback devices and slave playback devices.

[0059] In the embodiments of this disclosure, when it is determined that the target object performs a non-isolated viewing behavior based on the target object's behavior data toward the video to be played, display-related information of multiple playback devices associated with the non-isolated viewing behavior can be obtained, and the multiple playback devices can be divided into main playback devices and slave playback devices according to the display-related information.

[0060] In some embodiments, the multiple playback devices associated with the aforementioned non-exclusive viewing behavior may refer to all playback devices involved when the target object (such as a user) triggers multi-screen sharing or casting behavior, including but not limited to cloud disk video playback devices, as well as the target playback devices involved in the multi-screen sharing or casting behavior (for example, if a user casts a video from a cloud disk device to terminal A, then terminal A (or the playback device of terminal A, such as the player of terminal A) and the video playback device of the cloud disk device are both considered as playback devices associated with the exclusive viewing behavior).

[0061] In some embodiments, auxiliary transcoding capability information of each of the multiple playback devices can be determined based on display-related information of the multiple playback devices associated with non-isolated viewing behavior; based on the auxiliary transcoding capability information, the playback device with the strongest auxiliary transcoding capability among the multiple playback devices is designated as the master playback device, and the devices other than the master playback device among the multiple playback devices are designated as slave playback devices.

[0062] In some embodiments, the display-related information includes at least the resolution, color space information, frame rate, intelligent attributes (such as whether it is an intelligent device), and computing power information of the playback device. As one possible implementation, the formula for calculating the auxiliary transcoding capability information can be expressed as follows:

[0063]

[0064] Among them, C t,i For time t, the auxiliary transcoding capability information of the i-th playback device; IA i For the intelligent attributes of the i-th playback device; CP i This refers to the computing power information of the i-th playback device; DPI i =(W i H i Let W be the resolution of the i-th playback device. i H is the width in pixels. i RF (Radio Frequency) is the number of pixels at height. i CR is the frame rate of the i-th playback device; mThe minimum supported color gamut of the color space used by the video resource to be played; G is a constant used to compress and expand the value range to ensure consistency of the value range, for example, G can be 10. 4 ;CR i W represents the color space information of the i-th playback device. a H represents the target resolution width in the target resolution of the video to be played after transcoding; a This represents the target resolution height within the target resolution range after the video is transcoded.

[0065] For example, the auxiliary transcoding capability information C of each of the multiple playback devices can be calculated using the above formula (1) based on the display-related information of the multiple playback devices associated with the non-isolated viewing behavior. i Take the auxiliary transcoding capability information C from the multiple playback devices. i The most powerful device acts as the primary playback device, while other playback devices are designated as secondary playback devices. The primary playback device can play the main stream and push it to the secondary playback devices.

[0066] In step 103, the first transcoding parameters of the main bitstream are determined based on the display-related information from the playback device, the network information of the main playback device, and the resource information of the video to be played.

[0067] In some embodiments, the first transcoding parameters of the main bitstream may include, but are not limited to, the bitrate of the main bitstream, the frame rate of the main bitstream, the color space information of the main bitstream, and the resolution of the main bitstream.

[0068] In some embodiments, the bitrate of the main stream can be determined based on resolution, frame rate, device intelligence level, network information of the main playback device, and resource information of the video to be played, all derived from display-related information of the playback device. The device intelligence level may include, but is not limited to, one or more of the following: intelligence attributes; computing power information; and whether it is an operation execution device. The operation execution device can be understood as a playback device that interacts with a target object (such as a user) during multi-screen sharing or screen casting. For example, taking a device intelligence level that includes intelligence attributes, the formula for calculating the bitrate of the main stream can be expressed as follows:

[0069]

[0070] Among them, BR m The bitrate of the main stream; N f The frame rate is the value in the resource information of the video to be played; W p The resolution width is specified in the resource information of the video to be played; H p The resolution height is the value in the resource information of the video to be played; P zThe compression rate is the resource information of the video to be played; the network information N of the main playback device. in =(N in,s N in,j N in,l ), N in,s N represents the download bandwidth of the user network (the network of the main playback device). in,j N represents the jitter of the user network (the network of the main playback device). in,l Indicates latency in the user network (main playback device network); RF i W is the frame rate of the i-th playback device; i H represents the resolution width (number of pixels) of the i-th playback device. i Let n be the resolution height (number of pixels) of the i-th playback device; d -1 represents the total number of playback devices, n d P represents the number of the aforementioned playback devices; as Q is the system overhead ratio parameter; ua The acceptable level of image quality for the target audience (user) in the target audience (user) profile (which can be determined based on the target audience's viewing feedback on the played video); δ is a system parameter used to avoid consuming all resources and causing other applications to lag, for example, δ can be 0.7; IA i The IA represents the intelligent attribute of the i-th playback device, indicating whether the i-th playback device is an intelligent device. If the i-th playback device is an intelligent device, then the IA... i The value can be the first value (such as 1). If the i-th playback device is not a smart device, then the IA... i The value can be the second value (such as 0).

[0071] In some embodiments, the frame rate of the main stream is determined based on the maximum supported frame rate of the current playback device, the frame rate from the display-related information of the playback device, the target resolution of the video to be played after transcoding, and the bit rate from the second transcoding parameters of the unique bitstream of the playback device under the exclusive viewing behavior; the second transcoding parameters are determined based on the network information of the current playback device, the behavior information of the target object, and the resource information of the video to be played.

[0072] In one possible implementation, the bitrate in the second transcoding parameters of the unique bitstream under the exclusive viewing behavior of the playback device can be determined based on the network information of the current playback device (e.g., the playback device itself), the behavioral information of the target object, and the resource information of the video to be played. The behavioral information of the target object may include the target object's (user's) acceptance of image quality in the target object (user) profile. As an example, the formula for calculating the bitrate in the second transcoding parameters of the unique bitstream under the exclusive viewing behavior can be expressed as follows:

[0073]

[0074] Among them, BR a The bitrate in the second transcoding parameter of the unique bitstream of the playback device (e.g., the slave device) under exclusive viewing behavior; the network information N of the current playback device (e.g., the slave device). u =(N u,s N u,j N u,l ), N u,s N represents the download bandwidth of the user's network. u,j N represents the jitter of the user's network. u,l This indicates the latency of the user's network.

[0075] For example, based on the network information of the current playback device, the behavior information of the target object, and the resource information of the video to be played, the bitrate in the second transcoding parameter of the unique bitstream from the playback device under the exclusive viewing behavior can be calculated using the above formula (3). Then, using the maximum supported frame rate of the current playback device, the frame rate in the display-related information of the playback device, the target resolution of the video to be played after transcoding, and the bitrate in the second transcoding parameter, the frame rate of the main bitstream can be calculated using the following formula (4). As an example, the formula for calculating the frame rate of the main bitstream can be expressed as follows:

[0076]

[0077] Among them, RF m The frame rate of the main bitstream; RF u The maximum frame rate supported by the current playback device; W a H represents the target resolution width of the video to be played after transcoding; a The target resolution height of the video to be played after transcoding; max() is the maximum function; The floor function. As an example, the formulas for calculating the target resolution width and target resolution height of the transcoded video to be played can be as follows:

[0078]

[0079] Among them, the behavioral information A of the target object u =(W u H u ), W u The target resolution width selected for the target object, H u The target resolution height selected for the target object.

[0080] In some embodiments, the color space information of the main stream is determined based on color space information from display-related information of the playback device, color space information used by the video to be played, and color space information used by the current playback device. As an example, the formula for calculating the color space information of the main stream can be expressed as follows:

[0081]

[0082] Among them, CS m The color space information of the main bitstream; CS cr Color space information used for the video to be played; CS cu Color space information used by the current playback device; CS i The color space information is from the display-related information of the i-th playback device; min() is the minimum function.

[0083] In some embodiments, the average effective resolution applicable to multiple playback devices is determined as the resolution of the main stream based on the resolution of the video to be played, the target resolution of the target object, and the resolution from display-related information of the playback devices. The resolution of the main stream includes both width and height. As an example, the resolution of the main stream is defined by its DPI. m =(W m H m The formula for calculating ) can be expressed as follows:

[0084]

[0085] For example, the first transcoding parameters of the main bitstream can be calculated based on the display-related information from the playback device, the network information of the main playback device, and the resource information of the video to be played, using the above formulas (2), (4), (7), (8), and (9).

[0086] In step 104, the video to be played is transcoded based on the first transcoding parameters to obtain the main bitstream, and the main bitstream is sent to the main playback device and the slave playback device.

[0087] In one possible implementation, the video to be played can be transcoded using the aforementioned first transcoding parameters to obtain the main bitstream. This main bitstream is then pushed to the main playback device, which in turn pushes it to the secondary playback devices. Therefore, the transcoding of the main bitstream not only considers the main playback device but also comprehensively considers the optimal transcoding parameters for all playback devices as a whole.

[0088] In step 105, based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, and in conjunction with the device intelligence level of the playback device, the possibility information of automatic transcoding from the playback device is determined.

[0089] It should be noted that in some embodiments, the execution order of steps 104 and 105 can be interchanged or they can be executed simultaneously.

[0090] In embodiments of this disclosure, the visual parameter may include, but is not limited to, at least one of: resolution, color space, and frame rate. In some embodiments, such as Figure 2 As shown, the above-mentioned determination of the possibility of automatic transcoding from the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device, may include, but is not limited to, the following steps.

[0091] In step 201, based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, as well as the type extracted from the resources of the video to be played, the video high-frequency distribution ratio, and the video dynamic factor, a bitstream conversion matrix from the playback device is determined. The bitstream conversion matrix can refer to the conversion expansion or compression matrix used when converting from the main bitstream. In embodiments of this disclosure, the visual parameters may include, but are not limited to, at least one of: resolution, color space, and frame rate. The video high-frequency distribution ratio can be a key indicator for measuring the complexity of video content details and the difficulty of compression; more high-frequency details -> more complex content -> higher required bitrate -> greater challenge to the encoder's optimization capabilities.

[0092] In one possible implementation, the number of rows and columns of the bitstream conversion matrix can be determined based on the resolution in the visual parameters of the first transcoding parameters and the resolution in the visual parameters of the display-related information; the bitstream conversion matrix from the playback device is initialized based on the number of rows and columns of the bitstream conversion matrix; the initialized bitstream conversion matrix is ​​optimized according to the video type, video high-frequency distribution ratio and video dynamic factor extracted from the resources of the video to be played, to obtain the bitstream conversion matrix from the playback device.

[0093] As an example, the formulas for calculating the number of rows and columns of the bitstream conversion matrix can be expressed as follows:

[0094]

[0095] Where, n mr n is the number of rows in the bitstream conversion matrix; mc M is the number of columns in the bitstream conversion matrix; M is the number of image channels in the video to be played, for example, M can be 3. This is the floor function.

[0096] For example, based on the resolution in the visual parameters of the first transcoding parameters and the resolution in the visual parameters of the display-related information, the number of rows and columns of the bitstream conversion matrix can be calculated using the above formulas (10) and (11). The bitstream conversion matrix is ​​then initialized according to the number of rows and columns. As an example, the bitstream conversion matrix M is initialized. e,i The formula is expressed as follows:

[0097]

[0098] For example, the bitstream conversion matrix can be optimized based on the resource style of the video to be played (such as the video type extracted from the video resource, the video high-frequency distribution ratio, and the video dynamic factor), and the optimized bitstream conversion matrix can be used as the bitstream conversion matrix from the playback device. As an example, the optimization formula for this bitstream conversion matrix can be expressed as follows:

[0099]

[0100] Among them, S r P represents the video type extracted from the video resources to be played (the type is quantized into a natural number using a uniform quantization method); s Maximum value of type; HF r Video high frequency distribution ratio; VD r This represents the video dynamics factor. As an example, the video high-frequency distribution ratio is obtained as follows:

[0101]

[0102] Among them, HF r W represents the high-frequency distribution ratio in the video. p H represents the resolution width of the video to be played. p L represents the resolution height of the video to be played; N represents the number of sample frames extracted from the video's resources; L represents the height of the video to be played. hi，i，j，k L represents the color value at coordinates (j, k) of the high-frequency image obtained by applying a high-pass filter to the i-th sampled frame of the original image; i，x，y This represents the color value of the i-th sampled frame of the original image at coordinates (j, k).

[0103] As an example, the video dynamic factor is obtained in the following way:

[0104]

[0105] Among them, VD r For video dynamic factor; d i，j，k Let be the pixel difference between the i-th sampled frame and its previous frame at coordinates (j, k).

[0106] For example, use the resolution DPI from the display-related information. i Color space CS i Frame Rate RF i Based on its relationship with the main stream frame rate RF m Color space CS m Resolution DPI m The difference value, combined with the type S extracted by the cloud disk device from the video resource to be played. i High-frequency distribution ratio of video (HF) i Video Dynamics Factor (VD) i Using the above formulas (10) to (13), the bitstream conversion matrix from the playback device is calculated.

[0107] In step 202, based on the bitstream conversion matrix from the playback device and combined with the device intelligence level of the playback device, the possibility information of automatic transcoding from the playback device is determined.

[0108] In some embodiments, the self-transcoding probability information of the playback device can be calculated using the following formula (16) based on the bitstream conversion matrix from the playback device and in conjunction with the device intelligence level of the playback device. As an example, the formula for calculating the self-transcoding probability information of the playback device can be expressed as follows:

[0109]

[0110] Among them, P a，i For the i-th possibility of self-transcoding from the playback device; IA i W is the intelligent attribute in the device intelligence level of the i-th playback device; i H represents the resolution width of the i-th playback device. i W represents the resolution height of the i-th playback device. m The resolution width of the main bitstream, H m The resolution height of the main bitstream; AO i For the i-th device from the playback device's device intelligence level, determine whether it is an operation execution device; CP i For the computing power information of the i-th playback device; RF i BR is the frame rate of the i-th playback device; m The bitrate of the main stream; ch(CS) m The number of channels in the color space of the main bitstream; M e，i Let be the bitstream conversion matrix for the i-th playback device.

[0111] In step 106, the self-transcoding probability information is sent to the corresponding slave playback device. The self-transcoding probability information is used to determine whether the corresponding slave playback device will convert the main bitstream into an auxiliary bitstream.

[0112] In the embodiments of this disclosure, after obtaining the self-transcoding probability information from the playback device, the self-transcoding probability information can be sent to the corresponding playback device. When the playback device receives its own self-transcoding probability information, it can determine whether to convert the acquired main bitstream into an auxiliary bitstream based on the self-transcoding probability information. As an example, if the self-transcoding probability information does not meet the self-transcoding conditions, the main bitstream is used for playback; or, if the self-transcoding probability information meets the self-transcoding conditions, the bitstream conversion matrix of the playback device is obtained, and the main bitstream is converted using the bitstream conversion matrix to obtain the auxiliary bitstream of the playback device. Optional implementation methods can be found in the optional implementation methods of the relevant steps in the subsequent playback device (slave playback device) side method embodiments, and will not be repeated here. Therefore, the video quality is dynamically adjusted according to network conditions and device performance to ensure that users can have a smooth playback experience in different environments. In the case of multiple playback devices (also called display devices), the cloud disk backend server only provides the main bitstream and optimizes the main bitstream through the terminal playback device to obtain an optimized auxiliary bitstream. This improves the playback effect of the playback device with minimal additional server computing power consumption. Through flexible and intelligent management, buffering time and stuttering are reduced, improving user viewing satisfaction.

[0113] In the above embodiments, by providing an auxiliary bitstream to the playback device, only one transcoding is required from the backend server. During this process, the transcoding of the main bitstream not only considers the main playback device, but also comprehensively considers the optimal transcoding parameters of all playback devices as a whole. By calculating the possibility of screen casting and sharing, the playback device automatically optimizes the main bitstream based on its own transcoding probability information to obtain an optimized auxiliary bitstream. This improves the playback effect of the playback device with minimal additional server computing power consumption. Through flexible and intelligent management, buffering time and stuttering are reduced, improving user viewing satisfaction and effectively realizing the transmission of adaptive video streams.

[0114] Figure 3 This is a flowchart illustrating a video processing method according to an exemplary embodiment. It should be noted that the execution entity of the video processing method in this embodiment can be a video processing device, which can be implemented in software and / or hardware, and can be configured in an electronic device. For example, the electronic device can be a playback device (hereinafter referred to as a slave playback device). Figure 3 As shown, the video processing method may include, but is not limited to, the following steps.

[0115] In step 301, the main bitstream is obtained.

[0116] In the embodiments of this disclosure, the main bitstream is obtained by transcoding the video to be played based on a first transcoding parameter. The first transcoding parameter is determined based on display-related information from the playback device, network information from the main playback device, and resource information from the video to be played. Optional implementations of this main bitstream can be found in the optional implementations of the relevant steps in the cloud disk device-side method embodiments described above, and will not be repeated here.

[0117] In step 302, information on the possibility of automatic transcoding by the playback device (from the playback device) is obtained.

[0118] In the embodiments of this disclosure, the self-transcoding probability information is determined based on the difference between visual parameters in the display-related information of the playback device and visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device. The playback device is any one of the playback devices, which is determined by dividing multiple playback devices according to the display-related information of multiple playback devices associated with the target object when performing non-isolated viewing behavior. Optional implementations of this self-transcoding probability information can be found in the optional implementations of the relevant steps in the above-described cloud disk device-side method embodiments, and will not be repeated here.

[0119] It should be noted that in some embodiments, steps 301 and 302 above can be executed in an alternate order or simultaneously.

[0120] It should be noted that, in some embodiments, after obtaining its own self-transcoding probability information, the playback device (from the playback device) can determine whether it needs to convert the main bitstream into an auxiliary bitstream based on this self-transcoding probability information. For example, it can determine whether the self-transcoding probability information meets the self-transcoding conditions to determine whether the main bitstream needs to be converted into an auxiliary bitstream. Optionally, the self-transcoding probability information meeting the self-transcoding conditions may mean that the self-transcoding probability information is greater than a threshold (e.g., 0); the self-transcoding probability information not meeting the self-transcoding conditions may mean that the self-transcoding probability information is less than or equal to the threshold. If the self-transcoding probability information does not meet the self-transcoding conditions, step 303 can be executed. If the self-transcoding probability information meets the self-transcoding conditions, step 304 can be executed.

[0121] In step 303, the main stream is used for playback.

[0122] For example, if the information regarding the possibility of self-transcoding does not meet the conditions for self-transcoding (e.g., from playback device N), s,i The corresponding self-transcoding probability information P a,iIn the case of ≤0), from playback device N s,i It can be played directly using the main stream.

[0123] In step 304, the bitstream conversion matrix of the playback device (from the playback device) is obtained, and the main bitstream is converted using the bitstream conversion matrix to obtain the auxiliary bitstream of the playback device (from the playback device).

[0124] For example, if the self-transcoding possibility information meets the self-transcoding conditions, the bitstream conversion matrix of the playback device (from the playback device) can be obtained, and the main bitstream can be converted using this bitstream conversion matrix to obtain the auxiliary bitstream of the playback device (from the playback device). The bitstream conversion matrix of the playback device (from the playback device) can be sent to the playback device (from the playback device) by the cloud disk device. The method for determining the bitstream conversion matrix of the playback device (from the playback device) can be found in the optional implementation methods of the relevant steps in the above-described cloud disk device-side method embodiments, and will not be repeated here.

[0125] As an example, the conversion logic of a playback device (from the playback device) converting a main bitstream to an auxiliary bitstream can be as follows: for the auxiliary bitstream position at coordinates (x, y), its pixel value L′ x,y The calculation logic is as follows:

[0126]

[0127] Among them, L x,y This represents the original value of the position (x, y) in the main bitstream frame, n. mr n represents the number of rows in the bitstream conversion matrix. mc P represents the number of columns in the bitstream conversion matrix. cmi P represents the minimum usable color value in the corresponding color space. cma This represents the maximum usable color value in the corresponding color space. The calculation method for rlu(x1,y1,z1) is as follows:

[0128]

[0129] In the above embodiments, by providing an auxiliary bitstream to the playback device, only one transcoding is required from the backend server. During this process, the transcoding of the main bitstream not only considers the main playback device, but also comprehensively considers the optimal transcoding parameters of all playback devices as a whole. By calculating the possibility of screen casting and sharing, the playback device automatically optimizes the main bitstream based on its own transcoding probability information to obtain an optimized auxiliary bitstream. This improves the playback effect of the playback device with minimal additional server computing power consumption. Through flexible and intelligent management, buffering time and stuttering are reduced, improving user viewing satisfaction and effectively realizing the transmission of adaptive video streams.

[0130] In some embodiments, after obtaining the auxiliary bitstream, the playback device (or the device receiving the auxiliary bitstream) may determine whether frame interpolation processing is required for the auxiliary bitstream based on the playback device's frame interpolation capability information and / or frame interpolation necessity information before playing the corresponding auxiliary bitstream. In one embodiment, the need for frame interpolation processing can be determined based on the playback device's frame interpolation capability information. In another embodiment, the need for frame interpolation processing can be determined based on the playback device's frame interpolation necessity information. In yet another embodiment, the need for frame interpolation processing can be determined based on both the playback device's frame interpolation capability information and frame interpolation necessity information.

[0131] In one possible implementation, the frame interpolation capability information of the playback device can be calculated based on the current computing power, frame rate, resolution, and color depth of the playback device, along with the frame rate, color depth, device transmission bandwidth, and content latency acceptability of the main bitstream. As an example, the formula for calculating the frame interpolation capability information of the playback device can be expressed as follows:

[0132]

[0133] Among them, IF b,i This refers to the frame interpolation capability information of the playback device (from the playback device); the current computing power of the playback device is CP. i The frame rate is RF i The resolution is DPI i =(W i H i ), color depth is D c,i The frame rate of the main stream is RF. m Resolution is DPI m =(W m H m ), color depth is D m The device's transmission bandwidth is B. m,i Content delay acceptance rate is CA l .

[0134] In one possible implementation, the frame interpolation necessity information of the playback device can be determined based on the current playback device's resolution (DPI). i Color space CS i Computing power CP i and the expected computing power load of the equipment LC i The frame rate of the main stream is RF. m The current device's frame rate is RF. i Calculation. As an example, the formula for calculating the frame interpolation necessity information of the playback device can be expressed as follows:

[0135]

[0136] Among them, DPI i =(W i H i ) represents the resolution of the current playback device (the i-th slave playback device), CS i For the color space information of the current playback device, CP i LC provides computing power information for the current playback device. i The expected computing load (LC) of the current playback device i RF i RF is the frame rate of the current playback device. m The frame rate of the main bitstream.

[0137] For example, the frame interpolation necessity information IF of the playback device m,i This could be the frame interpolation capability information of the playback device (IF). b,i It is calculated under certain conditions, such as the frame interpolation capability information (IF) of the playback device. b,i If the value is greater than a threshold (e.g., 1), the frame interpolation necessity information IF of the playback device is calculated according to the above formula (20). m,i .

[0138] Optionally, in some embodiments, in the above... Figure 3 Based on the illustrated embodiments, as Figure 4 As shown, the video processing method involved in the playback device-side method embodiment may also include, but is not limited to, the following step 401. Step 401 is a step that is only executed when it is determined, based on the playback device's frame interpolation capability information and / or frame interpolation necessity information, that frame interpolation processing of the auxiliary bitstream is not required. If it is determined, based on the playback device's frame interpolation capability information and / or frame interpolation necessity information, that frame interpolation processing of the auxiliary bitstream is required, then the following step 402 will be executed.

[0139] In step 401, if it is determined that frame interpolation processing of the auxiliary bitstream is not required based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, the auxiliary bitstream is used for playback.

[0140] For example, if the playback device's frame interpolation capability information is less than or equal to a threshold (e.g., 1), it is determined that frame interpolation processing of the auxiliary bitstream is unnecessary, and the playback device can directly use the auxiliary bitstream for playback. If the playback device's frame interpolation capability information is greater than the threshold, the frame interpolation necessity information of the playback device is determined. If the playback device's frame interpolation necessity information is less than the threshold (e.g., 0.5), the playback device can directly use the auxiliary bitstream for playback.

[0141] Optionally, in some embodiments, in the above... Figure 3Based on the illustrated embodiments, as Figure 4 As shown, the video processing method involved in the playback device side method embodiment may also include, but is not limited to, the following step 402.

[0142] In step 402, if it is determined that the auxiliary bitstream needs to be processed by frame interpolation based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, the computational power consumption ratio when frame interpolation is performed using the motion compensation frame interpolation algorithm is obtained. If the computational power consumption ratio meets the preset conditions, the auxiliary bitstream is frame interpolated using the motion compensation frame interpolation algorithm and then played. Alternatively, if the computational power consumption ratio does not meet the preset conditions, the auxiliary bitstream is frame interpolated using the linear frame interpolation algorithm and then played.

[0143] In one embodiment, it can be determined whether frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation capability information of the playback device. If it is determined that frame interpolation processing of the auxiliary bitstream is not required based on the frame interpolation capability information of the playback device, the auxiliary bitstream is used for playback. If it is determined that frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation capability information of the playback device, the computational cost ratio when using a motion-compensated frame interpolation algorithm is obtained. If the computational cost ratio meets a preset condition, the auxiliary bitstream is interpolated using a motion-compensated frame interpolation algorithm and then played; or, if the computational cost ratio does not meet the preset condition, the auxiliary bitstream is interpolated using a linear frame interpolation algorithm and then played.

[0144] In another embodiment, it can be determined whether frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation necessity information of the playback device. If it is determined that frame interpolation processing of the auxiliary bitstream is not required based on the frame interpolation necessity information of the playback device, the auxiliary bitstream is used for playback. If it is determined that frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation necessity information of the playback device, the computational cost ratio when using the motion compensation frame interpolation algorithm for frame interpolation is obtained. If the computational cost ratio meets a preset condition, the auxiliary bitstream is interpolated using the motion compensation frame interpolation algorithm and then played; or, if the computational cost ratio does not meet the preset condition, the auxiliary bitstream is interpolated using a linear frame interpolation algorithm and then played.

[0145] In another embodiment, it can be determined whether frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation capability information and frame interpolation necessity information of the playback device. If it is determined that frame interpolation processing of the auxiliary bitstream is not required based on the frame interpolation capability information and frame interpolation necessity information of the playback device, the auxiliary bitstream is used for playback. If it is determined that frame interpolation processing of the auxiliary bitstream is required based on the frame interpolation capability information and frame interpolation necessity information of the playback device, the computational power consumption ratio when using the motion compensation frame interpolation algorithm for frame interpolation is obtained. If the computational power consumption ratio meets a preset condition, the auxiliary bitstream is played after frame interpolation using the motion compensation frame interpolation algorithm; or, if the computational power consumption ratio does not meet the preset condition, the auxiliary bitstream is played after frame interpolation using the linear frame interpolation algorithm.

[0146] For example, if the frame interpolation capability information of the playback device is greater than a threshold (e.g., 1), the frame interpolation necessity information of the playback device is determined. If the frame interpolation necessity information of the playback device is greater than or equal to a threshold (e.g., 0.5), it is determined that frame interpolation processing of the auxiliary bitstream is required. In this case, the computational cost ratio when using a motion-compensated frame interpolation algorithm can be obtained. As an example, this computational cost ratio P... if,mc The calculation formula can be expressed as follows:

[0147]

[0148] Among them, o if,mc The time complexity of the frame interpolation algorithm is calculated for the motion compensation frame interpolation algorithm.

[0149] For example, when the computing power consumption is higher than P if,mc Meet preset conditions, such as computing power consumption ratio P if,mc When the value is greater than or equal to a threshold (e.g., 1), a motion-compensated frame interpolation algorithm can be used to interpolate frames into the auxiliary bitstream before playback. When the computational cost is greater than P... if,mc If the preset conditions are not met, such as the computing power consumption ratio P if,mc When the value is less than a threshold (e.g., 1), a linear frame interpolation algorithm can be used to interpolate the auxiliary bitstream before playback.

[0150] Therefore, by combining the frame interpolation capabilities and necessity of the playback device, and automatically performing frame interpolation processing on the corresponding auxiliary bitstream, screen tearing can be further reduced, and the playback effect of the playback device can be further improved.

[0151] Figure 5 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment. The video processing apparatus can be configured on a cloud storage device. Figure 5 As shown, the video processing device may include: a first determining module 501, a dividing module 502, a second determining module 503, a transcoding processing module 504, a third determining module 505, and a sending module 506.

[0152] The first determining module 501 is used to determine the non-isolated viewing behavior of the target object based on the target object's behavior data on the video to be played. The non-isolated viewing behavior includes multi-screen sharing or screen casting behavior.

[0153] The partitioning module 502 is used to partition multiple playback devices into a main playback device and a slave playback device based on display-related information of multiple playback devices associated with non-isolated viewing behavior.

[0154] The second determining module 503 is used to determine the first transcoding parameters of the main bitstream based on the display-related information of the playback device, the network information of the main playback device, and the resource information of the video to be played.

[0155] The transcoding processing module 504 is used to transcode the video to be played based on the first transcoding parameters to obtain the main bitstream, and send the main bitstream to the main playback device and the slave playback device.

[0156] The third determining module 505 is used to determine the possibility information of automatic transcoding from the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device.

[0157] The sending module 506 is used to send the self-transcoding possibility information to the corresponding slave playback device. The self-transcoding possibility information is used to determine whether the corresponding slave playback device will convert the main bitstream into an auxiliary bitstream.

[0158] In some embodiments, the partitioning module 502 is configured to: determine the auxiliary transcoding capability information of each of the multiple playback devices based on the display-related information of the multiple playback devices associated with the non-isolated viewing behavior; and, based on the auxiliary transcoding capability information, designate the playback device with the strongest auxiliary transcoding capability among the multiple playback devices as the master playback device, and designate the devices other than the master playback device among the multiple playback devices as slave playback devices.

[0159] In some embodiments, the display-related information includes at least the resolution of the playback device, color space information, frame rate, smart attributes, and computing power information; wherein, the calculation formula for the auxiliary transcoding capability information is expressed as follows:

[0160]

[0161] Among them, C t,i For time t, the auxiliary transcoding capability information of the i-th playback device; IA i For the intelligent attributes of the i-th playback device; CP i This refers to the computing power information of the i-th playback device; DPI i =(W i Hi Let W be the resolution of the i-th playback device. i H is the width in pixels. i RF (Radio Frequency) is the number of pixels at height. i CR is the frame rate of the i-th playback device; m CR represents the minimum supported color gamut of the color space used by the video resource to be played; G is a constant used to compress and expand the value range to ensure value range consistency; i W represents the color space information of the i-th playback device. a H represents the target resolution width in the target resolution of the video to be played after transcoding; a This represents the target resolution height within the target resolution range after the video is transcoded.

[0162] In some embodiments, the first transcoding parameters of the main stream include: the bitrate of the main stream, the frame rate of the main stream, the color space information of the main stream, and the resolution of the main stream; wherein, the second determining module 503 is used to: determine the bitrate of the main stream based on the resolution, frame rate, device intelligence level, network information of the main playback device, and resource information of the video to be played from the display-related information of the playback device; determine the frame rate of the main stream based on the maximum supported frame rate of the current playback device, the frame rate from the display-related information of the playback device, the target resolution of the video to be played after transcoding, and the bitrate from the second transcoding parameters of the unique stream of the playback device under the exclusive viewing behavior; the second transcoding parameters are determined based on the network information of the current playback device, the behavior information of the target object, and the resource information of the video to be played; determine the color space information of the main stream based on the color space information from the display-related information of the playback device, the color space information used by the video to be played, and the color space information used by the current playback device; and determine the average effective resolution applicable to multiple playback devices as the resolution of the main stream based on the resolution of the video to be played, the target resolution selected by the target object, and the resolution from the display-related information of the playback device.

[0163] In some embodiments, the third determining module 505 is configured to: determine a bitstream conversion matrix from the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, as well as the type extracted from the resources of the video to be played, the video high-frequency distribution ratio, and the video dynamic factor, wherein the bitstream conversion matrix is ​​a conversion expansion or compression matrix used when converting from the main bitstream; and determine the self-transcoding probability information of the playback device based on the bitstream conversion matrix from the playback device and in combination with the device intelligence level of the playback device.

[0164] In some embodiments, the third determining module 505 is configured to: determine the number of rows and columns of the bitstream conversion matrix based on the resolution in the visual parameters of the first transcoding parameters and the resolution in the visual parameters of the display-related information; initialize the bitstream conversion matrix from the playback device based on the number of rows and columns of the bitstream conversion matrix; and optimize the initialized bitstream conversion matrix according to the video type, video high-frequency distribution ratio, and video dynamic factor extracted from the resources of the video to be played, to obtain the bitstream conversion matrix from the playback device.

[0165] In some embodiments, the video high-frequency distribution ratio is obtained in the following way:

[0166]

[0167] Among them, HF r W represents the high-frequency distribution ratio in the video. p H represents the resolution width of the video to be played. p L represents the resolution height of the video to be played; N represents the number of sample frames extracted from the video's resources; L represents the height of the video to be played. hi,i,j,k L represents the color value of the high-frequency image at coordinates (j,k) obtained by applying a high-pass filter to the i-th sampled frame of the original image; i,x,y This represents the color value of the i-th sampled frame of the original image at coordinates (j, k).

[0168] In some embodiments, the video dynamic factor is obtained in the following way:

[0169]

[0170] Among them, VD r For video dynamic factor; d i,j,k Let be the pixel difference between the i-th sampled frame and its previous frame at coordinates (j,k).

[0171] In some embodiments, the formula for calculating the possibility of automatic transcoding from the playback device is expressed as follows:

[0172]

[0173] Among them, P a,i For the i-th possibility of self-transcoding from the playback device; IA i W is the intelligent attribute in the device intelligence level of the i-th playback device; i H represents the resolution width of the i-th playback device. i W represents the resolution height of the i-th playback device. m The resolution width of the main bitstream, H m The resolution height of the main bitstream; AO iFor the i-th device from the playback device's device intelligence level, determine whether it is an operation execution device; CP i For the computing power information of the i-th playback device; RF i BR is the frame rate of the i-th playback device; m The bitrate of the main stream; ch(CS) m The number of channels in the color space of the main bitstream; M e,i Let be the bitstream conversion matrix for the i-th playback device.

[0174] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0175] Figure 6 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment. The video processing apparatus can be configured on a cloud storage device. Figure 6 As shown, the video processing device may include: a first acquisition module 601, a second acquisition module 602, and a playback module 603.

[0176] The first acquisition module 601 is used to acquire the main bitstream, which is obtained by transcoding the video to be played based on the first transcoding parameters. The first transcoding parameters are determined based on the display-related information of the playback device, the network information of the main playback device, and the resource information of the video to be played.

[0177] The second acquisition module 602 is used to acquire the self-transcoding possibility information of the playback device. The self-transcoding possibility information is determined based on the difference between the visual parameters in the display-related information of the playback device and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device. The playback device is any playback device among the playback devices. The playback devices are determined by dividing multiple playback devices according to the display-related information of multiple playback devices associated with the target object when performing non-isolated viewing behavior.

[0178] The playback module 603 is used to: play the main bitstream when the self-transcoding possibility information does not meet the self-transcoding conditions; or, when the self-transcoding possibility information meets the self-transcoding conditions, obtain the bitstream conversion matrix of the playback device and use the bitstream conversion matrix to convert the main bitstream to obtain the auxiliary bitstream of the playback device.

[0179] In some embodiments, the playback module 603 is further configured to: play the auxiliary bitstream when it is determined, based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, that frame interpolation processing of the auxiliary bitstream is not required; or, when it is determined, based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, that frame interpolation processing of the auxiliary bitstream is required, obtain the computational power consumption ratio when frame interpolation is performed using a motion compensation frame interpolation algorithm, and play the auxiliary bitstream after frame interpolation using a motion compensation frame interpolation algorithm when the computational power consumption ratio meets a preset condition; or, play the auxiliary bitstream after frame interpolation using a linear frame interpolation algorithm when the computational power consumption ratio does not meet the preset condition.

[0180] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0181] This disclosure also provides a video processing system, which may include a cloud storage device and a playback device. The cloud storage device can be configured to execute the video processing method described in the cloud storage device-side method embodiments above. The playback device can be configured to execute the video processing method described in the playback device-side method embodiments above. Further details are omitted here.

[0182] To facilitate a clearer understanding of this disclosure by those skilled in the art, the following will be combined with... Figure 7 The scheme disclosed herein is described in detail.

[0183] like Figure 7 As shown, the video processing method may include, but is not limited to, the following steps.

[0184] In step 701, the cloud disk device video playback device captures the user's non-exclusive viewing behavior in real time, that is, captures the user's instructions for multi-screen sharing or screen casting on the device.

[0185] For example, the cloud drive device playback adds multi-screen sharing and screen casting function detection capabilities, which are activated when video playback begins, and continuously monitor whether the screen casting function of the playback device is enabled, whether the internal capture card is enabled, and whether the external capture card is enabled during video resource playback.

[0186] By detecting the above, it can be determined whether the user is playing the video simultaneously on multiple terminal devices during the current video playback process. When more than one device is used for simultaneous playback and display, it is considered non-exclusive viewing behavior. Similarly, when only one device is used for playback and display, it is considered exclusive viewing behavior.

[0187] In step 702, when it is detected that the user has only performed a private browsing action, a scheme is determined using transcoding parameters that are adaptive to network quality and system load, a unique bitstream is generated, and the transcoding is performed according to the current network conditions of the playback device N.u User behavior A u Resource Information R t Together, they determine the transcoding parameters P of the unique bitstream. s .

[0188] It is worth noting that the network quality and system load adaptive transcoding parameter determination scheme is different from the FFmpeg lossless transcoding and HandBrake lossless transcoding methods. The network quality and system load adaptive transcoding disclosed herein, under limited resource conditions, automatically sets the transcoding control parameters of the original video according to the current player's capabilities and resource status, and uses lossy transcoding to perform lossy compression on the file to reduce resource requirements while maximizing playback quality.

[0189] When it is detected that the user has only performed a private viewing behavior (i.e., the screen mirroring function is off and neither the built-in nor external capture card is started), the following transcoding parameters, which are adaptive to network quality and system load, are used to determine the scheme to generate a unique bitstream and adjust it according to the current network conditions of the playback device N. u User behavior A u Together, they determine the second transcoding parameter P of the unique bitstream. s =(BR a DPI a ,RF a CS ca ), where BR a The bitrate is represented by the formula (3) above. a =(W a H a W represents the target resolution. a H is the target resolution width. a The calculation logic for the target resolution height is as follows:

[0190]

[0191]

[0192] RF a The frame rate is represented by the following calculation logic:

[0193]

[0194] Among them, RF a Indicates the original frame rate of the video, RF u This indicates the maximum frame rate supported by the current playback device.

[0195] CS ca The color space is represented by the following calculation logic:

[0196]

[0197] Among them, CS cu CS is the color space used by the current playback device (i.e., the color gamut in the set color mode). cr The color space used in the original video.

[0198] In step 703, when the cloud storage device detects that a user has triggered multi-screen sharing or screen casting, the cloud storage device reads information from all display devices (also called playback devices). t (i.e., the display-related information mentioned above), for the i-th display device, its content includes resolution DPI. i Color space CS i Frame Rate RF i Is it an intelligent device (IA)? i Computing power CP i Color gamut CR obtained from the arrangement of screen panel models i Color Depth D c，i Is it an operation execution device (AO)? i , that is I t，i = (DPI) i CS i RF i IA i CP i AO i CR i D c，i ).

[0199] For example, when a user is detected to have performed multi-screen sharing or screen casting (i.e., screen casting is enabled, or a built-in capture card is enabled, or an external capture card is inserted and enabled), the cloud disk device reads all display device information to obtain basic information about all display devices. n d Given the total number of display devices, for the i-th display device, its basic display device information I... t,i Content includes resolution DPI i Color space CS i Frame Rate RF i Is it an intelligent device (IA)? i (1 for yes, 0 for no), Computing power CP i Color gamut CR obtained from the arrangement of screen panel models i Color Depth DC ，i Is it an operation execution device (AO)? i , that is I t，i = (DPI) i CS i RF i IA iCP i AO i CR i D c，i For example, such as Figure 8 As shown, the above screen mirroring behavior can be a one-to-one behavior, such as... Figure 9 As shown, the above screen mirroring behavior can be a one-to-many screen mirroring behavior.

[0200] In step 704, the cloud disk device iterates through all playback devices N. a For the i-th playback device N n，i Using basic information I t,i DPI in the middle i Color space CS i Is it an intelligent device (IA)? i Computing power CP i Calculate the auxiliary transcoding capability information C for each playback device. t，i The device with the strongest auxiliary transcoding capability information is selected as the main playback device N. m This is used to play the main stream and push it to the secondary playback device.

[0201] For example, the above formula (1) can be used to calculate the i-th playback device N. n，i Auxiliary transcoding capability information C t，i After the calculation is complete, the auxiliary transcoding capability information for all playback devices is as follows: Choose the playback device with the largest auxiliary transcoding capability as the primary playback device N. m The remaining devices are from playback device N. s =N a -N m During subsequent video playback, the cloud storage server will push the transcoded main playback stream to the main playback device N. m Main playback device N m Push the main stream to the playback device N s .

[0202] In step 705, the cloud disk device iterates through all playback devices N. s According to each playback device N s,i resolution DPI i Color space CS i Frame Rate RF i Is it an intelligent device (IA)? i Computing power CP i Is it an operation execution device (AO)? i With the main playback device N m Network conditions N u Combined user behavior information Au =(W u H u Q ua ), calculate the transcoding parameters P of the main bitstream. s =(BR m ,RF m CS m DPI m ), where the main stream bitrate BR m Frame rate RF m Color space CS m Resolution DPI m Among them, the main stream bitrate BR m Frame rate RF m Color space CS m Resolution DPI m The respective calculation formulas can be found in formulas (2), (4), (7), (8) and (9) above, and will not be repeated here.

[0203] In step 706, the cloud disk backend server iterates through all playback devices N. s For the i-th playback device N s,i Using basic information I t,i DPI in the middle i Color space CS i Frame Rate RF i RF of the combined main stream frame rate m Color space CS m Resolution DPI m The difference, along with the type S extracted from the video resource to be played. r High-frequency distribution ratio of video (HF) r Video Dynamics Factor (VD) r The calculation yields the result from playback device N. s Corresponding bitstream conversion matrix Among them, the i-th playback device N n,i The bitstream conversion matrix M e,i The calculation formula can be found in formula (13) above, and will not be repeated here.

[0204] In step 707, the cloud disk device traverses from playback device N s For the i-th playback device N s,i Based on whether it is an intelligent device (IA) i Computing power information CP i Is it an operation execution device (AO)? i The bitstream conversion matrix is ​​used to calculate the bitstream from the playback N. s,i Information P on the possibility of automatic transcoding a,iInformation P regarding the possibility of automatic transcoding a,i The calculation formula can be found in formula (16) above, and will not be repeated here.

[0205] For example, a cloud storage device can send the self-transcoding probability information of each playback device to the corresponding playback device, so that the playback device can determine whether to convert the main stream into an auxiliary stream based on the corresponding self-transcoding probability information.

[0206] In step 708, from playback device N s,i Read the self-transcoding probability information P corresponding to itself a,i When P a,i If the value is ≤0, play the main stream directly; otherwise, proceed to step 709.

[0207] For example, from playback device N s,i Read the self-transcoding probability information P corresponding to itself a,i When P a,i When the value is ≤0, the main stream is used directly for playback. In this process, the main stream is the unique stream obtained by transcoding from the cloud disk backend server (also called the cloud disk device) and played through the main playback device n. m After forwarding without any processing, the data is received from the playback device n. s,i The obtained bitstream. At this point, from playback device n... s,i The stream can be directly sent to the player for playback.

[0208] In step 709, from playback device N s,i Read the self-transcoding probability information P corresponding to itself a,i When P a,i When >0, use the bitstream conversion matrix M e,i The main bitstream is converted to obtain the auxiliary bitstream from the playback device.

[0209] For example, from playback device N s,i Read the self-transcoding probability information P corresponding to itself a,i When P a,i When >0, use the corresponding bitstream conversion matrix M e,i The main bitstream is converted to obtain the auxiliary bitstream from the playback device. The specific conversion logic is as follows: for the position x, y of the auxiliary bitstream, its pixel value L′ x,y The calculation logic is as described in the above formula (17), which will not be repeated here.

[0210] In step 710, before playing its corresponding auxiliary bitstream, the playback device determines the appropriate bitstream based on its current computing power (CP). i Frame rate RF i Resolution DPI iColor depth D c,i RF of the main stream m Resolution DPI m Color Depth D m Equipment transmission bandwidth B m,i Content Delay Acceptance (CA) l Calculate the frame interpolation capability information from the playback device. b,i .

[0211] In step 711, when the frame interpolation capability IF b,i When the resolution is greater than 1, the DPI will be based on the current playback device's resolution. i Color space CS i Computing power information CP i and the expected computing power load of the equipment LC i The frame rate of the main stream is RF. m The current device's frame rate is RF. i The current playback device N is calculated. s,i Frame interpolation necessity information (IF) m,i .

[0212] In step 712, when the frame insertion necessity IF m,i When ≥0.5, calculate the computational cost ratio P when using the motion-compensated frame interpolation algorithm for frame interpolation. if,mc When P if,mc When P ≥ 1, a motion compensation frame interpolation algorithm is used to interpolate frames into the auxiliary bitstream before playback. if,mc When the value is less than 1, a linear frame interpolation algorithm is used to interpolate the auxiliary bitstream before playback.

[0213] In summary, this disclosure employs an auxiliary bitstream provided to the playback device. Its advantage lies in determining whether the user's playback behavior is exclusive and then using either a single bitstream scheme or a hybrid method combining the main bitstream and auxiliary bitstream to provide the video playback bitstream, effectively achieving adaptive video stream transmission. This method dynamically adjusts video quality based on network conditions and device performance, ensuring a smooth playback experience for users in various environments. In multi-display device scenarios, the cloud storage backend server only provides the main bitstream, and the terminal playback device automatically optimizes the main bitstream to obtain an optimized auxiliary bitstream. This improves the playback effect on the display device with minimal additional server computing power consumption. Through flexible and intelligent management, buffering time and stuttering are reduced, improving user viewing satisfaction.

[0214] It is worth noting that this disclosure provides a method for determining unique bitstream transcoding parameters under unique viewing behavior. It employs a transcoding parameter determination scheme that adapts to network quality and system load, enabling the generation of a unique bitstream for each user's playback device. By monitoring the network status, user behavior, and system resource information of the current playback device in real time, the system intelligently adjusts the transcoding parameters to achieve optimal playback performance. When network bandwidth fluctuates or device performance is affected, the system can dynamically change the video resolution, bitrate, and frame rate to ensure a smooth viewing experience. Simultaneously, the analysis of user behavior data allows the system to better adapt to users' viewing habits, thereby optimizing the bitstream in advance for adaptation. This adaptive strategy improves the video viewing quality.

[0215] It is important to emphasize that this disclosure adopts the method of providing an auxiliary bitstream to the playback device, requiring only one transcoding from the backend server. In this process, the transcoding of the main bitstream not only takes into account the main playback device, but also comprehensively considers the optimal transcoding parameters of all playback devices as a whole. By calculating the expansion / compression and frame interpolation capabilities of the playback device from the main bitstream during screen casting and sharing, the playback device can complete the picture quality optimization within the playback device based on its capabilities, thereby improving the playback effect with a small amount of service computing power.

[0216] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0217] like Figure 10 The diagram shown is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein. Exemplarily, the electronic device may be a cloud storage device (such as a cloud storage server) or a playback device, which may be a mobile phone, tablet, computer, or television, etc.

[0218] like Figure 10As shown, the electronic device includes one or more processors 1001, a memory 1002, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 10 Take processor 1001 as an example.

[0219] The memory 1002 is the non-transitory computer-readable storage medium provided in this disclosure. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the video processing method provided in this disclosure. The non-transitory computer-readable storage medium of this disclosure stores computer instructions for causing a computer to perform the video processing method provided in this disclosure.

[0220] Memory 1002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the video processing method in the embodiments of this disclosure (e.g., appendix). Figure 5 The first determining module 501, the dividing module 502, the second determining module 503, the transcoding processing module 504, the third determining module 505, and the sending module 506 shown are attached. Figure 6 The first acquisition module 601, the second acquisition module 602, and the playback module 603 are shown. The processor 1001 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 1002, thereby implementing the video processing method in the above method embodiments.

[0221] The memory 1002 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1002 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1002 may optionally include memory remotely located relative to the processor 1001, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0222] The electronic device may also include an input device 1003 and an output device 1004. The processor 1001, memory 1002, input device 1003, and output device 1004 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0223] Input device 1003 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1004 may include display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, liquid crystal display (LCD), light-emitting diode (LED) display, and plasma display. In some embodiments, the display device may be a touch screen.

[0224] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0225] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0226] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0227] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0228] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0229] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0230] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, The method includes: Based on the target object's behavioral data regarding the video to be played, determine that the target object performs non-isolated behavior; Based on the display-related information of the multiple playback devices associated with the non-exclusive viewing behavior, the multiple playback devices are divided into primary playback devices and secondary playback devices; Based on the display-related information of the playback device, the network information of the main playback device, and the resource information of the video to be played, the first transcoding parameters of the main bitstream are determined; The video to be played is transcoded based on the first transcoding parameters to obtain the main bitstream, and the main bitstream is sent to the main playback device and the slave playback device; Based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, and in conjunction with the device intelligence level of the playback device, the possibility information of the playback device's automatic transcoding is determined. The self-transcoding probability information is sent to the corresponding slave playback device. The self-transcoding probability information is used by the corresponding slave playback device to determine whether to convert the main bitstream into an auxiliary bitstream. The determination of whether to convert the main bitstream into an auxiliary bitstream includes: if the self-transcoding probability information of the corresponding slave playback device does not meet the self-transcoding conditions, playing the main bitstream; if the self-transcoding probability information of the corresponding slave playback device meets the self-transcoding conditions, converting the main bitstream using the bitstream conversion matrix of the slave playback device to obtain the auxiliary bitstream of the slave playback device.

2. The method according to claim 1, characterized in that, The step of classifying the multiple playback devices into primary playback devices and secondary playback devices based on display-related information associated with the non-exclusive viewing behavior includes: Based on the display-related information of multiple playback devices associated with the non-isolated viewing behavior, determine the auxiliary transcoding capability information of each of the multiple playback devices; Based on the auxiliary transcoding capability information, the playback device with the strongest auxiliary transcoding capability among the multiple playback devices is designated as the main playback device, and the other playback devices besides the main playback device are designated as slave playback devices.

3. The method according to claim 2, characterized in that, The display-related information includes at least the playback device's resolution, color space information, frame rate, intelligent attributes, and computing power information; wherein, the calculation formula for the auxiliary transcoding capability information is expressed as follows: in, For a moment t Next, the i The auxiliary transcoding capability information of each playback device; For the first i The intelligent attributes of each playback device; For the first i The computing power information of each playback device; For the first i The resolution of each playback device. The width in pixels. The height is the number of pixels; For the first i The frame rate of each playback device; The minimum supported color gamut of the color space used by the video resource to be played; This is a constant used to compress and expand the range of values ​​to ensure a consistent range. For the first i The color space information of each playback device; The target resolution width is the target resolution width in the target resolution after the video to be played is transcoded. The target resolution height is the target resolution height in the target resolution after the video to be played is transcoded.

4. The method according to claim 1, characterized in that, The first transcoding parameters of the main bitstream include: the bitrate of the main bitstream, the frame rate of the main bitstream, the color space information of the main bitstream, and the resolution of the main bitstream; The step of determining the first transcoding parameters of the main bitstream based on the display-related information of the secondary playback device, the network information of the primary playback device, and the resource information of the video to be played includes: The bitrate of the main stream is determined based on the resolution, frame rate, device intelligence level, network information of the main playback device, and resource information of the video to be played, as well as the display-related information of the playback device. The frame rate of the main stream is determined based on the maximum supported frame rate of the current playback device, the frame rate in the display-related information of the playback device, the target resolution of the transcoded video to be played, and the bitrate in the second transcoding parameters of the unique bitstream of the playback device under the exclusive viewing behavior; the second transcoding parameters are determined based on the network information of the current playback device, the behavior information of the target object, and the resource information of the video to be played. The color space information of the main stream is determined based on the color space information in the display-related information of the playback device, the color space information used by the video to be played, and the color space information used by the current playback device. Based on the resolution of the video to be played, the target resolution of the target object, and the resolution in the display-related information of the playback device, the average effective resolution applicable to the multiple playback devices is determined as the resolution of the main bitstream.

5. The method according to claim 1, characterized in that, The step of determining the possibility information of automatic transcoding by the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device, includes: Based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, as well as the type, video high-frequency distribution ratio and video dynamic factor extracted from the resources of the video to be played, the bitstream conversion matrix from the playback device is determined. The bitstream conversion matrix is ​​a conversion expansion or compression matrix used when converting from the main bitstream. Based on the bitstream conversion matrix from the playback device and the device intelligence level of the playback device, the possibility information of the playback device's self-transcoding is determined.

6. The method according to claim 5, characterized in that, The step of determining the bitstream conversion matrix from the playback device based on the difference between the visual parameters in the display-related information and the visual parameters in the first transcoding parameters of the main bitstream, as well as the type extracted from the resources of the video to be played, the video high-frequency distribution ratio, and the video dynamic factor, includes: Based on the resolution in the visual parameters of the first transcoding parameters and the resolution in the visual parameters of the display-related information, the number of rows and columns of the bitstream conversion matrix are determined; The bitstream conversion matrix from the playback device is initialized based on the number of rows and columns of the bitstream conversion matrix. Based on the video type, high-frequency distribution ratio, and dynamic factor extracted from the video resources to be played, the initialized bitstream conversion matrix is ​​optimized to obtain the bitstream conversion matrix from the playback device.

7. The method according to claim 5 or 6, characterized in that, The high-frequency distribution ratio of the video is obtained in the following way: in, The high-frequency distribution ratio of the video; The width of the video to be played is its resolution. The resolution height of the video to be played; The number of sample frames extracted from the resources of the video to be played; For the original image, the first i The high-frequency image obtained by using a high-pass filter for each sampled frame is located in the coordinate system. The color value of the location; Indicates the original image number 1 i Each sampled frame in coordinates The color value of the location.

8. The method according to claim 5 or 6, characterized in that, The video dynamic factor is obtained in the following way: in, The video dynamic factor; The width of the video to be played is its resolution. The resolution height of the video to be played; The number of sample frames extracted from the resources of the video to be played; For the first i Each sampled frame is located at coordinates relative to the previous frame. Pixel differences in location.

9. The method according to claim 5 or 6, characterized in that, The formula for calculating the possibility information of automatic transcoding from the playback device is as follows: in, For the first i Information regarding the possibility of automatic transcoding from the playback device; For the first i The intelligent attributes derived from the device intelligence level of the playback device; For the first i The resolution width of each playback device. For the first i The resolution height of each playback device; The resolution width of the main bitstream. The resolution height of the main bitstream; For the first i The device intelligence level of the playback device includes whether it is an operation execution device parameter, where the operation execution device refers to the playback device that interacts with the target object; For the first i Computing power information of each playback device; For the first i The frame rate of each playback device; The bitrate of the main bitstream; The number of channels in the color space of the main bitstream; For the first i A bitstream conversion matrix for each playback device.

10. A video processing method, characterized in that, The method is performed by a playback device, and the method includes: The main bitstream is obtained by transcoding the video to be played based on a first transcoding parameter. The first transcoding parameter is determined based on the display-related information of the secondary playback device, the network information of the primary playback device, and the resource information of the video to be played. The primary playback device and the secondary playback device are determined by dividing the multiple playback devices according to the display-related information of multiple playback devices associated with the target object when performing non-independent viewing behavior. Obtain the self-transcoding probability information of the playback device; the self-transcoding probability information is determined based on the difference between the visual parameters in the display-related information of the playback device and the visual parameters in the first transcoding parameters of the main bitstream, combined with the device intelligence level of the playback device, and the playback device is any one of the playback devices; If the self-transcoding probability information of the playback device does not meet the self-transcoding conditions, the main bitstream is used for playback; or, if the self-transcoding probability information of the playback device meets the self-transcoding conditions, the bitstream conversion matrix of the playback device is obtained, and the main bitstream is converted using the bitstream conversion matrix to obtain the auxiliary bitstream of the playback device.

11. The method according to claim 10, characterized in that, The method further includes: If, based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, it is determined that frame interpolation processing of the auxiliary bitstream is not required, then the auxiliary bitstream is used for playback; or... If, based on the frame interpolation capability information and / or frame interpolation necessity information of the playback device, it is determined that frame interpolation processing is required for the auxiliary bitstream, the computational power consumption ratio when using the motion compensation frame interpolation algorithm for frame interpolation is obtained. If the computational power consumption ratio meets a preset condition, the auxiliary bitstream is played after frame interpolation using the motion compensation frame interpolation algorithm; or, if the computational power consumption ratio does not meet the preset condition, the auxiliary bitstream is played after frame interpolation using the linear frame interpolation algorithm.

12. An electronic device, characterized in that, include: One or more processors; The processor is configured to invoke instructions to cause the electronic device to perform the method of any one of claims 1-9 and 10-11.

13. A storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method of any one of claims 1-9 and 10-11.

14. A program product comprising at least one of a program and instructions, characterized in that, When at least one of the programs or instructions is executed by an electronic device, it implements the steps of the method according to any one of claims 1-9 and 10-11.

Images