Video transcoding method, transcoding task deployment method and related device

Abstract

The invention discloses a video transcoding method, a transcoding task deployment method and a related device, and belongs to the technical field of networks. The video transcoding method comprises the following steps: receiving a first video code stream sent by a video sending end, and decoding the first video code stream to obtain a reconstructed video signal, the reconstructed video signal comprising a plurality of reconstructed video frames; acquiring equipment state information of a video sending end or video frame transcoding information of the first video code stream; based on the device state information or the video frame transcoding information, determining a transcoding resource adjustment strategy of a first reconstructed video frame in the plurality of reconstructed video frames, the transcoding resource adjustment strategy indicating a transcoding server to adjust resources used for encoding the first reconstructed video frame, so that the total resource occupation amount of the transcoding server is within a preset range; transcoding the plurality of reconstructed video frames based on the transcoding resource adjustment strategy of the plurality of reconstructed video frames to obtain a second video code stream; and sending the second video code stream. Therefore, the quality of the transcoding service is improved.

Description

[0001] This application claims priority to Chinese patent application No. 202310735897.1, filed on June 20, 2023, and entitled “A video transcoding method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of network technology, and in particular to a video transcoding method, a transcoding task deployment method, and related devices. Background Art

[0003] Video transcoding is used to convert video streams from one format to another to adapt to different network bandwidths, different device processing capabilities, and different user needs.

[0004] In the related art, the video transmitter transmits the encoded first video stream to the transcoding service in the cloud. The transcoding service generates multiple transcoding tasks based on the first video stream according to multiple preset transcoding templates. Each transcoding task corresponds to a transcoding template, and each transcoding template corresponds to a video format. The multiple transcoding tasks are assigned to multiple transcoding servers. When the transcoding server executes the transcoding task, it first decodes the first video stream to obtain a reconstructed video signal, and then transcodes the video signal according to the transcoding parameters such as the encoding format, resolution, bit rate, etc. in the transcoding template corresponding to the transcoding task to obtain the second video stream. After the transcoding is completed, the transcoding service distributes the multiple second video streams obtained by transcoding to different video playback terminals to adapt to different playback terminals and different network environments.

[0005] However, when the transcoding server transcodes the first video code stream according to the preset transcoding template, in the case of a sudden change in the video picture, the transcoding resource usage of the transcoding server may be too high or too low, thereby affecting the quality of the video transcoding service. Summary of the invention

[0006] The present application provides a video transcoding method, a transcoding task deployment method and related devices, which can adaptively transcode a video stream so that the total resource usage of the transcoding server is stabilized within a preset range. The technical solution is as follows:

[0007] In a first aspect, a video transcoding method is provided, which is applied to a transcoding server, and the method includes:

[0008] Receive a first video code stream sent by a video transmitter; decode the first video code stream to obtain a reconstructed video signal, wherein the reconstructed video signal includes multiple reconstructed video frames; obtain device status information of the video transmitter or video frame transcoding information of the first video code stream; wherein the device status information includes one or more of coding resource occupancy, device temperature change, and device power consumption change, and the video frame transcoding information includes one or more of motion vectors, residual information, texture complexity, transcoding format, and transcoding bit rate of the multiple reconstructed video frames; determine a transcoding resource adjustment strategy for a first reconstructed video frame among the multiple reconstructed video frames based on the device status information or the video frame transcoding information, wherein the transcoding resource adjustment strategy instructs the transcoding server to adjust the resources used when encoding the first reconstructed video frame so that the total resource occupancy of the transcoding server is within a preset range; based on the transcoding resource adjustment strategy of the multiple reconstructed video frames, transcode the multiple reconstructed video frames to obtain a second video code stream; and send the second video code stream.

[0009] It can be seen that when the transcoding server performs the transcoding task according to the transcoding template or the customized transcoding parameters to transcode the first video code stream sent by the video sending end, the transcoding server can determine the transcoding resource adjustment strategy of the transcoding server when encoding each reconstructed video frame based on the device status information of the video sending end in the process of generating the first video code stream and the video frame transcoding information of the first video code stream, thereby transcoding multiple reconstructed video frames based on the transcoding resource adjustment strategy corresponding to each reconstructed video frame to obtain the second video code stream. The transcoding resource adjustment strategy instructs the transcoding server to adjust the transcoding resources used when encoding the reconstructed video frames, that is, the transcoding server adaptively transcodes each reconstructed video frame based on the total amount of resources occupied when encoding each reconstructed video frame, so as to ensure that the total amount of resources occupied by the transcoding server is within a preset range, avoid the problem of excessive or low resource occupation of the transcoding server, and improve the quality of the transcoding service and the resource utilization of the transcoding server.

[0010] Optionally, determining a transcoding resource adjustment strategy for a first reconstructed video frame among the multiple reconstructed video frames based on the device status information or the video frame transcoding information includes: estimating a total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information or the video frame transcoding information; and determining a transcoding resource adjustment strategy for the first reconstructed video frame based on the total amount of resources occupied.

[0011] Since the device status information or the video frame transcoding information can directly or indirectly reflect the encoding resource consumption of each original video frame in the first video code stream, the change in transcoding resources when the transcoding server encodes each reconstructed video frame can be estimated based on the device status information or the video frame transcoding information, and then based on the change in the transcoding resources of the transcoding server, it is determined whether the total resource occupation of the transcoding server is within a preset range. Moreover, when the total resource occupation of the transcoding server encoding the reconstructed video frame is not within the preset range, the transcoding resource adjustment strategy when encoding the reconstructed video frame is determined, and adaptive encoding is performed according to the transcoding resource adjustment strategy, so that the total resource occupation of the transcoding server can be stabilized within the preset range.

[0012] Optionally, estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information or the video frame transcoding information includes: inputting the device status information or the video frame transcoding information into a trained transcoding resource estimation model to obtain a change in transcoding resources output by the transcoding resource estimation model; and estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the change in transcoding resources and the current resource occupancy of the transcoding server.

[0013] That is, based on the device status information or the video frame transcoding information, the trained transcoding resource estimation model is used to estimate the change in the server's transcoding resources. In this way, the learning characteristics of the neural network model and the advantages of processing data are used to accurately estimate the change in the transcoding resources when the transcoding server encodes the first reconstructed video frame, thereby improving the estimation accuracy of the transcoding resources.

[0014] Optionally, estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information includes: determining a change in encoding resources of the video sending end during the encoding of a first original video frame based on the device status information, the first original video frame corresponding to the first reconstructed video frame; if the change in encoding resources is greater than a coding resource threshold, estimating a change in transcoding resources when the transcoding server encodes the first reconstructed video frame based on the change in encoding resources; estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the change in transcoding resources and the current resource occupancy of the transcoding server.

[0015] Since the device status information can reflect the consumption of encoding resources in the process of the video transmitter generating the first video code stream, the change in encoding resources when the video transmitter encodes the first original video frame corresponding to the first reconstructed video frame can be determined based on the device status information. Then, based on the change in encoding resources when the video transmitter encodes the first original video frame, the change in transcoding resources when the transcoding server encodes the first reconstructed video frame is estimated. Finally, based on the sum of the change in transcoding resources when the transcoding server encodes the first reconstructed video frame and the current resource occupancy of the transcoding server, the total resource occupancy when the transcoding server encodes the first reconstructed video frame is determined.

[0016] Optionally, estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the video frame transcoding information includes: inputting the video frame transcoding information corresponding to the first reconstructed video frame into a trained transcoding complexity estimation model to obtain the video frame transcoding complexity output by the transcoding complexity estimation model; if the video frame transcoding complexity is greater than a complexity threshold, estimating a change in transcoding resources when the transcoding server encodes the first reconstructed video frame based on the video frame transcoding complexity; estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the change in transcoding resources and the current resource occupancy of the transcoding server.

[0017] Since the video frame transcoding information is the information obtained by decoding the first video code stream and the information related to transcoding, based on the video frame transcoding information, the video frame transcoding complexity when the first reconstructed video frame is transcoded can be estimated. Then, based on the video frame transcoding complexity, the change in transcoding resources when the transcoding server encodes the first reconstructed video frame is estimated. Finally, based on the sum of the change in transcoding resources and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is determined.

[0018] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; the transcoding resource adjustment strategy of the first reconstructed video frame is determined based on the total resource occupancy, including: if the total resource occupancy is greater than the upper limit of resource occupancy, then the transcoding resource adjustment strategy of the first reconstructed video frame is determined to be the first transcoding strategy; if the total resource occupancy is less than the lower limit of resource occupancy, then the transcoding resource adjustment strategy of the first reconstructed video frame is determined to be the second transcoding strategy.

[0019] The upper limit of resource occupancy and the lower limit of resource occupancy jointly define a preset range. When the total resource occupancy of the transcoding server is within the preset range, that is, the total resource occupancy is greater than or equal to the lower limit of resource occupancy, and the total resource occupancy is less than or equal to the upper limit of resource occupancy, the transcoding server does not need to adjust the transcoding resources used when encoding the first reconstructed video frame, and continues to transcode the first reconstructed video frame according to the transcoding parameters contained in the transcoding template. When the total resource occupancy of the transcoding server is not within the preset range, that is, the total resource occupancy is less than the lower limit of resource occupancy, or greater than the upper limit of resource occupancy, in order to ensure the quality of the transcoding service and improve the resource utilization of the transcoding server, the transcoding server needs to adjust the transcoding resources used when encoding the first reconstructed video frame.

[0020] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy, and the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding reconstructed video frames; the transcoding resource adjustment strategy based on the multiple reconstructed video frames transcodes the multiple reconstructed video frames, including: encoding the first reconstructed video frame based on a first reference frame number and / or a first motion search range, the first reference frame number is the number of reference frames used when encoding the first reconstructed video frame, and the first reference frame number is smaller than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the first motion search range is the motion search range used when encoding the first reconstructed video frame, and the first motion search range is smaller than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

[0021] When encoding the first reconstructed video frame, the more reference frames are used, the more transcoding resources are consumed; similarly, when encoding the first reconstructed video frame, when determining the motion of the object in the first video frame, the larger the motion search range is used, the more transcoding resources are consumed. Therefore, when it is necessary to reduce the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, it can be achieved by reducing the reference frame data and / or narrowing the motion search range.

[0022] Optionally, the transcoding resource adjustment strategy includes a second transcoding strategy, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding the reconstructed video frames; the transcoding resource adjustment strategy based on the multiple reconstructed video frames transcodes the multiple reconstructed video frames, including: encoding the first reconstructed video frame based on a second reference frame number and / or a second motion search range, the second reference frame number is the number of reference frames used when encoding the first reconstructed video frame, and the second reference frame number is greater than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the second motion search range is the motion search range used when encoding the first reconstructed video frame, and the second motion search range is greater than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

[0023] That is, when it is necessary to increase the amount of transcoding resources used by the transcoding server to encode the first reconstructed video frame, this can be achieved by increasing reference frame data and / or expanding the motion search range.

[0024] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; the method also includes: if the transcoding resource adjustment strategy is the first transcoding strategy, increasing the main frequency of the processor in the transcoding server; if the transcoding resource adjustment strategy is the second transcoding strategy, reducing the main frequency of the processor in the transcoding server.

[0025] That is, by adjusting the transcoding performance of the transcoding server, the total resource usage of the transcoding server is within a preset range. Increasing the main frequency of the processor in the transcoding server can enhance the processing capability of the transcoding server, thereby reducing the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, so that the total resource usage of the transcoding server can be within a preset range; reducing the main frequency of the processor in the transcoding server can reduce the processing capability of the transcoding server, thereby increasing the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, so that the total resource usage of the transcoding server can be within a preset range.

[0026] In a second aspect, a transcoding task deployment method is provided, which is applied to a transcoding task deployment device, and the method includes:

[0027] Receive a first video code stream sent by a video sending end, and device status information of the video sending end during the generation of the first video code stream, the device status information including one or more of encoding resource occupancy, device temperature change, and device power consumption change; based on the device status information, determine the amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, and obtain at least one transcoding resource amount; based on the at least one transcoding resource amount, determine at least one transcoding server, each of the at least one transcoding server is used to transcode the first video code stream according to a corresponding transcoding template to obtain a second video code stream, and the total amount of resource occupancy of the transcoding server when transcoding the first video code stream is within a preset range.

[0028] It can be seen that when the transcoding task deployment device deploys the transcoding task of the first video code stream in the server cluster, it can use the device status information of the video sending end during the generation process of the first video code stream to estimate the amount of transcoding resources required when transcoding the first video code stream according to at least one transcoding template, and then determine at least one transcoding server from the server cluster that transcodes the first video code stream according to at least one transcoding template based on the transcoding resource amount and the current resource occupancy of each server. Since the amount of transcoding resources is determined based on the device status information, the amount of transcoding resources can reflect the actual resource consumption of the server when transcoding the first video code stream. Therefore, the deployment of transcoding tasks based on the amount of transcoding resources improves the deployment accuracy of transcoding tasks. At the same time, it avoids the situation where the total amount of resource occupancy of the transcoding server is too high or too low after the transcoding task is deployed, thereby improving the quality of transcoding services and the resource utilization of the transcoding server.

[0029] Optionally, determining the amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template based on the device status information includes: determining a reference amount of resources corresponding to the first transcoding template based on the device status information and transcoding parameters included in the first transcoding template, the first transcoding template being any one of the at least one transcoding template; and determining the reference amount of resources corresponding to the first transcoding template as the amount of transcoding resources required when transcoding the first video code stream according to the first transcoding template.

[0030] That is, for any server in the server cluster, the amount of transcoding resources when the server transcodes the first video stream according to the first transcoding template is equal to the reference resource amount of the first transcoding template, and the total resource occupancy of the server when transcoding the first video stream according to the first transcoding template is equal to the sum of the reference resource amount of the first transcoding template and the transcoding resource occupancy of the server.

[0031] Optionally, determining at least one transcoding server based on the at least one transcoding resource amount includes: obtaining the current resource occupancy of each server in the server cluster; determining the total resource occupancy of each server when transcoding the first video stream according to the at least one transcoding template based on the current resource occupancy of each server and the at least one transcoding resource amount; and determining a server in the server cluster whose total resource occupancy is less than a transcoding resource threshold as the at least one transcoding server.

[0032] That is, based on the current total resource usage of the server and the amount of transcoding resources required for each transcoding task, the transcoding server on which the transcoding task can be deployed is determined, thereby improving the deployment accuracy of the transcoding task, avoiding the situation where the total resource usage of the transcoding server is too high or too low after the transcoding task is deployed, and improving the quality of the transcoding service and the resource utilization of the transcoding server.

[0033] In a third aspect, a video transcoding device is provided, wherein the video transcoding device has the function of implementing the video transcoding method in the first aspect. The video transcoding device includes at least one module, and the at least one module is used to implement the video transcoding method provided in the first aspect.

[0034] In a fourth aspect, a transcoding task deployment device is provided, wherein the transcoding task deployment device has the function of implementing the transcoding task deployment method behavior in the second aspect. The transcoding task deployment device includes at least one module, and the at least one module is used to implement the transcoding task deployment method provided in the second aspect.

[0035] In a fifth aspect, a computing device cluster is provided, the computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory;

[0036] The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the video transcoding method described in the first aspect above, or executes the transcoding task deployment method described in the second aspect above.

[0037] In a sixth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the video transcoding method described in the first aspect, or executes the transcoding task deployment method described in the second aspect.

[0038] In a seventh aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the video transcoding method described in the first aspect above, or executes the transcoding task deployment method described in the second aspect above.

[0039] The technical effects obtained in the above-mentioned third to seventh aspects are similar to the technical effects obtained by the corresponding technical means in the first and second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the architecture of a video transcoding system provided in an embodiment of the present application;

[0041] Figure 2 It is a flowchart of a video transcoding method provided in an embodiment of the present application;

[0042] Figure 3 It is a flowchart of a transcoding task deployment method provided in an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of the architecture of another video transcoding system provided in an embodiment of the present application;

[0044] Figure 5 is a schematic diagram of a video adaptive transcoding process provided by an embodiment of the present application;

[0045] Figure 6 is a schematic diagram of another video adaptive transcoding process provided by an embodiment of the present application;

[0046] Figure 7 It is a schematic diagram of a transcoding task deployment process provided by an embodiment of the present application;

[0047] Figure 8 It is a flow chart of a video transcoding device provided in an embodiment of the present application;

[0048] Fig. 9 It is a flow chart of a transcoding task deployment device provided in an embodiment of the present application;

[0049] Fig.10 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0050] Fig.11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0051] Fig.12 It is a schematic diagram of the network connection relationship of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0053] Before explaining in detail the video transcoding method and transcoding task deployment method provided in the embodiments of the present application, the application scenarios of the embodiments of the present application are first introduced.

[0054] Video transcoding is used to convert video streams from one format to another to adapt to different network bandwidths, different device processing capabilities, and different user needs. Among them, the transcoding service is built on cloud computing services, which can save users the high investment of purchasing, building, and managing transcoding software and hardware, and avoid complex issues such as configuration optimization and transcoding parameter adaptation; at the same time, with the elastic scaling characteristics of cloud services, it can meet the actual transcoding business needs.

[0055] The main process of the transcoding service is as follows: the video sender uploads the encoded first video stream to the cloud, and the cloud-based transcoding service decodes the first video stream and then encodes it into a second video stream of the corresponding format, size, and bit rate, so as to distribute the second video stream to different viewers.

[0056] Taking the live broadcast transcoding scenario as an example, the cloud-based transcoding service needs to adapt to the different video playback terminals and different network environments of a large number of viewers. It is a typical one-to-many real-time communication system. Figure 1 , the anchor acts as the video transmitter, and the anchor's camera collects the original video signal, and then encodes the original video signal into a binary first video stream through the anchor's video encoder, and the first video stream is transmitted to the transcoding service in the cloud. The console deploys the transcoding task of the first video stream in at least one transcoding server in the transcoding cluster by calling the corresponding application programming interface (API), and the transcoding server decodes the first video stream to obtain a reconstructed video signal, which includes multiple reconstructed video frames; then, according to the transcoding parameters contained in at least one transcoding template, the multiple reconstructed video frames are encoded to obtain at least one second video stream, and each second video stream corresponds to a transcoding template. In this way, the transcoded multiple second video streams can meet the video playback needs of different viewers at the video playback end.

[0057] The video transmitter and the video player communicate with the transcoding service in the cloud in a many-to-one manner, which can be implemented through a content delivery network (CDN). The transcoding service can also store the transcoded second video stream in the cloud, so as to respond to requests from different video players and send the corresponding second video stream to different video players.

[0058] The cloud-based transcoding service is specifically provided by a transcoding cluster, which includes multiple computing devices, each of which can be a server. When a first video code stream is uploaded to the transcoding service, the transcoding service generates at least one transcoding task based on the first video code stream and at least one transcoding template, and assigns the at least one transcoding task to at least one transcoding server in the transcoding cluster for execution. Among them, one transcoding task will be assigned to one transcoding server, one transcoding task corresponds to one transcoding template, and one transcoding template corresponds to one video format.

[0059] During the video transcoding process, at least one second video code stream output by the transcoding service is obtained after transcoding based on the transcoding template. When providing transcoding services to the outside, most cloud vendors meet actual transcoding needs by pre-setting general transcoding templates or custom transcoding templates.

[0060] As an example, the following Table 1 shows a general transcoding template, which can provide four transcoding templates: smooth, standard definition, high definition and ultra-high definition, and each transcoding template corresponds to a video quality; Table 2 shows the setting parameters and parameter descriptions of a custom transcoding template, which can provide users with video transcoding services with custom encoding methods, resolutions, bit rates and other specific transcoding parameters.

[0061] Table 1

[0062] Video quality level Resolution Bit rate (Kbps) Fluency (low definition, LD) 640*360 400 Standard definition (SD) 854*480 600 High Definition (HD) 1280*720 1000 Full high definition (FHD) 1920*1080 2000

[0063] When transcoding the first video code stream based on the general transcoding template shown in Table 1 above, the transcoding service will generate four transcoding tasks based on the first video code stream according to the general transcoding template, and each transcoding task corresponds to a transcoding template. Then, the transcoding service will deploy the four transcoding tasks in the transcoding cluster, so that the transcoding server that deploys the transcoding tasks will transcode the first video code stream according to the corresponding transcoding template to obtain the corresponding second video code stream.

[0064] Table 2

[0065]

[0066] Based on the custom transcoding template shown in Table 2, when transcoding the first video stream, the user can set and adjust specific transcoding parameters such as encoding mode, resolution, bit rate, etc., so that the transcoding service can provide more formats of second video streams to meet the needs of different video playback terminals. The user here can be the administrator responsible for managing the cloud transcoding service, or the viewer corresponding to any video playback terminal.

[0067] Since the transcoding parameters provided by the general transcoding template and the custom transcoding template have preset attributes and are relatively fixed, they can only provide transcoding services with fixed encoding standards, fixed resolutions, and fixed bit rates. As a result, when a transcoding server executes a transcoding task, it can only transcode according to the transcoding parameters specified by a transcoding template, and cannot flexibly switch the transcoding parameters to achieve adaptive transcoding.

[0068] Moreover, when the transcoding server performs the transcoding task according to the transcoding template, if the total resource usage of the transcoding server is within the preset range (for example, the total resource usage is greater than the lower limit of the resource usage, and the total resource usage is less than the upper limit of the resource usage), the transcoding server can work normally, and the computing power of the transcoding server can be fully utilized; if the total resource usage of the transcoding server is greater than the upper limit of the resource usage, there will be accumulated transcoding delays when the transcoding server performs the transcoding task, resulting in a decrease in the quality of the transcoding service; if the total resource usage of the transcoding server is less than the lower limit of the resource usage, the computing power of the transcoding server is not fully utilized when performing the transcoding task, resulting in a low resource utilization rate of the transcoding server. That is, when the transcoding server performs the transcoding task according to the transcoding template, if the total resource usage of the transcoding server is too high or too low, it will significantly affect the user's experience of using the transcoding service.

[0069] In other words, when transcoding the first video stream based on the universal transcoding template or the custom transcoding template preset in the cloud, at least one of the following problems may occur:

[0070] (1) During the operation of the transcoding task, since the picture content of each original video frame in the first video code stream may change dramatically, the transcoding resources used by the transcoding server when encoding the corresponding reconstructed video frame may also change suddenly. The sudden change in the amount of transcoding resources of the transcoding server may cause the total resource usage of the transcoding server to be too high or too low, and cannot be stabilized within the preset range.

[0071] (2) When the transcoding service schedules transcoding tasks among multiple servers in the transcoding cluster, it is difficult to accurately estimate the total resource usage of the server when executing each transcoding task. As a result, after the transcoding task starts running, the total resource usage of the transcoding server may be too high or too low.

[0072] In response to the above problem (1), an embodiment of the present application provides a video transcoding method. When a transcoding server performs a transcoding task of a first video code stream, it dynamically adjusts the transcoding resources used when encoding the reconstructed video frames by analyzing the device status information of the video transmitter when generating the first video code stream, or the video frame transcoding information obtained by decoding the first video code stream, so that the resource occupancy of the transcoding server is within a preset range, thereby improving the resource utilization and transcoding efficiency of the transcoding server. In response to the above problem (2), an embodiment of the present application provides a transcoding task deployment method. When performing a transcoding task of a first video code stream in a transcoding cluster, the transcoding task deployment device combines the device status information of the video transmitter during the generation of the first video code stream and the transcoding parameters contained in at least one transcoding template to accurately estimate the amount of transcoding resources required when transcoding the first video code stream according to at least one transcoding template, thereby improving the accuracy of estimating the amount of transcoding resources. Therefore, based on a more accurate amount of transcoding resources, a transcoding server for deploying transcoding tasks is determined among multiple servers, which improves the accuracy of transcoding task deployment while ensuring resource utilization when the transcoding server executes transcoding tasks.

[0073] Next, the video transcoding method provided in the embodiment of the present application is explained in detail.

[0074] Please refer to Figure 2 , Figure 2 is a flowchart of a video transcoding method provided in an embodiment of the present application. The method can be applied to a transcoding server. The transcoding server can be Figure 1 For any server in the transcoding cluster shown, the method includes the following steps.

[0075] Step 201: Receive a first video code stream sent by a video sending end.

[0076] It should be noted that the video sending end is a device that generates the first video code stream, which can be a terminal or a server, and the embodiment of the present application does not limit this. After the video sending end sends the encoded first video code stream to the transcoding service in the cloud, the transcoding service determines the transcoding server that transcodes the first video code stream from the server cluster, and the transcoding server executes the video transcoding method shown in steps 201 to 206 to adaptively transcode the first video code stream to obtain the second video code stream.

[0077] In some embodiments, the transcoding service may also first schedule the first transcoding server to decode the first video stream to obtain a reconstructed video signal; then schedule the second transcoding server to transcode the reconstructed video signal to obtain a second video stream; and then schedule the third transcoding server to distribute the second video stream to other devices. That is, steps 201-205 can be completed by one transcoding server or by multiple transcoding servers, and the embodiments of the present application do not limit this. In the embodiments of the present application, only one transcoding server is taken as an example to explain the process of adaptively transcoding the first video stream by the transcoding server according to the video transcoding method, and does not constitute a limitation on the number of transcoding servers used when executing the video transcoding method.

[0078] Step 202: Decode the first video code stream to obtain a reconstructed video signal, where the reconstructed video signal includes a plurality of reconstructed video frames.

[0079] The first video code stream is a video code stream obtained after the video transmitter encodes the original video signal, and the original video signal includes multiple original video frames, and the picture content information of each original video frame may be different. Correspondingly, after the transcoding server decodes the first video code stream, the multiple reconstructed video frames obtained correspond to the multiple original video frames one by one, that is, one reconstructed video frame corresponds to one original video frame.

[0080] It should be noted that the specific implementation process of decoding the first video code stream can be referred to the relevant prior art, and the embodiment of the present application will not elaborate on this.

[0081] Step 203: Obtain device status information of the video sending end or video frame transcoding information of the first video code stream.

[0082] In some embodiments, the device status information includes one or more of the encoding resource occupancy, the device temperature change, and the device power consumption change, and the video frame transcoding information includes one or more of the motion vectors, residual information, texture complexity, transcoding format, and transcoding bit rate of multiple reconstructed video frames.

[0083] It should be understood that the device status information directly reflects the changes in the encoding resources of the video sending end during the process of generating the first video code stream. Therefore, based on one or more of the encoding resource occupancy, device temperature change and device power consumption change corresponding to each original video frame, the encoding resource change when the video sending end encodes the original video frame is determined.

[0084] Among them, the higher the encoding resource occupancy, the more encoding resources are consumed by the video sender when encoding the original video frame, and the more transcoding resources are required when the transcoding server transcodes the reconstructed video frame corresponding to the original video frame; the lower the encoding resource occupancy, the fewer encoding resources are consumed by the video sender when encoding the original video frame, and the fewer transcoding resources are required when the transcoding server transcodes the reconstructed video frame corresponding to the original video frame.

[0085] The device temperature change and the device power consumption change can indirectly reflect the change of encoding resources when the video sender encodes the original video frame. When the device temperature change is large, or the device power consumption change is large, it means that the video sender consumes more encoding resources when encoding the original video frame; when the device temperature change is small, or the device power consumption change is small, it means that the video sender consumes fewer encoding resources when encoding the original video frame.

[0086] In a possible implementation, the implementation method for obtaining the device status information may be: when receiving the first video code stream sent by the video sending end, the device status information in the process of generating the first video code stream is obtained from the video sending end. Alternatively, after receiving the first video code stream, based on the video identifier of the first video code stream, the device status information in the process of generating the first video code stream is obtained from the video sending end. The embodiments of the present application are not limited to this.

[0087] It should be understood that the video frame transcoding information is the video frame information determined by the transcoding server based on the transcoding requirements after decoding the first video code stream. Therefore, the video frame transcoding information corresponding to each reconstructed video frame can indirectly reflect the encoding resource consumption of the video transmitter when encoding the corresponding original video frame, and / or the transcoding resource consumption when transcoding the reconstructed video frame. Each reconstructed video frame corresponds to a motion vector, residual information, texture complexity, transcoding format and transcoding bit rate.

[0088] Among them, motion vectors, residual information and texture complexity can indirectly reflect the encoding resource consumption of the video sender. The more encoding resources the video sender consumes, the more transcoding resources the transcoding server needs for transcoding. Similarly, the fewer encoding resources the video sender consumes, the fewer transcoding resources the transcoding server needs for transcoding.

[0089] The larger the motion vector, the larger the residual information, and the greater the texture complexity, the more encoding resources the video sender consumes when encoding the original video frame, and the more transcoding resources the transcoding server requires when transcoding the reconstructed video frame corresponding to the original video frame; the smaller the motion vector, the smaller the residual information, and the smaller the texture complexity, the fewer encoding resources the video sender consumes when encoding the original video frame, and the fewer transcoding resources the transcoding server requires when transcoding the reconstructed video frame corresponding to the original video frame.

[0090] Among them, the transcoding format and transcoding bit rate can directly reflect the transcoding resources required by the transcoding server for transcoding the reconstructed video frames. The more complex the transcoding format and the higher the transcoding bit rate, the more transcoding resources are required; the simpler the transcoding format and the lower the transcoding bit rate, the fewer transcoding resources are required.

[0091] In a possible implementation, the implementation method for obtaining the video frame transcoding information may be: after decoding the first video code stream, based on the reconstructed video signal, determine the motion vector, residual information and texture complexity of each reconstructed video frame. Based on the transcoding template used by the transcoding server when transcoding the first video code stream, determine the transcoding format and transcoding bit rate used when transcoding multiple reconstructed video frames. The motion vectors, residual information, texture complexity, transcoding format and transcoding bit rate of the multiple reconstructed video frames are used to determine the video frame transcoding information of the first video code stream.

[0092] Step 204: Based on the device status information or the video frame transcoding information, determine the transcoding resource adjustment strategy for the first reconstructed video frame among the multiple reconstructed video frames, the transcoding resource adjustment strategy instructs the transcoding server to adjust the resources used when encoding the first reconstructed video frame so that the total resource occupancy of the transcoding server is within a preset range.

[0093] It should be noted that, since the device status information or video frame transcoding information can directly or indirectly reflect the encoding resource consumption of each original video frame in the first video code stream, it is possible to estimate the change in transcoding resources when the transcoding server encodes each reconstructed video frame when transcoding each reconstructed video frame based on the device status information or video frame transcoding information, and then determine whether the total resource usage of the transcoding server is within a preset range based on the change in transcoding resources of the transcoding server. Moreover, when the total resource usage of the transcoding server encoding the reconstructed video frame is not within the preset range, determine the transcoding resource adjustment strategy when encoding the reconstructed video frame, and perform adaptive encoding according to the transcoding resource adjustment strategy, so that the total resource usage of the transcoding server can be stabilized within the preset range.

[0094] The preset range may be: the total resource occupation is greater than or equal to the lower limit of the resource occupation, and the total resource occupation is less than or equal to the upper limit of the resource occupation.

[0095] Since the process of determining the transcoding resource adjustment strategies corresponding to multiple reconstructed video frames is the same, the specific process of determining the transcoding resource adjustment strategy is introduced by taking one of the reconstructed video frames as an example. For ease of description, the reconstructed video frame is referred to as the first reconstructed video frame.

[0096] In some embodiments, the implementation process of the above step 204 may include the following steps (1)-(2).

[0097] (1) Based on the device status information or the video frame transcoding information, the total resource usage of the transcoding server when encoding the first reconstructed video frame is estimated.

[0098] Since the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame can be estimated based on the device state information or the video frame transcoding information, the implementation methods for estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame are different based on different information.

[0099] The first implementation method is to input the device status information or video frame transcoding information into a trained transcoding resource estimation model to obtain the transcoding resource change output by the transcoding resource estimation model; based on the transcoding resource change and the current resource occupancy of the transcoding server, estimate the total resource occupancy of the transcoding server when encoding the first reconstructed video frame.

[0100] The transcoding resource change indicates the change in the transcoding resources of the transcoding server when the transcoding server performs transcoding processing on the first reconstructed video frame, and directly reflects the transcoding resources required by the transcoding server when the transcoding server performs transcoding processing on the first reconstructed video frame. After determining the transcoding resources required for the first reconstructed video frame, the total resource usage of the transcoding server when the transcoding server performs transcoding processing on the first reconstructed video frame can be estimated.

[0101] In this implementation, based on the device status information or the video frame transcoding information, the trained transcoding resource estimation model is used to estimate the change in the server's transcoding resources. In this way, the learning characteristics of the neural network model and the advantages of processing data are used to accurately estimate the change in the transcoding resources when the transcoding server encodes the first reconstructed video frame, thereby improving the estimation accuracy of the transcoding resources.

[0102] The second implementation method is to estimate the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information. The implementation process is as follows: based on the device status information, determine the change in encoding resources of the video sending end during the encoding process of the first original video frame, and the first original video frame corresponds to the first reconstructed video frame; if the change in encoding resources is greater than the encoding resource threshold, then based on the change in encoding resources, estimate the change in transcoding resources when the transcoding server encodes the first reconstructed video frame; based on the change in transcoding resources and the current resource occupancy of the transcoding server, determine the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame.

[0103] That is, since the device status information can reflect the consumption of encoding resources in the process of the video transmitter generating the first video code stream, the change in encoding resources when the video transmitter encodes the first original video frame corresponding to the first reconstructed video frame can be determined based on the device status information. Then, based on the change in encoding resources when the video transmitter encodes the first original video frame, the change in transcoding resources when the transcoding server encodes the first reconstructed video frame is estimated. Finally, based on the sum of the change in transcoding resources when the transcoding server encodes the first reconstructed video frame and the current resource occupancy of the transcoding server, the total resource occupancy when the transcoding server encodes the first reconstructed video frame is determined.

[0104] Among them, when determining the change in encoding resources for encoding the first original video frame at the video sending end, and determining the change in transcoding resources for encoding the first reconstructed video frame at the transcoding server, the resource change can be estimated based on the trained neural network model, or the resource change can be calculated through a related algorithm. The embodiments of the present application do not impose any restrictions on this.

[0105] As an example, the device state information corresponding to the first original video frame is input into the trained first resource estimation model, and the first resource estimation model is used to determine the amount of change in encoding resources when the video transmitter encodes the first original video frame. The amount of change in encoding resources when the video transmitter encodes the first original video frame is input into the trained second resource estimation model, and the second resource estimation model is used to determine the amount of change in transcoding resources when the transcoding server encodes the first reconstructed video frame.

[0106] It should be noted that when the amount of change in coding resources is greater than the coding resource threshold, the amount of coding resources used by the video transmitter when encoding the first original video frame may have a sudden change, causing the amount of transcoding resources used by the transcoding server when transcoding the first reconstructed video frame to also mutate, and the sudden change in the amount of transcoding resources may cause the total amount of resources occupied by the transcoding server to exceed the preset range. Therefore, when the amount of change in coding resources is greater than the coding resource threshold, in order to ensure that the total amount of resources occupied by the transcoding server can be stabilized within the preset range, the transcoding server needs to dynamically adjust the transcoding resources used when transcoding the first reconstructed video frame so that the total amount of resources occupied by the transcoding server can be stabilized within the preset range.

[0107] When the total resource usage of the transcoding server is stable within a preset range, the transcoding quality and server performance of the transcoding server are not affected, and the transcoding operation can be performed normally. When the amount of change in encoding resources is less than or equal to the encoding resource threshold, it means that the amount of encoding resources consumed by the video sender when encoding the first original video frame has changed slightly. Although this change will cause the amount of transcoding resources used by the transcoding server to change during transcoding, this change in the amount of transcoding resources will not cause the total resource usage of the transcoding server to exceed the preset range. Therefore, when the amount of change in encoding resources is less than or equal to the encoding resource threshold, there is no need for the transcoding server to adjust the transcoding resources used when transcoding the first reconstructed video frame. The transcoding server continues to transcode the first reconstructed video frame according to the transcoding parameters contained in the transcoding template.

[0108] The third implementation method is to estimate the total amount of resources occupied by the server when encoding the first reconstructed video frame based on the video frame transcoding information. The implementation process is as follows: input the video frame transcoding information corresponding to the first reconstructed video frame into the trained transcoding complexity estimation model to obtain the video frame transcoding complexity output by the transcoding complexity estimation model; if the video frame transcoding complexity is greater than the complexity threshold, based on the video frame transcoding complexity, estimate the change in transcoding resources when the transcoding server encodes the first reconstructed video frame; based on the change in transcoding resources and the current resource occupancy of the transcoding server, estimate the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame.

[0109] That is, since the video frame transcoding information is the information obtained by decoding the first video code stream and the information related to transcoding, based on the video frame transcoding information, the video frame transcoding complexity when the first reconstructed video frame is transcoded can be estimated. Then, based on the video frame transcoding complexity, the change in transcoding resources when the transcoding server encodes the first reconstructed video frame is estimated. Finally, based on the sum of the change in transcoding resources and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is determined.

[0110] It should be understood that the higher the complexity of video frame transcoding, the greater the amount of transcoding resources that the transcoding server needs to use when encoding the first reconstructed video frame, that is, the greater the change in transcoding resources when the transcoding server encodes the first reconstructed video frame. Conversely, the lower the complexity of video frame transcoding, the smaller the amount of transcoding resources that the transcoding server needs to use when encoding the first reconstructed video frame, that is, the smaller the change in transcoding resources when the transcoding server encodes the first reconstructed video frame.

[0111] When determining the amount of transcoding resource change when the transcoding server encodes the first reconstructed video frame, the resource change can be estimated based on a trained neural network model, or the resource change can be calculated by a related algorithm, which is not limited in this embodiment of the present application.

[0112] As an example, the video frame transcoding complexity of the first reconstructed video frame is input into a trained third resource estimation model, and the third resource estimation model is used to determine the transcoding resource change when the transcoding server encodes the first reconstructed video frame.

[0113] It should be noted that when the video frame transcoding complexity is greater than the complexity threshold, the amount of transcoding resources required by the transcoding server when encoding the first reconstructed video frame may suddenly change, thereby causing the total resource usage of the transcoding server to exceed the preset range. Therefore, when the video frame transcoding complexity is greater than the complexity threshold, in order to ensure that the total resource usage of the transcoding server is within the preset range, the transcoding server needs to dynamically adjust the transcoding resources used when transcoding the first reconstructed video frame, so that the total resource usage of the transcoding server can be stabilized within the preset range.

[0114] When the total resource usage of the transcoding server can be stabilized within a preset range, the transcoding quality and server performance of the transcoding server will not be affected, and the transcoding operation can be performed normally. Therefore, when the video frame transcoding complexity is less than or equal to the complexity threshold, the amount of transcoding resources used by the transcoding server to encode the first reconstructed video frame will change, but this slight change will not cause the total resource usage of the transcoding server to exceed the preset range. Therefore, when the video frame transcoding complexity is less than or equal to the complexity threshold, the transcoding server does not need to adjust the transcoding resources used when transcoding the first reconstructed video frame, and the transcoding server continues to transcode the first reconstructed video frame according to the transcoding parameters contained in the transcoding template.

[0115] (2) Based on the total amount of resource usage, determine a transcoding resource adjustment strategy for the first reconstructed video frame.

[0116] Among them, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy. The first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames.

[0117] In one possible implementation, based on the total amount of resource occupancy, the implementation process of determining the transcoding resource adjustment strategy for the first reconstructed video frame is as follows: if the total amount of resource occupancy is greater than the upper limit of resource occupancy, then the transcoding resource adjustment strategy for the first reconstructed video frame is determined to be the first transcoding strategy; if the total amount of resource occupancy is less than the lower limit of resource occupancy, then the transcoding resource adjustment strategy for the first reconstructed video frame is determined to be the second transcoding strategy.

[0118] It should be noted that if the total resource occupancy is less than or equal to the upper limit of resource occupancy, and greater than or equal to the lower limit of resource occupancy, there is no need to adjust the transcoding resources used when encoding the first reconstructed video frame. The transcoding server continues to transcode the first reconstructed video frame according to the transcoding parameters contained in the transcoding template.

[0119] That is, the upper limit of resource occupancy and the lower limit of resource occupancy jointly define a preset range. When the total resource occupancy of the transcoding server is within the preset range, that is, the total resource occupancy is greater than or equal to the lower limit of resource occupancy, and the total resource occupancy is less than or equal to the upper limit of resource occupancy, the transcoding server does not need to adjust the transcoding resources used when encoding the first reconstructed video frame, and continues to transcode the first reconstructed video frame according to the transcoding parameters contained in the transcoding template. When the total resource occupancy of the transcoding server is not within the preset range, that is, the total resource occupancy is less than the lower limit of resource occupancy, or greater than the upper limit of resource occupancy, in order to ensure the quality of the transcoding service and improve the resource utilization of the transcoding server, the transcoding server needs to adjust the transcoding resources used when encoding the first reconstructed video frame.

[0120] Step 205: Based on the transcoding resource adjustment strategy of the multiple reconstructed video frames, transcoding the multiple reconstructed video frames to obtain a second video code stream.

[0121] It should be noted that when the embodiment of the present application adjusts the amount of transcoding resources used when encoding the first reconstructed video frame, it can be achieved by adjusting the transcoding parameters of the first reconstructed video frame, or by adjusting the transcoding performance of the transcoding server. The embodiment of the present application does not impose any restrictions on this.

[0122] When transcoding the reconstructed video frame, under the same transcoding resource adjustment strategy, the process of transcoding the reconstructed video frame is the same. Next, taking the first reconstructed video frame as an example, the process of transcoding the first reconstructed video frame based on the first transcoding strategy or the second transcoding strategy is introduced.

[0123] In the first transcoding processing method, the transcoding resource adjustment strategy includes a first transcoding strategy, and the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding the first reconstructed video frame. At this time, the first reconstructed video frame is encoded based on the first reference frame number and / or the first motion search range.

[0124] Among them, the first number of reference frames is the number of reference frames used when encoding the first reconstructed video frame, and the first number of reference frames is smaller than the number of reference frames corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame, and the first motion search range is the motion search range used when encoding the first reconstructed video frame, and the first motion search range is smaller than the motion search range corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame.

[0125] Optionally, if a default number of reference frames and a default motion search range for encoding reconstructed video frames are preset, when the transcoding resource adjustment strategy is determined to be the first transcoding strategy, that is, when the amount of transcoding resources used in encoding the first reconstructed video frame needs to be reduced, the first number of reference frames used in encoding the first reconstructed video frame is less than the default number of reference frames, and the first motion search range is less than the default motion search range.

[0126] That is, when it is necessary to reduce the amount of transcoding resources used in encoding the first reconstructed video frame, the number of reference frames used in encoding the first reconstructed video frame can be reduced relative to the number of reference frames and the motion search range used in encoding the previous reconstructed video frame, and the motion search range used in encoding the first reconstructed video frame can be reduced; the number of reference frames used in encoding the first reconstructed video frame can also be reduced directly based on the default number of reference frames and the default motion search range, and the motion search range used in encoding the first reconstructed video frame can be reduced. The embodiments of the present application are not limited to this.

[0127] It should be noted that when encoding the first reconstructed video frame, the more reference frames are used, the more transcoding resources are consumed; similarly, when encoding the first reconstructed video frame, when determining the motion of the object in the first video frame, the larger the motion search range used, the more transcoding resources are consumed. Therefore, when it is necessary to reduce the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, this can be achieved by reducing the reference frame data and / or narrowing the motion search range.

[0128] Of course, the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame may also be reduced in other ways, such as reducing the transcoding bit rate, reducing the motion vector, etc., and this embodiment of the present application does not impose any restrictions on this.

[0129] The second transcoding processing method, the transcoding resource adjustment strategy includes a second transcoding strategy, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding the first reconstructed video frame. At this time, the first reconstructed video frame is encoded based on the second reference frame number and / or the second motion search range.

[0130] Among them, the second number of reference frames is the number of reference frames used when encoding the first reconstructed video frame, and the second number of reference frames is greater than the number of reference frames corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame, and the second motion search range is the motion search range used when encoding the first reconstructed video frame, and the second motion search range is greater than the motion search range corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame.

[0131] Optionally, if a default number of reference frames and a default motion search range for encoding reconstructed video frames are preset, when the transcoding resource adjustment strategy is determined to be the second transcoding strategy, that is, when the amount of transcoding resources used for encoding the first reconstructed video frame needs to be increased, the first number of reference frames used when encoding the first reconstructed video frame is greater than the default number of reference frames, and the first motion search range is greater than the default motion search range.

[0132] That is, when the amount of transcoding resources used in encoding the first reconstructed video frame needs to be increased, the number of reference frames used in encoding the first reconstructed video frame can be increased relative to the number of reference frames and the motion search range used in encoding the previous reconstructed video frame, and the motion search range used in encoding the first reconstructed video frame can be expanded; or the number of reference frames used in encoding the first reconstructed video frame can be increased directly based on the default number of reference frames and the default motion search range, and the motion search range used in encoding the first reconstructed video frame can be expanded. The embodiments of the present application are not limited to this.

[0133] Therefore, when it is necessary to increase the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, this can be achieved by increasing the reference frame data and / or expanding the motion search range. Of course, the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame can also be increased by other means, such as increasing the transcoding bit rate, increasing the motion vector, etc., and the embodiments of the present application do not limit this.

[0134] It should be noted that the first and second situations mentioned above are to reduce or increase the amount of transcoding resources when encoding the first reconstructed video frame by adjusting the transcoding parameters of the first reconstructed video frame, that is, to adaptively transcode the first reconstructed video frame. That is, the transcoding server does not transcode the first reconstructed video frame according to the transcoding parameters included in the transcoding template, but adaptively transcodes the first reconstructed video frame based on the determined transcoding resource adjustment strategy, and the transcoding parameters used in the adaptive transcoding process are different from the transcoding parameters in the transcoding template.

[0135] In addition, the transcoding performance of the transcoding server may be adjusted so that the total resource usage of the transcoding server is within a preset range, as shown in the third case below.

[0136] In the third transcoding processing mode, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, wherein the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames. Based on this, if the transcoding resource adjustment strategy is the first transcoding strategy, the main frequency of the processor in the transcoding server is increased; if the transcoding resource adjustment strategy is the second transcoding strategy, the main frequency of the processor in the transcoding server is reduced.

[0137] Among them, the main frequency is the clock frequency of the processor. The operation of the transcoding server is performed step by step under the control of the clock signal. Each clock signal cycle completes one step of operation. Therefore, the clock frequency largely reflects the speed at which the processor processes data.

[0138] That is, when the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame is greater than or equal to the upper limit of resource occupation, it means that the transcoding server may be in an overloaded state when encoding the first reconstructed video frame. Therefore, the main frequency of the processor in the transcoding server can be increased to enhance the processing capability of the transcoding server, thereby reducing the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, so that the total amount of resources occupied by the transcoding server can be within a preset range.

[0139] Similarly, when the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame is less than or equal to the lower limit of the resource occupation, it means that the resources of the transcoding server are not effectively utilized when the transcoding server is encoding the first reconstructed video frame. Therefore, the main frequency of the processor in the transcoding server can be reduced to reduce the processing power of the transcoding server, thereby increasing the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame, so that the total amount of resources occupied by the transcoding server can be within a preset range.

[0140] Combining the above three transcoding processing methods, when the transcoding resource adjustment strategy of the first reconstructed video frame is the first transcoding strategy, the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame can be reduced by reducing the reference frame data, narrowing the motion search range, or increasing the main frequency of the processor in the transcoding server, so as to ensure that the total resource occupancy of the transcoding server can be stabilized within a preset range; and when the transcoding resource adjustment strategy of the first reconstructed video frame is the second transcoding strategy, the amount of transcoding resources used by the transcoding server when encoding the first reconstructed video frame can be increased by increasing the reference frame data, expanding the motion search range, or reducing the main frequency of the processor in the transcoding server, so as to ensure that the total resource occupancy of the transcoding server can be stabilized within a preset range.

[0141] In this way, after determining the transcoding resource adjustment strategy for each reconstructed video frame, the transcoding server can sequentially encode the multiple reconstructed video frames based on the transcoding resource adjustment strategy for each reconstructed video frame to obtain a second video code stream. The second video code stream is the video code stream after the transcoding server transcodes the first video code stream according to the transcoding template corresponding to the transcoding task.

[0142] Step 206: Send the second video code stream.

[0143] In some embodiments, after the transcoding server generates the second video code stream, the second video code stream may be stored. In response to a request from the video player, the transcoding server may send the second video code stream to a corresponding video player.

[0144] Among them, the embodiment of the present application does not limit the sending time of the second video code stream and the receiving device.

[0145] In an embodiment of the present application, when the transcoding server performs a transcoding task according to a transcoding template or a custom transcoding parameter to transcode the first video code stream sent by the video sending end, the transcoding server can determine the transcoding resource adjustment strategy of the transcoding server when encoding each reconstructed video frame based on the device status information of the video sending end in the process of generating the first video code stream and the video frame transcoding information of the first video code stream, thereby performing transcoding processing on multiple reconstructed video frames based on the transcoding resource adjustment strategy corresponding to each reconstructed video frame to obtain a second video code stream. The transcoding resource adjustment strategy instructs the transcoding server to adjust the transcoding resources used when encoding the reconstructed video frames, that is, the transcoding server performs adaptive transcoding processing on each reconstructed video frame based on the total amount of resource occupation when encoding each reconstructed video frame, so as to ensure that the total amount of resource occupation of the transcoding server is within a preset range, avoid the problem of excessive or low resource occupation of the transcoding server, and improve the quality of the transcoding service and the resource utilization of the transcoding server.

[0146] Based on the above embodiment, when controlling the total resource usage of the transcoding server to be within a preset range, the embodiment of the present application can not only deploy the transcoding task on the transcoding server, but also use the above video transcoding method to adaptively transcode the first video stream so that the total resource usage of the transcoding server can be stabilized within the preset range. Figure 3 The transcoding task deployment method shown analyzes and evaluates the total resource usage of each server in the server cluster when executing the transcoding task, so that after the transcoding task is deployed, the total resource usage of the transcoding server can be stabilized within a preset range. That is, the resource usage of the transcoding server is controlled from the source of task deployment.

[0147] Please refer to Figure 3 , Figure 3 is a flowchart of a transcoding task deployment method provided in an embodiment of the present application. The method is applied to a transcoding task deployment device. The transcoding task deployment device can be Figure 1 The management terminal in the console shown can also be any server in the transcoding cluster. Figure 3 , the method comprises the following steps.

[0148] Step 301: receiving a first video code stream sent by a video sending end, and device state information of the video sending end during the generation process of the first video code stream.

[0149] The device status information includes one or more of the amount of coding resource usage, the amount of device temperature change, and the amount of device power consumption change.

[0150] As described above, the device status information of the video sending end can directly reflect the encoding resource consumption when the video sending end encodes the first video stream. When transcoding the first video stream, the device status information can be used to estimate the amount of transcoding resources required for transcoding the first video bit rate.

[0151] Step 302: Based on the device state information, determine the amount of transcoding resources required when the first video stream is transcoded according to at least one transcoding template, and obtain at least one transcoding resource amount.

[0152] In some embodiments, the implementation process of step 302 may be: based on the device status information and the transcoding parameters included in the first transcoding template, determine the reference resource amount corresponding to the first transcoding template, the first transcoding template being any one of the at least one transcoding template; determine the reference resource amount corresponding to the first transcoding template as the transcoding resource amount required when transcoding the first video stream according to the first transcoding template.

[0153] Referring to the transcoding templates shown in Table 1 above, if the video quality of the first transcoding template is ultra-high definition, the corresponding transcoding parameters are: resolution of 1920*1080 and bit rate of 2000Kbps. When determining the reference resource amount corresponding to the first transcoding template, the device status information can be added to the first transcoding template to expand the transcoding parameter items of the first transcoding template, thereby estimating the reference resource amount when transcoding according to the first transcoding template based on the fact that the first transcoding template contains multiple transcoding parameter items.

[0154] For example, the transcoding parameter items of the expanded first transcoding template include: resolution, bit rate, encoding resource occupancy, device temperature change and device power consumption change. These transcoding parameter items can be used as transcoding implementation restrictions to assist in estimating the reference resource amount when transcoding according to the first transcoding template.

[0155] In one possible implementation, the process of determining the reference resource amount corresponding to the first transcoding template is as follows: before executing the video transcoding task, based on the transcoding parameters and device status information included in each transcoding template, a video transcoding task is tested to measure the server resource amount required for the server to execute the transcoding task under the constraints of these transcoding parameters and device status information, and then based on the server resource amount, determine the reference resource amount corresponding to each transcoding template.

[0156] For example, the actually measured server resource amount is directly determined as the reference resource amount corresponding to the transcoding template, that is, the reference resource amount is equal to the server resource amount.

[0157] After determining the reference resource amount corresponding to each transcoding template in at least one transcoding template, the reference resource amount corresponding to the transcoding template can be determined as the transcoding resource amount required when transcoding the first video code stream according to the transcoding template.

[0158] Step 303: Based on at least one transcoding resource amount, at least one transcoding server is determined, each of the at least one transcoding server is used to transcode the first video stream according to a corresponding transcoding template to obtain a second video stream.

[0159] It should be noted that when the first video stream is transcoded according to at least one transcoding template, one transcoding template corresponds to a reference resource amount, one transcoding template corresponds to a transcoding task, and one transcoding task is to transcode the first video stream according to a transcoding template to obtain a second video stream.

[0160] In other words, for a transcoding task, the amount of transcoding resources required by the server to execute the transcoding task is the reference amount of resources of the transcoding template corresponding to the transcoding task.

[0161] In a possible implementation, the implementation process of step 303 may be: obtaining the transcoding resource occupancy of each server in the server cluster; determining the total resource occupancy of each server when transcoding the first video stream according to at least one transcoding template based on the transcoding resource occupancy of each server and at least one transcoding resource amount; determining the servers in the server cluster whose total resource occupancy is less than the transcoding resource threshold as at least one transcoding server.

[0162] That is, for any server in the server cluster, the amount of transcoding resources when the server transcodes the first video stream according to the first transcoding template is equal to the reference resource amount of the first transcoding template, and the total resource occupancy of the server when transcoding the first video stream according to the first transcoding template is equal to the sum of the reference resource amount of the first transcoding template and the transcoding resource occupancy of the server.

[0163] As an example, assume that the server cluster includes 6 servers, namely server A to server F, and the first video code stream needs to be transcoded according to three transcoding templates. Therefore, the first video code stream corresponds to three transcoding tasks: transcoding the first video code stream according to the first transcoding template, transcoding the first video code stream according to the second transcoding template, and transcoding the first video code stream according to the third transcoding template. Based on this, 3 transcoding servers need to be selected from the 6 servers to perform the above three transcoding tasks. Each transcoding task corresponds to a transcoding template, and the transcoding server transcodes the first video code stream according to the transcoding parameters corresponding to the transcoding template.

[0164] The implementation process of determining the three transcoding servers is as follows: first, based on the transcoding resource usage of each server and the transcoding resource usage of the three transcoding tasks (which can also be understood as the reference resource usage corresponding to the three transcoding templates), calculate the total resource usage of each server when performing the three transcoding tasks. Then, based on the total resource usage of each server when performing the three transcoding tasks, determine the server whose total resource usage is less than the transcoding resource threshold when performing each transcoding task, and obtain the candidate server corresponding to each transcoding task. Finally, determine the transcoding server that performs the transcoding task from the candidate servers corresponding to each transcoding task.

[0165] The following Table 3 shows the total amount of resources occupied when each server in the server cluster transcodes the first video code stream according to three transcoding templates.

[0166] Table 3

[0167]

[0168] Based on the above Table 3, after determining the total resource usage when each server performs three transcoding tasks, for the first transcoding task (transcoding the first video code stream according to the first transcoding template), based on the total resource usage when server A-server F transcodes the first video code stream according to the first transcoding template, i.e., M1+N1, M2+N1, M3+N1, M4+N1, M5+N1, and M6+N1, determine the candidate servers whose total resource usage is less than the transcoding resource threshold from server A-server F. For example, if M2+N1, M3+N1, and M4+N1 are all less than the transcoding resource threshold, server B, server C, and server D are determined as candidate servers for deploying the first transcoding task. Then, one server is selected from the three candidate servers as the transcoding server that transcodes the first video code stream according to the first transcoding template. For example, server C is determined as the transcoding server that transcodes the first video code stream according to the first transcoding template.

[0169] Similarly, based on the total resource occupancy and the transcoding resource threshold when server A-server F transcodes the first video stream according to the second transcoding template, the transcoding server that transcodes the first video stream according to the second transcoding template is determined; based on the total resource occupancy and the transcoding resource threshold when server A-server F transcodes the first video stream according to the third transcoding template, the transcoding server that transcodes the first video stream according to the third transcoding template is determined.

[0170] Thus, based on the above step 303, the transcoding server for transcoding the first video code stream according to each transcoding template can be determined. That is, based on the number of transcoding templates, the same number of servers are determined from the server cluster as transcoding servers for transcoding the first video code stream, and then the transcoding task of the first video code stream is deployed on the corresponding transcoding server.

[0171] In a possible implementation, after determining the transcoding server, the transcoding task deployment device can send the first video stream and the corresponding transcoding template to the corresponding transcoding server, so that the transcoding server transcodes the first video stream according to the transcoding template to obtain the second video stream.

[0172] In an embodiment of the present application, when the transcoding task deployment device deploys the transcoding task of the first video code stream in the server cluster, it can estimate the amount of transcoding resources required when transcoding the first video code stream according to at least one transcoding template with the help of the device status information of the video sending end during the generation process of the first video code stream, and then determine at least one transcoding server from the server cluster that transcodes the first video code stream according to at least one transcoding template based on the transcoding resource amount and the current resource occupancy of each server. Since the amount of transcoding resources is determined based on the device status information, the amount of transcoding resources can reflect the actual resource consumption of the server when transcoding the first video code stream. Therefore, the deployment accuracy of the transcoding task is improved by deploying the transcoding task based on the amount of transcoding resources. At the same time, it avoids the situation where the total amount of resource occupancy of the transcoding server is too high or too low after the transcoding task is deployed, thereby improving the quality of the transcoding service and the resource utilization of the transcoding server.

[0173] Based on the above embodiments, taking the live transcoding scenario as an example, the specific implementation process of adjusting the transcoding scheme by using the device status information in the process of generating the first video stream by the video sending end or the video frame transcoding information determined after the cloud transcoding service decodes the first video stream is explained.

[0174] See also Figure 4 As shown in the schematic diagram of the architecture of the video transcoding system, the video sending end includes multiple anchor devices and an end-side information collection module, and the cloud-based transcoding service mainly includes functional modules such as a decoding information collection module, a decision module, and an adaptive transcoding module.

[0175] Optionally, the terminal side information collection module of the video sending end can correspond to the anchor terminal device one by one, be integrated in each anchor device, or be independently set in multiple anchor devices, and be used to collect device status information when at least one anchor device generates the first video code stream.

[0176] When executing the video transcoding method provided in the embodiment of the present application, the end-side information collection module is used to obtain the device status information of each anchor device when generating the first video code stream, and transmit it to the decision module in the cloud in real time. The decoding information collection module acts on the physical layer of the live transcoding. After decoding the first video code stream uploaded to the transcoding service in the cloud, it determines the video frame transcoding information corresponding to the first video code stream, and transmits the video frame transcoding information to the decision module in real time. The decision module analyzes the device status information collected by the end side during the generation of the first video code stream by the anchor device, as well as the video frame transcoding information of the first video code stream collected by the cloud, to decide on an adaptive transcoding scheme and a transcoding task deployment scheme when the first video code stream is transcoded.

[0177] Among them, the device status information includes one or more of the encoding resource occupancy, the device temperature change and the device power consumption change, and the video frame transcoding information includes one or more of the motion vector, residual information, texture complexity, transcoding format and transcoding bit rate of the first reconstructed video frame.

[0178] When executing the transcoding task deployment method provided in the embodiment of the present application, the cloud-based adaptive transcoding module acts on the live broadcast scheduling layer to allocate the transcoding task of the first video stream in the transcoding cluster corresponding to the transcoding service based on the decided transcoding task deployment plan. The transcoding server that deploys the transcoding task then adaptively transcodes the first video stream based on the decided adaptive transcoding plan by adjusting transcoding parameters, processor computing power, etc., so that the total resource usage of the transcoding server can be stabilized within a preset range during the execution of the transcoding task.

[0179] See also Figure 5 , based on the device status information collected on the end side, the implementation process of determining the adaptive transcoding scheme is as follows: First, the end side information collection module collects the device status information of the video sending end in the process of generating the first video code stream, and the device status information includes one or more of the encoding resource occupancy, device temperature change, and device power consumption change. Since there is a certain connection between the device status information and the transcoding resource change required by the transcoding server to perform the transcoding task, the decision module determines the transcoding resource change required by the transcoding server to perform the transcoding task based on the device status information through neural networks, mathematical models, etc., and then estimates the total resource occupancy of the transcoding server when transcoding the first video code stream based on the transcoding resource change and the current resource occupancy of the transcoding server, and then determines the adaptive transcoding scheme of the first video code stream based on the total resource occupancy.

[0180] Specifically, when determining the adaptive transcoding scheme for encoding the first reconstructed video frame based on the device status information, it is possible to first determine whether the device status information of the video transmitter has a sudden change when generating the first original video frame corresponding to the first reconstructed video frame. When the device status information has a sudden change, the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame may exceed a preset range, and at this time, it is necessary to decide on an adaptive transcoding scheme for the first reconstructed video frame; when the device status information has not a sudden change, the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame will not exceed a preset range, and the first reconstructed video frame can continue to be transcoded according to the transcoding scheme of the previous reconstructed video frame.

[0181] As an example, whether the device status information has undergone a sudden change can be determined based on whether the amount of change in the device status information per unit time is greater than a threshold. For example, the amount of change in the device temperature per unit time is defined as ΔT, and the threshold is t; the amount of change in the encoding resource usage of the video transmitter per unit time is defined as ΔO, and the threshold is o; the amount of change in the device power consumption per unit time is defined as ΔP, and the threshold is p. When ΔT>t, ΔO>o, or ΔP>p, it is determined that the device status information has undergone a sudden change.

[0182] It should be noted that the sudden change in the device status information of the video transmitter is most likely due to a sudden change in the video image when the original video frame is generated. For example, a very stable image changes to a very shaky image, or a scene switch occurs. At this time, the encoding resource usage of the video transmitter when encoding the original video frame will also suddenly change.

[0183] As an example, if the picture changes from stable to drastic, the number of coding units (CU) in a frame that select the intra mode or advanced motion vector prediction (AMVP) mode (an inter-frame coding mode) will increase, and the number of CUs that select the skip mode (a special case of the Merge mode) or the Merge mode (an inter-frame coding mode) will decrease; if a scene switch occurs, the video encoder may directly encode a key frame (I frame). In these cases, the bit rate of the video transmitter will suddenly increase when encoding the original video frame, and the encoding complexity of the video encoder of the video transmitter will increase when encoding the original video frame. Therefore, the load of the processing unit in the video transmitter will become heavier, resulting in an increase in the encoding resource usage, device temperature, and device power consumption of the video transmitter. Correspondingly, when the transcoding server transcodes the reconstructed video frame corresponding to the original video frame, the resource usage of the transcoding server will also increase, and the transcoding (i.e., decoding first and then encoding) complexity of the transcoding server will also increase.

[0184] When determining the adaptive transcoding scheme of the reconstructed video frame based on the total amount of resource usage when encoding and reconstructing the video frame by the transcoding server, if the total amount of resource usage of the transcoding server is greater than the upper limit of resource usage, the transcoding parameters are adjusted (for example, reducing the number of reference frames, narrowing the motion search range) to reduce the computing power overhead of the transcoding server, or the processing capability of the transcoding server is enhanced (for example, increasing the main frequency of the processor in the transcoding server); if the total amount of resource usage of the transcoding server is less than the lower limit of resource usage, the transcoding parameters are adjusted (for example, increasing the number of reference frames, expanding the motion search range) to improve the image quality of the reconstructed video frame, or the processing capability of the transcoding server is reduced (for example, reducing the main frequency of the processor in the transcoding server). Ultimately, the total amount of resource usage of the transcoding server can be restored to a normal level, that is, between the lower limit of resource usage and the upper limit of resource usage, and will not be too high or too low.

[0185] As an example, if the upper limit T1 of the resource occupancy of the transcoding server is 65% and the lower limit T1 is 55%, when the video picture of the video sending end is relatively stable, the total resource occupancy of the transcoding server can be stabilized at about 60% when the transcoding server performs transcoding according to the preset transcoding template. However, when the picture of the original video frame generated by the video sending end changes suddenly (for example, the game picture suddenly changes drastically), when the transcoding server encodes the reconstructed video frame corresponding to the original video frame, the total resource occupancy of the transcoding server may suddenly increase to more than 65%, or suddenly decrease to less than 55%, resulting in reduced quality of transcoding service.

[0186] See also Figure 6 Based on the video frame transcoding information determined after the cloud decodes the first video code stream, the implementation process of determining the adaptive transcoding scheme is as follows: First, after the first video code stream is decoded, the decoding information collection module in the cloud determines the video frame transcoding information corresponding to the first video code stream, and the video frame transcoding information includes one or more of motion vector, residual information, texture complexity, transcoding format and transcoding bit rate. Based on the video frame transcoding information, the decision module determines the video frame transcoding complexity when the transcoding server performs the transcoding task through a neural network, a machine learning model, a mathematical model, etc., and then estimates the transcoding resource change when the transcoding server encodes and reconstructs the video frame based on the video frame transcoding complexity, and then estimates the total resource occupancy of the transcoding server when transcoding the first video code stream based on the transcoding resource change and the current resource occupancy of the transcoding server, and then determines the adaptive transcoding scheme of the first video code stream based on the total resource occupancy.

[0187] The video frame encoding complexity may be the time required to encode the reconstructed video frame using a fixed server computing power.

[0188] Before actual use, the video frame transcoding information provided by the decoder is used as the input of the network model, and the encoding complexity of the current frame is used as the output. A model is pre-trained (it can be a neural network or some more traditional mathematical models), and then this model is used for reasoning during actual use, that is, the video frame transcoding information of any reconstructed video frame is input to predict the video frame transcoding complexity of the reconstructed video frame.

[0189] Similarly, when determining the adaptive transcoding scheme for the reconstructed video frame based on the total amount of resources occupied when encoding and reconstructing the video frame on the transcoding server, if the total amount of resources occupied by the transcoding server is greater than the upper limit of the resource occupation, the transcoding parameters are adjusted to reduce the computing power overhead of the transcoding server, or the processing capability of the transcoding server is enhanced (for example, by increasing the main frequency of the processor in the transcoding server); if the total amount of resources occupied by the transcoding server is less than the lower limit of the resource occupation, the transcoding parameters are adjusted to improve the image quality of the reconstructed video frame, or the processing capability of the transcoding server is reduced (for example, by reducing the main frequency of the processor in the transcoding server). Ultimately, the total amount of resources occupied by the transcoding server can be restored to a normal level, that is, between the lower limit of the resource occupation and the upper limit of the resource occupation, and will not be too high or too low.

[0190] See also Figure 7 , based on the device status information collected on the end side, the implementation process of deploying transcoding tasks among multiple servers is as follows: the end side information collection module collects the device status information of the video sending end in the process of generating the first video code stream, and the device status information includes one or more of the encoding resource occupancy, device temperature change, and device power consumption change. Based on the various transcoding parameters in the known transcoding template, the decision module uses the device status information collected on the end side to expand the transcoding template and also adds several auxiliary information provided by the end side to the transcoding template, so as to more accurately estimate the amount of transcoding resources required when transcoding the first video code stream according to each transcoding template. Finally, based on the amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, the transcoding tasks are allocated on multiple servers in the transcoding cluster, so that the total resource occupancy of the transcoding server that deploys the transcoding task is stabilized within a preset range.

[0191] The implementation process of estimating the amount of transcoding resources is as follows: before running the transcoding task, actually run each transcoding template once to measure the required server resources, and determine the reference resource amount corresponding to each transcoding template based on the server resources. During the actual transcoding, determine which transcoding template the transcoding task corresponds to, and the amount of transcoding resources required to execute the transcoding task can be estimated.

[0192] In summary, the embodiments of the present application can utilize the device status information during the process of the video sending end generating the first video code stream, and deploy the transcoding task of the first video code stream in the transcoding cluster in the cloud, so that the transcoding server that deploys the transcoding task can effectively utilize the transcoding resources of the transcoding server while meeting the resource requirements of the transcoding task. When the transcoding server executes the first video transcoding task, it can also adaptively transcode the first video code stream based on the device status information during the process of the video sending end generating the first video code stream or the video frame transcoding information determined after the cloud transcoding service decodes the first video code stream, thereby improving the resource utilization of the transcoding service, avoiding the total amount of transcoding service resource occupation being too high or too low, and improving the quality of the transcoding service.

[0193] Figure 8 800 is a schematic diagram of the structure of a video transcoding device provided in an embodiment of the present application. The video transcoding device 800 can be implemented by software, hardware or a combination of both to become part or all of a transcoding server. Figure 8 The video transcoding device 800 includes: a video receiving module 801, a video decoding module 802, an information acquisition module 803, a strategy determination module 804, a video transcoding module 805 and a video sending module 806.

[0194] The video receiving module 801 is used to receive a first video code stream sent by a video sending end;

[0195] The video decoding module 802 is used to decode the first video code stream to obtain a reconstructed video signal, where the reconstructed video signal includes a plurality of reconstructed video frames;

[0196] The information acquisition module 803 is used to acquire device status information of the video transmitter or video frame transcoding information of the first video code stream; wherein the device status information includes one or more of the amount of coding resource usage, the amount of device temperature change, and the amount of device power consumption change; the video frame transcoding information includes one or more of the motion vectors, residual information, texture complexity, transcoding format, and transcoding bit rate of multiple reconstructed video frames;

[0197] A strategy determination module 804 is used to determine a transcoding resource adjustment strategy for a first reconstructed video frame among multiple reconstructed video frames based on device state information or video frame transcoding information, wherein the transcoding resource adjustment strategy instructs the transcoding server to adjust resources used when encoding the first reconstructed video frame so that the total resource occupation of the transcoding server is within a preset range;

[0198] The video transcoding module 805 is used to perform transcoding processing on the multiple reconstructed video frames based on the transcoding resource adjustment strategy of the multiple reconstructed video frames to obtain a second video code stream;

[0199] The video sending module 806 is used to send the second video code stream.

[0200] Optionally, the policy determination module 804 includes:

[0201] A resource estimation unit, configured to estimate the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information or the video frame transcoding information;

[0202] The strategy determination unit is used to determine the transcoding resource adjustment strategy of the first reconstructed video frame based on the total resource occupancy.

[0203] Optionally, the resource estimation unit is specifically configured to:

[0204] Inputting device status information or video frame transcoding information into a trained transcoding resource estimation model to obtain a transcoding resource change output by the transcoding resource estimation model;

[0205] Based on the transcoding resource change amount and the current resource occupation of the transcoding server, the total resource occupation of the transcoding server when encoding the first reconstructed video frame is estimated.

[0206] Optionally, the resource estimation unit is specifically configured to:

[0207] Determining, based on the device state information, a change in encoding resources of a video transmitter during encoding of a first original video frame, the first original video frame corresponding to a first reconstructed video frame;

[0208] If the coding resource change amount is greater than the coding resource threshold, then estimating the transcoding resource change amount when the transcoding server encodes the first reconstructed video frame based on the coding resource change amount;

[0209] Based on the transcoding resource change amount and the current resource occupation of the transcoding server, the total resource occupation of the transcoding server when encoding the first reconstructed video frame is estimated.

[0210] Optionally, the resource estimation unit is specifically configured to:

[0211] Inputting the video frame transcoding information corresponding to the first reconstructed video frame into the trained transcoding complexity estimation model to obtain the video frame transcoding complexity output by the transcoding complexity estimation model;

[0212] If the video frame transcoding complexity is greater than the complexity threshold, based on the video frame transcoding complexity, estimating a change in transcoding resources when the transcoding server encodes the first reconstructed video frame;

[0213] Based on the transcoding resource change amount and the current resource occupation of the transcoding server, the total resource occupation of the transcoding server when encoding the first reconstructed video frame is estimated.

[0214] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames;

[0215] The strategy determination unit 804 is specifically configured to:

[0216] If the total resource occupation is greater than the resource occupation upper limit, determining the transcoding resource adjustment strategy of the first reconstructed video frame as the first transcoding strategy;

[0217] If the total resource occupancy is less than the lower limit of the resource occupancy, the transcoding resource adjustment strategy of the first reconstructed video frame is determined to be the second transcoding strategy.

[0218] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy, and the first transcoding strategy is used to reduce the amount of transcoding resources used when reconstructing video frame encoding;

[0219] The video transcoding module 805 is specifically used for:

[0220] The first reconstructed video frame is encoded based on the first reference frame number and / or the first motion search range, wherein the first reference frame number is the number of reference frames used when encoding the first reconstructed video frame, and the first reference frame number is smaller than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the first motion search range is the motion search range used when encoding the first reconstructed video frame, and the first motion search range is smaller than the motion search range corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame.

[0221] Optionally, the transcoding resource adjustment strategy includes a second transcoding strategy, and the second transcoding strategy is used to increase the amount of transcoding resources used when reconstructing the video frame encoding;

[0222] The video transcoding module 805 is specifically used for:

[0223] The first reconstructed video frame is encoded based on the second number of reference frames and / or the second motion search range, the second number of reference frames is the number of reference frames used when encoding the first reconstructed video frame, and the second number of reference frames is greater than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the second motion search range is the motion search range used when encoding the first reconstructed video frame, and the second motion search range is greater than the motion search range corresponding to the previous reconstructed video frame adjacent to the first reconstructed video frame.

[0224] Optionally, the transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; the video transcoding device 800 also includes:

[0225] The main frequency adjustment module is used to increase the main frequency of the processor in the transcoding server if the transcoding resource adjustment strategy is the first transcoding strategy; if the transcoding resource adjustment strategy is the second transcoding strategy, then reduce the main frequency of the processor in the transcoding server.

[0226] Among them, the above-mentioned video receiving module, video decoding module, information acquisition module, strategy determination module, video transcoding module and video sending module can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the video transcoding module is introduced below by taking the video transcoding module as an example. Similarly, the implementation of the video receiving module, video decoding module, information acquisition module, strategy determination module and video sending module can refer to the implementation of the video transcoding module.

[0227] As an example of a software functional unit, a video transcoding module may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above-mentioned computing instance may be one or more. For example, the video transcoding module may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0228] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0229] As an example of a hardware functional unit, the video transcoding module may include at least one computing device, such as a server, etc. Alternatively, module A may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0230] The multiple computing devices included in the video transcoding module can be distributed in the same region or in different regions. The multiple computing devices included in the video transcoding module can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the video transcoding module can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0231] It should be noted that, in other embodiments, the video transcoding module can be used to execute any step in the video transcoding method, and the video receiving module, video decoding module, information acquisition module, policy determination module and video sending module can also be used to execute any step in the video transcoding method. The steps that the video receiving module, video decoding module, information acquisition module, policy determination module, video transcoding module and video sending module are responsible for implementing can be specified as needed. The different steps in the video transcoding method are respectively implemented by the video receiving module, video decoding module, information acquisition module, policy determination module, video transcoding module and video sending module to realize all the functions of the video transcoding device.

[0232] In an embodiment of the present application, in the process of performing transcoding tasks according to a transcoding template or a custom transcoding parameter to transcode the first video code stream sent by the video sending end, the video transcoding device can determine the transcoding resource adjustment strategy when encoding each reconstructed video frame based on the device status information of the video sending end in the process of generating the first video code stream and the video frame transcoding information of the first video code stream, thereby performing transcoding processing on multiple reconstructed video frames based on the transcoding resource adjustment strategy corresponding to each reconstructed video frame to obtain a second video code stream. The transcoding resource adjustment strategy instructs the transcoding server to adjust the transcoding resources used when encoding the reconstructed video frames, that is, the video transcoding device performs adaptive transcoding processing on each reconstructed video frame based on the total amount of resource occupation when encoding each reconstructed video frame, so as to ensure that the total amount of resource occupation of the transcoding server is within a preset range, avoid the problem of excessive or low resource occupation of the transcoding server, and improve the quality of the transcoding service and the resource utilization of the transcoding server.

[0233] It should be noted that: the video transcoding device provided in the above embodiment only uses the division of the above functional modules as an example when transcoding the video code stream. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the video transcoding device provided in the above embodiment and the video transcoding method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0234] Fig. 9 900 is a schematic diagram of a transcoding task deployment device provided in an embodiment of the present application. The transcoding task deployment device 900 can be implemented by software, hardware, or a combination of both to become part or all of the transcoding task deployment device. Fig. 9 The transcoding task deployment device 900 includes: a video receiving module 901, a resource estimation module 902 and a task deployment module 903.

[0235] The video receiving module 901 is used to receive the first video code stream sent by the video sending end, and the device status information of the video sending end during the generation of the first video code stream, where the device status information includes one or more of the coding resource occupancy, the device temperature change and the device power consumption change;

[0236] The resource estimation module 902 is used to determine, based on the device state information, the amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, and obtain at least one transcoding resource amount;

[0237] The task deployment module 903 is used to determine at least one transcoding server based on at least one transcoding resource amount, each of the at least one transcoding server is used to transcode a first video stream according to a corresponding transcoding template to obtain a second video stream, and the total resource occupancy of the transcoding server when transcoding the first video stream is within a preset range.

[0238] Optionally, the resource estimation module 902 includes:

[0239] A template resource estimation unit, configured to determine a reference resource amount corresponding to a first transcoding template based on device state information and transcoding parameters included in the first transcoding template, wherein the first transcoding template is any one of the at least one transcoding template;

[0240] The transcoding resource estimation unit is used to determine the reference resource amount corresponding to the first transcoding template as the transcoding resource amount required when transcoding the first video code stream according to the first transcoding template.

[0241] Optionally, the task deployment module 903 includes:

[0242] A resource information acquisition unit, used to acquire the current resource usage of each server in the server cluster;

[0243] a resource occupancy estimation unit, configured to determine, based on a current resource occupancy of each server and at least one transcoding resource amount, a total resource occupancy of each server when transcoding the first video code stream according to at least one transcoding template;

[0244] The determination unit is used to determine a server in the server cluster whose total resource usage is less than a transcoding resource threshold as at least one transcoding server.

[0245] Among them, the above-mentioned video receiving module, resource estimation module and task deployment module can all be implemented by software, or can be implemented by hardware. Exemplarily, the resource estimation module is taken as an example to introduce the implementation method of the resource estimation module. Similarly, the implementation method of the video receiving module and the task deployment module can refer to the implementation method of the resource estimation module.

[0246] As an example of a software functional unit, a resource estimation module may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the resource estimation module may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

[0247] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0248] As an example of a hardware functional unit, the resource estimation module may include at least one computing device, such as a server, etc. Alternatively, the resource estimation module may also be a device implemented by ASIC or PLD, etc. The PLD may be implemented by CPLD, FPGA, GAL or any combination thereof.

[0249] The multiple computing devices included in the resource estimation module can be distributed in the same region or in different regions. The multiple computing devices included in the resource estimation module can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the resource estimation module can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0250] It should be noted that, in other embodiments, the resource estimation module can be used to execute any step in the transcoding task deployment method, the video receiving module can be used to execute any step in the transcoding task deployment method, and the task deployment module can be used to execute any step in the transcoding task deployment method. The steps that the video receiving module, the resource estimation module, and the task deployment module are responsible for implementing can be specified as needed. The full functions of the transcoding task deployment device are realized by respectively implementing different steps in the transcoding task deployment method through the video receiving module, the resource estimation module, and the task deployment module.

[0251] In an embodiment of the present application, when the transcoding task deployment device deploys the transcoding task of the first video code stream in the server cluster, it can estimate the amount of transcoding resources required when transcoding the first video code stream according to at least one transcoding template with the help of the device status information of the video sending end during the generation process of the first video code stream, and then determine at least one transcoding server from the server cluster that transcodes the first video code stream according to at least one transcoding template based on the transcoding resource amount and the current resource occupancy of each server. Since the amount of transcoding resources is determined based on the device status information, the amount of transcoding resources can reflect the actual resource consumption of the server when transcoding the first video code stream. Therefore, the deployment accuracy of the transcoding task is improved by deploying the transcoding task based on the amount of transcoding resources. At the same time, it avoids the situation where the total amount of resource occupancy of the transcoding server is too high or too low after the transcoding task is deployed, thereby improving the quality of the transcoding service and the resource utilization of the transcoding server.

[0252] It should be noted that: when the transcoding task deployment device provided in the above embodiment deploys the transcoding task of the first video code stream in the server cluster, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the transcoding task deployment device provided in the above embodiment and the transcoding task deployment method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0253] See also Fig.10 , Fig.10 1 is a schematic diagram of a computing device provided in an embodiment of the present application, wherein the computing device 1000 includes: a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other via the bus 1002. The embodiment of the present application does not limit the number of processors and memories in the computing device 1000.

[0254] When implementing the technical solution provided in the embodiments of the present application, the computing device 1000 may be a server that executes transcoding tasks, or a transcoding task deployment device that deploys video transcoding tasks in a server cluster.

[0255] The bus 1002 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 The bus 1002 may include a path for transmitting information between various components of the computing device 1000 (eg, the memory 1006, the processor 1004, and the communication interface 1008).

[0256] The processor 1004 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0257] The memory 1006 may include a volatile memory, such as a random access memory (RAM). The processor 1004 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0258] In the embodiment of the present application, the memory 1006 stores an executable program code, and the processor 1004 executes the executable program code to respectively implement the functions of the aforementioned video receiving module 801, the video decoding module 802, the information acquisition module 803, the strategy determination module 804, the video transcoding module 805 and the video sending module 806, thereby implementing Figure 2 That is, the memory 1006 stores the video transcoding method for executing the above Figure 2 Instructions for the video transcoding method shown.

[0259] It should be noted that Fig.10 Only the above-mentioned Figure 2 The program code of the video transcoding method shown in FIG. 1 is used as an example for explanation. Of course, the memory 1006 can also store Figure 3The instructions of the transcoding task deployment method shown enable the processor 1004 to execute the executable program code to respectively implement the functions of the aforementioned video receiving module 901, resource estimation module 902 and task deployment module 903.

[0260] The communication interface 1008 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.

[0261] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, for example, a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0262] like Fig.11 As shown, the computing device cluster includes at least one computing device 1000. The memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same Figure 2 Instructions for the video transcoding method shown.

[0263] In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the video transcoding method. In other words, the combination of one or more computing devices 1000 may jointly execute instructions for executing Figure 2 Instructions for the video transcoding method shown.

[0264] It should be noted that the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the video transcoding device. That is, the instructions stored in the memory 1006 in different computing devices 1000 can implement the functions of one or more modules of the aforementioned video receiving module 801, video decoding module 802, information acquisition module 803, strategy determination module 804, video transcoding module 805 and video sending module 806.

[0265] It should be noted that Fig.11 Only the above-mentioned Figure 2 The instructions of the video transcoding method shown in FIG. 1 are used as an example for explanation. Of course, the memory 1006 can also store Figure 3 The instructions of the transcoding task deployment method shown enable the processor 1004 to execute the executable program code to respectively implement the functions of one or more modules among the aforementioned video receiving module 901, resource estimation module 902 and task deployment module 903.

[0266] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network, etc. Fig.12 A possible implementation is shown, such as Fig.12 As shown, two computing devices 1000A and 1000B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0267] Using computing device clusters to achieve the above Figure 2 As an example, in the video transcoding method shown in Fig.12 In the illustrated implementation, the memory 1006 in the computing device 1000A stores instructions for executing the functions of the video receiving module 801 and the video sending module 806. Meanwhile, the memory 1006 in the computing device 1000B stores instructions for executing the functions of the video decoding module 802, the information acquisition module 803, the strategy determination module 804, and the video transcoding module 805.

[0268] Fig.12 The connection method between the computing device clusters shown can be based on the consideration that the video transcoding method provided in the present application may require transcoding of a large number of video code streams, so it is considered to implement the video receiving and sending function and the video transcoding function in different computing devices, and the functions implemented by the video decoding module 802, the information acquisition module 803, the strategy determination module 804 and the video transcoding module 805 are handed over to the computing device 1000B for execution, so as to reduce the computing amount of the computing device 1000A.

[0269] It should be understood that Fig.12 The functions of the computing device 1000A shown in FIG. 1000A may also be completed by multiple computing devices 1000. Similarly, the functions of the computing device 1000B may also be completed by multiple computing devices 1000.

[0270] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.11 and Fig.12 The connection mode of the computing device cluster. The difference is that the memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the video transcoding method, or the memory 1006 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the transcoding task deployment method.

[0271] In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the video transcoding method. In other words, the combination of one or more computing devices 1000 may jointly execute instructions for executing the video transcoding method.

[0272] Of course, in some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may also store partial instructions for executing the transcoding task deployment method. In other words, the combination of one or more computing devices 1000 may jointly execute instructions for executing the transcoding task deployment method.

[0273] The present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above Figure 2 The video transcoding method shown, or Figure 3 The transcoding task deployment method shown.

[0274] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above Figure 2 The video transcoding method shown, or Figure 3 The transcoding task deployment method shown.

[0275] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0276] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0277] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A video transcoding method, characterized in that: Applied to a transcoding server, the method comprises: Receive a first video code stream sent by a video sending end; Decoding the first video code stream to obtain a reconstructed video signal, wherein the reconstructed video signal includes a plurality of reconstructed video frames; Obtaining device status information of the video transmitter or video frame transcoding information of the first video code stream; wherein the device status information includes one or more of coding resource occupancy, device temperature change, and device power consumption change, and the video frame transcoding information includes one or more of motion vectors, residual information, texture complexity, transcoding format, and transcoding bit rate of the multiple reconstructed video frames; Determine, based on the device state information or the video frame transcoding information, a transcoding resource adjustment strategy for a first reconstructed video frame among the multiple reconstructed video frames, the transcoding resource adjustment strategy instructing the transcoding server to adjust resources used when encoding the first reconstructed video frame so that the total resource occupation of the transcoding server is within a preset range; Based on the transcoding resource adjustment strategy of the multiple reconstructed video frames, transcoding the multiple reconstructed video frames to obtain a second video stream; Send the second video stream.

2. The method according to claim 1, characterized in that The determining, based on the device status information or the video frame transcoding information, a transcoding resource adjustment strategy for a first reconstructed video frame among the multiple reconstructed video frames includes: Based on the device status information or the video frame transcoding information, estimating the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame; Based on the total resource occupancy, a transcoding resource adjustment strategy for the first reconstructed video frame is determined.

3. The method according to claim 2, characterized in that The estimating, based on the device status information or the video frame transcoding information, the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame includes: Inputting the device status information or the video frame transcoding information into a trained transcoding resource estimation model to obtain a transcoding resource change output by the transcoding resource estimation model; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

4. The method according to claim 2, characterized in that The estimating, based on the device status information, the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame includes: Determining, based on the device state information, a change in encoding resources of the video transmitter during encoding of a first original video frame, the first original video frame corresponding to the first reconstructed video frame; If the coding resource change amount is greater than the coding resource threshold, estimating the transcoding resource change amount when the transcoding server encodes the first reconstructed video frame based on the coding resource change amount; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

5. The method according to claim 2, characterized in that The estimating, based on the video frame transcoding information, the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame includes: Inputting the video frame transcoding information corresponding to the first reconstructed video frame into a trained transcoding complexity estimation model to obtain the video frame transcoding complexity output by the transcoding complexity estimation model; If the video frame transcoding complexity is greater than the complexity threshold, based on the video frame transcoding complexity, estimating a change in transcoding resources when the transcoding server encodes the first reconstructed video frame; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

6. The method according to any one of claims 2 to 5, characterized in that: The transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, wherein the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; The determining, based on the total amount of resource occupancy, a transcoding resource adjustment strategy for the first reconstructed video frame includes: If the total resource occupancy is greater than the resource occupancy upper limit, determining the transcoding resource adjustment strategy of the first reconstructed video frame to be the first transcoding strategy; If the total resource occupancy is less than the lower limit of the resource occupancy, the transcoding resource adjustment strategy of the first reconstructed video frame is determined to be the second transcoding strategy.

7. The method according to any one of claims 1 to 6, characterized in that: The transcoding resource adjustment strategy includes a first transcoding strategy, and the first transcoding strategy is used to reduce the amount of transcoding resources used when reconstructing video frame encoding; The transcoding resource adjustment strategy based on the multiple reconstructed video frames, transcoding the multiple reconstructed video frames, includes: The first reconstructed video frame is encoded based on a first reference frame number and / or a first motion search range, wherein the first reference frame number is the number of reference frames used when encoding the first reconstructed video frame, and the first reference frame number is smaller than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the first motion search range is the motion search range used when encoding the first reconstructed video frame, and the first motion search range is smaller than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

8. The method according to any one of claims 1 to 6, characterized in that: The transcoding resource adjustment strategy includes a second transcoding strategy, and the second transcoding strategy is used to increase the amount of transcoding resources used when reconstructing video frame encoding; The transcoding resource adjustment strategy based on the multiple reconstructed video frames, transcoding the multiple reconstructed video frames, includes: The first reconstructed video frame is encoded based on a second number of reference frames and / or a second motion search range, wherein the second number of reference frames is the number of reference frames used when encoding the first reconstructed video frame, and the second number of reference frames is greater than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the second motion search range is the motion search range used when encoding the first reconstructed video frame, and the second motion search range is greater than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

9. The method according to any one of claims 1 to 6, characterized in that: The transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, wherein the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; the method further includes: If the transcoding resource adjustment strategy is the first transcoding strategy, increasing the main frequency of the processor in the transcoding server; If the transcoding resource adjustment strategy is the second transcoding strategy, the main frequency of the processor in the transcoding server is reduced.

10. A transcoding task deployment method, characterized in that: Applied to a transcoding task deployment device, the method comprises: receiving a first video code stream sent by a video sending end, and device status information of the video sending end during the generation of the first video code stream, wherein the device status information includes one or more of an encoding resource occupancy, a device temperature change, and a device power consumption change; Based on the device state information, determine the amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, and obtain at least one transcoding resource amount; Based on the at least one transcoding resource amount, at least one transcoding server is determined, each of the at least one transcoding server is used to transcode the first video stream according to a corresponding transcoding template to obtain a second video stream, and the total resource occupation of the transcoding server when transcoding the first video stream is within a preset range.

11. The method according to claim 10, characterized in that The determining, based on the device state information, an amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, includes: Determine, based on the device state information and a transcoding parameter included in a first transcoding template, a reference resource amount corresponding to the first transcoding template, where the first transcoding template is any one of the at least one transcoding template; The reference resource amount corresponding to the first transcoding template is determined as the transcoding resource amount required when transcoding the first video stream according to the first transcoding template.

12. The method according to claim 10, characterized in that The determining at least one transcoding server based on the at least one transcoding resource amount includes: Get the current resource usage of each server in the server cluster; Determine, based on the current resource occupation of each server and the at least one transcoding resource amount, the total resource occupation of each server when transcoding the first video stream according to the at least one transcoding template; A server in the server cluster whose total resource occupancy is less than a transcoding resource threshold is determined as the at least one transcoding server.

13. A video transcoding device, characterized in that: Included in a transcoding server, the device includes: A video receiving module, used for receiving a first video code stream sent by a video sending end; A video decoding module, configured to decode the first video code stream to obtain a reconstructed video signal, wherein the reconstructed video signal includes a plurality of reconstructed video frames; An information acquisition module, used to acquire device status information of the video transmitter or video frame transcoding information of the first video code stream; wherein the device status information includes one or more of coding resource occupancy, device temperature change, and device power consumption change; and the video frame transcoding information includes one or more of motion vectors, residual information, texture complexity, transcoding format, and transcoding bit rate of the multiple reconstructed video frames; a strategy determination module, configured to determine a transcoding resource adjustment strategy for a first reconstructed video frame among the multiple reconstructed video frames based on the device state information or the video frame transcoding information, wherein the transcoding resource adjustment strategy instructs the transcoding server to adjust resources used when encoding the first reconstructed video frame so that the total resource occupation of the transcoding server is within a preset range; A video transcoding module, configured to perform transcoding processing on the multiple reconstructed video frames based on a transcoding resource adjustment strategy of the multiple reconstructed video frames to obtain a second video code stream; The video sending module is used to send the second video code stream.

14. The device according to claim 13, characterized in that The strategy determination module includes: a resource estimation unit, configured to estimate the total amount of resources occupied by the transcoding server when encoding the first reconstructed video frame based on the device status information or the video frame transcoding information; A strategy determination unit is used to determine a transcoding resource adjustment strategy for the first reconstructed video frame based on the total resource occupancy.

15. The device according to claim 14, characterized in that The resource estimation unit is specifically used to: Inputting the device status information or the video frame transcoding information into a trained transcoding resource estimation model to obtain a transcoding resource change output by the transcoding resource estimation model; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

16. The device according to claim 14, characterized in that The resource estimation unit is specifically used to: Determining, based on the device state information, a change in encoding resources of the video transmitter during encoding of a first original video frame, the first original video frame corresponding to the first reconstructed video frame; If the coding resource change amount is greater than the coding resource threshold, estimating the transcoding resource change amount when the transcoding server encodes the first reconstructed video frame based on the coding resource change amount; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

17. The device according to claim 14, characterized in that The resource estimation unit is specifically used to: Inputting the video frame transcoding information corresponding to the first reconstructed video frame into a trained transcoding complexity estimation model to obtain the video frame transcoding complexity output by the transcoding complexity estimation model; If the video frame transcoding complexity is greater than the complexity threshold, based on the video frame transcoding complexity, estimating a change in transcoding resources when the transcoding server encodes the first reconstructed video frame; Based on the transcoding resource change and the current resource occupancy of the transcoding server, the total resource occupancy of the transcoding server when encoding the first reconstructed video frame is estimated.

18. The device according to any one of claims 14 to 17, characterized in that The transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, wherein the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; The strategy determination unit is specifically used to: If the total resource occupancy is greater than the resource occupancy upper limit, determining the transcoding resource adjustment strategy of the first reconstructed video frame to be the first transcoding strategy; If the total resource occupancy is less than the lower limit of the resource occupancy, the transcoding resource adjustment strategy of the first reconstructed video frame is determined to be the second transcoding strategy.

19. The device according to any one of claims 13 to 18, characterized in that: The transcoding resource adjustment strategy includes a first transcoding strategy, and the first transcoding strategy is used to reduce the amount of transcoding resources used when reconstructing video frame encoding; The video transcoding module is specifically used for: The first reconstructed video frame is encoded based on a first reference frame number and / or a first motion search range, wherein the first reference frame number is the number of reference frames used when encoding the first reconstructed video frame, and the first reference frame number is smaller than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the first motion search range is the motion search range used when encoding the first reconstructed video frame, and the first motion search range is smaller than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

20. The device according to any one of claims 13 to 18, characterized in that The transcoding resource adjustment strategy includes a second transcoding strategy, and the second transcoding strategy is used to increase the amount of transcoding resources used when reconstructing video frame encoding; The video transcoding module is specifically used for: The first reconstructed video frame is encoded based on a second number of reference frames and / or a second motion search range, wherein the second number of reference frames is the number of reference frames used when encoding the first reconstructed video frame, and the second number of reference frames is greater than the number of reference frames corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame, and the second motion search range is the motion search range used when encoding the first reconstructed video frame, and the second motion search range is greater than the motion search range corresponding to a previous reconstructed video frame adjacent to the first reconstructed video frame.

21. The device according to any one of claims 13 to 18, characterized in that The transcoding resource adjustment strategy includes a first transcoding strategy and a second transcoding strategy, wherein the first transcoding strategy is used to reduce the amount of transcoding resources used when encoding and reconstructing video frames, and the second transcoding strategy is used to increase the amount of transcoding resources used when encoding and reconstructing video frames; the device also includes: A main frequency adjustment module, configured to increase the main frequency of the processor in the transcoding server if the transcoding resource adjustment strategy is the first transcoding strategy; The main frequency adjustment module is further configured to reduce the main frequency of the processor in the transcoding server if the transcoding resource adjustment strategy is the second transcoding strategy.

22. A transcoding task deployment device, characterized in that: Included in the transcoding task deployment device, the device includes: A video receiving module, configured to receive a first video code stream sent by a video sending end, and device status information of the video sending end during the generation of the first video code stream, wherein the device status information includes one or more of an encoding resource occupancy, a device temperature change, and a device power consumption change; A resource estimation module, configured to determine, based on the device state information, an amount of transcoding resources required when the first video code stream is transcoded according to at least one transcoding template, and obtain at least one amount of transcoding resources; The task deployment module is used to determine at least one transcoding server based on the at least one transcoding resource amount, each of the at least one transcoding server is used to transcode the first video code stream according to a corresponding transcoding template to obtain a second video code stream, and the total resource occupancy of the transcoding server when transcoding the first video code stream is within a preset range.

23. The device according to claim 22, characterized in that The resource estimation module comprises: a template resource estimation unit, configured to determine a reference resource amount corresponding to a first transcoding template based on the device state information and a transcoding parameter included in the first transcoding template, wherein the first transcoding template is any one of the at least one transcoding template; The transcoding resource estimation unit is used to determine the reference resource amount corresponding to the first transcoding template as the transcoding resource amount required when transcoding the first video stream according to the first transcoding template.

24. The device according to claim 22, characterized in that The task deployment module includes: A resource information acquisition unit, used to acquire the current resource usage of each server in the server cluster; a resource occupancy estimation unit, configured to determine, based on the current resource occupancy of each server and the at least one transcoding resource amount, the total resource occupancy of each server when transcoding the first video code stream according to the at least one transcoding template; The determining unit is configured to determine a server in the server cluster whose total resource occupancy is less than a transcoding resource threshold as the at least one transcoding server.

25. A computing device cluster, comprising at least one computing device, characterized in that: Each computing device includes a processor and memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the video transcoding method as described in any one of claims 1-9, or executes the transcoding task deployment method as described in any one of claims 10-12.

26. A computer program product comprising instructions, characterized in that When the instruction is executed by a computing device cluster, the computing device cluster executes the video transcoding method as described in any one of claims 1 to 9, or executes the transcoding task deployment method as described in any one of claims 10 to 12.

27. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the video transcoding method as described in any one of claims 1 to 9, or executes the transcoding task deployment method as described in any one of claims 10 to 12.

Images