Encoding and decoding resource allocation method and apparatus, and electronic device

Abstract

Embodiments of the present application provide a coding and decoding resource allocation method and apparatus, and an electronic device. The method comprises: first, obtaining a first-type operation for a media stream to be processed; then determining a target protocol, a target resolution, and a target frame rate corresponding to the first-type operation; according to the target protocol, the target resolution, and the target frame rate, determining the quantity of coding and decoding resources to be consumed by the first-type operation, to obtain an allocation quantity; under the condition that the quantity of currently-idle coding and decoding resources is not smaller than the allocation quantity, allocating, from the currently-idle coding and decoding resources, coding and decoding resources of the allocation quantity to said media stream; and under the condition that the quantity of the currently-idle coding and decoding resources is smaller than the allocation quantity, generating prompt information indicating that the coding and decoding resources are insufficient. Corresponding coding and decoding resources are determined by combining the target protocol, the resolution, and the frame rate, so that resource calculation errors can be reduced, and reasonable control and allocation of the coding and decoding resources is realized.

Description

Method, device and electronic device for allocating coding and decoding resources Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and electronic device for allocating coding and decoding resources. Background Art

[0002] During the image encoding, decoding, and transmission process, some recorders, such as NVRs (Network Video Recorders), use the same VCP (Video Codec Processor, referred to herein as the hardware unit) to implement encoding and decoding functions. When the hardware unit has reached a bottleneck, if it is still needed to implement operations such as preview, it will affect the encoding and decoding performance and processing effects of the ongoing operations, such as preview lag, data anomalies, and data loss.

[0003] Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, and electronic device for allocating codec resources to reasonably control and allocate codec resources. The specific technical solution is as follows:

[0005] An embodiment of the present application provides a method for allocating coding and decoding resources, the method comprising:

[0006] Get the first type of operation for the media stream to be processed;

[0007] Determining a target protocol, a target resolution, and a target frame rate corresponding to the first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0008] Determining the amount of codec resources required for the first type of operation according to the target protocol, the target resolution, and the target frame rate, and using the obtained amount as the amount to be allocated;

[0009] When the number of currently idle codec resources is not less than the number to be allocated, allocating the number of codec resources to be allocated to the to-be-processed media stream from the currently idle codec resources; and / or,

[0010] When the number of currently idle codec resources is less than the number to be allocated, prompt information indicating that the codec resources are insufficient is generated.

[0011] In one possible implementation, the method further includes:

[0012] When the number of currently idle codec resources is less than the number to be allocated, not performing the first type of operation based on the codec resources;

[0013] A second type of operation is performed on the to-be-processed media stream using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0014] In one possible implementation, the method further includes:

[0015] If the first type of operation does not require consumption of codec resources, or the electronic device does not support processing the first type of operation, the first type of operation is not performed based on the codec resources;

[0016] A second type of operation is performed on the to-be-processed media stream using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0017] In one possible implementation, the method further includes:

[0018] During the initialization process, the processing capabilities of all codec resources are quantified to obtain the total number of all codec resources.

[0019] In a possible implementation, quantifying the processing capabilities of all codec resources to obtain the total number of all codec resources includes:

[0020] Based on the model of the codec resource hardware, determine the number of reference media streams that can be simultaneously processed by all codec resources when a preset reference encoding protocol or a preset reference decoding protocol is adopted, and use the obtained number as the total number of all codec resources, wherein the resolution of the reference media stream is the preset reference resolution, and the frame rate of the reference media stream is the preset reference frame rate.

[0021] In a possible implementation, determining the amount of codec resources required for the first type of operation based on the target protocol, the target resolution, and the target frame rate to obtain the amount to be allocated includes:

[0022] Determining, according to a predetermined first correspondence, a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol to obtain a first weight, wherein the first correspondence includes weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol, the preset benchmark encoding protocol or the preset benchmark decoding protocol being the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources;

[0023] Determining weights of the target resolution and the target frame rate relative to a baseline media stream, and using the determined weights as second weights, wherein the resolution of the baseline media stream is a preset baseline resolution, the frame rate of the baseline media stream is a preset baseline frame rate, and the baseline media stream is the media stream used when determining the total amount of all codec resources;

[0024] The amount of codec resources required for the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

[0025] In a possible implementation, determining the amount of codec resources required for the first type of operation based on the target protocol, the target resolution, and the target frame rate to obtain the amount to be allocated includes:

[0026] Determining a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol according to a predetermined first weight table to obtain a first weight; wherein the first weight table records the weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol;

[0027] determining whether a second weight of the target resolution and the target frame rate relative to the reference media stream exists in a predetermined second weight table, wherein the second weight table records the weight of each resolution and frame rate relative to the reference media stream; if so, determining the second weight of the target resolution and the target frame rate relative to the reference media stream based on the second weight table; and / or, if not, dividing the product of the target resolution and the target frame rate by the product of the resolution and the frame rate of the reference media stream to obtain the second weight;

[0028] The amount of codec resources required for the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

[0029] In a possible implementation manner, determining the weights of the target resolution and the target frame rate relative to the reference media stream to obtain the second weight includes:

[0030] Calculating a product of the target resolution and the target frame rate to obtain a first product;

[0031] Calculating a product of the preset reference resolution and the preset reference frame rate to obtain a second product;

[0032] A ratio of the first product to the second product is calculated to obtain a second weight.

[0033] In a possible implementation, the method further includes: determining a first weight corresponding to the target protocol based on the number of video frames after scalable video coding (SVC) scalability and the number of video frames before SVC scalability.

[0034] The embodiment of the present application further provides a coding and decoding resource allocation device, the device comprising:

[0035] An acquisition module, used for acquiring a first type of operation for a media stream to be processed;

[0036] A first determining module, configured to determine a target protocol, a target resolution, and a target frame rate corresponding to the first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0037] A second determining module is configured to determine the amount of codec resources required for the first type of operation according to the target protocol, the target resolution, and the target frame rate, and use the obtained amount as the amount to be allocated;

[0038] An allocation module is configured to allocate the number of codec resources to be allocated from the currently idle codec resources to the media stream to be processed when the number of currently idle codec resources is not less than the number to be allocated; and / or a generation module is configured to generate a prompt message indicating insufficient codec resources when the number of currently idle codec resources is less than the number to be allocated.

[0039] In a possible implementation, the device further includes:

[0040] The first processing module is configured to: not perform the first type of operation based on the codec resources when the number of currently idle codec resources is less than the number to be allocated; and perform the second type of operation on the media stream to be processed using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0041] In a possible implementation, the device further includes:

[0042] The second processing module is configured to: perform the first type of operation not based on codec resources when the first type of operation does not require codec resources or the electronic device does not support processing the first type of operation; and utilize processor resources to perform the second type of operation on the media stream to be processed, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0043] In a possible implementation, the device further includes:

[0044] The quantization module is used to quantify the processing capabilities of all codec resources during the initialization process to obtain the total number of all codec resources.

[0045] In a possible implementation, the quantization module is specifically configured to:

[0046] Based on the model of the codec resource hardware, determine the number of reference media streams that can be simultaneously processed by all codec resources when a preset reference encoding protocol or a preset reference decoding protocol is adopted, and use the obtained number as the total number of all codec resources, wherein the resolution of the reference media stream is the preset reference resolution, and the frame rate of the reference media stream is the preset reference frame rate.

[0047] In a possible implementation, the second determining module includes:

[0048] a first determination submodule, configured to determine, according to a predetermined first correspondence, a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, to obtain a first weight, wherein the first correspondence includes weights of various types of encoding protocols or decoding protocols relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, the preset benchmark encoding protocol or the preset benchmark decoding protocol being the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources;

[0049] a second determination submodule, configured to determine weights of the target resolution and the target frame rate relative to a reference media stream, and use the determined weights as second weights, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used when determining the total number of all codec resources;

[0050] A calculation submodule is used to calculate the amount of codec resources required for the first type of operation based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

[0051] In a possible implementation manner, the second determining submodule is specifically configured to:

[0052] Calculating a product of the target resolution and the target frame rate to obtain a first product;

[0053] Calculating a product of the preset reference resolution and the preset reference frame rate to obtain a second product;

[0054] A ratio of the first product to the second product is calculated to obtain a second weight.

[0055] An embodiment of the present application further provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0056] Memory for storing computer programs;

[0057] The processor is configured to implement any of the above-mentioned methods for allocating coding and decoding resources when executing a program stored in the memory.

[0058] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned methods for allocating coding and decoding resources is implemented.

[0059] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described methods for allocating coding and decoding resources.

[0060] Beneficial effects of the embodiments of the present application:

[0061] The codec resource allocation method, device and electronic device provided in the embodiments of the present application first obtain a first type of operation for the media stream to be processed, then determine the target protocol, target resolution and target frame rate corresponding to the first type of operation, determine the number of codec resources required for the first type of operation based on the target protocol, target resolution and target frame rate, and obtain the number to be allocated. When the number of currently idle codec resources is not less than the number to be allocated, allocate the number of codec resources to be allocated from the currently idle codec resources to the media stream to be processed. When the number of currently idle codec resources is less than the number to be allocated, generate a prompt message indicating insufficient codec resources. By combining the target protocol, resolution and frame rate to determine the codec resources required to perform the operation requiring codec resources on the code stream to be processed, resource calculation errors can be reduced and reasonable control and allocation of codec resources can be achieved.

[0062] Moreover, when the number of currently idle codec resources is less than the number to be allocated, the first type of operation is not performed, and the media stream to be processed is received at the same time, and the processor resources are used to perform operations that do not require codec resources on the media stream to be processed. Therefore, even if the operation that requires codec resources cannot be performed due to insufficient current codec resources, other operations that do not require codec resources can also be performed normally, thereby ensuring that the codec-related operation process is smoother.

[0063] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] FIG1 is a schematic diagram of a first flow chart of a method for allocating coding and decoding resources provided in an embodiment of the present application;

[0065] FIG2 is a schematic diagram of a second flow chart of a method for allocating coding and decoding resources according to an embodiment of the present application;

[0066] FIG3 is a schematic diagram of a third flow chart of the method for allocating coding and decoding resources provided in an embodiment of the present application;

[0067] FIG4 is a schematic diagram of a fourth flow chart of the method for allocating coding and decoding resources provided in an embodiment of the present application;

[0068] FIG5a is a schematic diagram of a fifth flow chart of the method for allocating coding and decoding resources provided in an embodiment of the present application;

[0069] FIG5 b is a first schematic diagram showing prompt information indicating insufficient codec resources in the codec resource allocation method provided in an embodiment of the present application;

[0070] FIG5c is a second schematic diagram showing a prompt message indicating insufficient codec resources in the codec resource allocation method provided in an embodiment of the present application;

[0071] FIG6 is a schematic structural diagram of a coding and decoding resource allocation device according to an embodiment of the present application;

[0072] FIG7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to achieve reasonable control and allocation of codec resources, embodiments of the present application provide a codec resource allocation method, apparatus, electronic device, storage medium, and computer program product containing instructions.

[0074] Below, we first introduce the codec resource allocation method provided in the embodiments of the present application. This method is applied to electronic devices. Specifically, the electronic device can be any electronic device that can provide media stream codec resource allocation services, such as a mobile phone, tablet, personal computer, server, etc. The codec resource allocation method provided in the embodiments of the present application can be implemented by at least one of software, hardware circuits, and logic circuits provided in the electronic device.

[0075] As shown in FIG1 , FIG1 is a schematic diagram of a first flow chart of a method for allocating coding and decoding resources provided in an embodiment of the present application, the method comprising:

[0076] S110, obtaining a first type of operation for the media stream to be processed;

[0077] S120, determining the target protocol, target resolution, and target frame rate corresponding to the above-mentioned first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol; wherein, for the encoding process, the target protocol (i.e., the target encoding protocol) is the encoding standard that the first type of operation conforms to, that is, the encoding standard that the data after the first type of operation conforms to, the target resolution is the resolution that the data after the first type of operation conforms to, and the target frame rate is the frame rate that the data after the first type of operation conforms to. For the decoding process, the target protocol (i.e., the target decoding protocol) is the decoding standard that the first type of operation conforms to, that is, the decoding standard that the data before the first type of operation conforms to, the target resolution is the resolution that the data before the first type of operation conforms to, and the target frame rate is the frame rate that the data before the first type of operation conforms to.

[0078] S130. Determine the amount of codec resources required for the first type of operation based on the target protocol, the target resolution, and the target frame rate, and use the obtained amount as the amount to be allocated. The amount of codec resources required for the first type of operation refers to the amount of codec resources required to perform the first type of operation, that is, how many codec resources are needed to complete the first type of operation.

[0079] S140: If the number of currently idle codec resources is not less than the number to be allocated, allocate the number of codec resources to be allocated to the to-be-processed media stream from the currently idle codec resources;

[0080] S150: If the number of currently idle codec resources is less than the number to be allocated, generate prompt information indicating insufficient codec resources. The number of currently idle codec resources refers to the number of codec resources currently remaining in the electronic device, that is, the number of unoccupied codec resources.

[0081] A media stream can be either video or audio. Some operations related to media streams, such as parsing, storage, and forwarding, can be referred to as second-category operations. These second-category operations only require stream parsing, storage, and forwarding, and do not require previewing or playback. Therefore, these second-category operations can be completed using either processor resources or codec resources. In this embodiment, second-category operations are defined as operations that require processor resources but do not require codec resources. Furthermore, some operations related to media streams, such as previewing and playback, can be referred to as first-category operations. First-category operations require codec resources. Therefore, when codec resource-related operations are enabled, a determination is first made as to whether the media stream operation being processed is a first-category operation or a second-category operation. If it is a first-category operation that requires codec resources, the amount of codec resources required is calculated based on the target protocol, target resolution, and target frame rate. The system then determines whether sufficient codec resources are available. If sufficient, codec resources are allocated. Furthermore, after allocating codec resources, the currently available codec resources are updated. If insufficient codec resources are available, a prompt indicating insufficient codec resources is generated.

[0082] It should be noted that the number of codec resources refers to a parameter obtained by quantifying the codec capability, which is used to measure the size of the codec capability. Quantifying the codec capability into calculable units helps to judge the degree of match between resources and needs. In addition, it should be noted that "codec" in this article is a synonym for "coding or decoding". In the decoding scenario, the word means "decoding"; in the encoding scenario, it means "coding". For example, a method for allocating codec resources can be understood as: a method for allocating coding resources, or a method for allocating decoding resources. Correspondingly, the embodiments given in this application and the schemes to be protected can be correspondingly understood as a scheme related to coding, or a scheme related to decoding, and each complete scheme will not be described one by one here.

[0083] By combining the target protocol, resolution, and frame rate to determine the codec resources required to perform operations that require codec resources on the processed bitstream, resource calculation errors can be reduced and reasonable control and allocation of codec resources can be achieved.

[0084] In a possible implementation, the above method further includes:

[0085] When the number of currently idle codec resources is less than the above-mentioned number to be allocated, the above-mentioned first type of operation is not performed (that is, the above-mentioned first type of operation is not performed based on the codec resources); the above-mentioned media stream to be processed is received, and the second type of operation is performed on the above-mentioned media stream to be processed using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and codec resources are different data processing resources.

[0086] When the number of currently idle codec resources is less than the number to be allocated, it means that there are currently insufficient codec resources available. Although codec resources are required to complete operations such as media stream parsing, storage, and forwarding, they can also be completed through processor resources. Therefore, since there are insufficient codec resources available, no encoding or decoding operations are performed at this time. Instead, the media stream to be processed is received, and processor resources are used to perform operations on the media stream to be processed, such as parsing, storing, and forwarding. In this way, when codec resources are insufficient, even if no encoding or decoding operations are performed, processor resources can still be reasonably used to maintain the parsing, storage, and forwarding of the code stream.

[0087] In a possible implementation, the above method further includes:

[0088] When the above-mentioned first-category operation does not require consumption of codec resources, or the electronic device does not support processing the above-mentioned first-category operation, the above-mentioned first-category operation is not performed (that is, the above-mentioned first-category operation is not performed based on codec resources); the above-mentioned media stream to be processed is received, and the second-category operation is performed on the above-mentioned media stream to be processed using processor resources, wherein the second-category operation and the first-category operation are different operations, and the processor resources and codec resources are different data processing resources.

[0089] When the first type of operation does not require consumption of encoding and decoding resources or the electronic device does not support processing the first type of operation, the first type of operation is not performed. Instead, after receiving the media stream to be processed, the processor resources are used to perform the second type of operation on the media stream to be processed, such as parsing, storage, forwarding, etc.

[0090] Or when the electronic device cannot support the processing of the first type of operation, for example, the electronic device only supports the processing of H.264 and H.265 protocol codec operations, when the first type of operation is the codec operation of other types of protocols other than H.264 and H.265 protocols, then the electronic device does not support the processing of the first type of operation. After the electronic device receives the media stream to be processed, it uses the processor resources to perform operations on the media stream to be processed. In this way, the first type of operation does not need to consume codec resources, or when the electronic device does not support the processing of the first type of operation, even if the codec operation is not performed, the processor resources can still be reasonably used to maintain the parsing, storage, and forwarding of the code stream.

[0091] In a possible implementation, the above method further includes:

[0092] During the initialization process, the processing capabilities of all codec resources are quantified to obtain the total number of all codec resources.

[0093] After initialization, the processing capabilities of all codec resources need to be quantified to obtain the total number of all codec resources. When quantifying the processing capabilities of all codec resources, a baseline media stream can be set first, and the processing capabilities of all codec resources can be quantified based on the baseline media stream. To clarify the solution and layout, the following is a detailed description in conjunction with another embodiment.

[0094] In a possible implementation, during the initialization process, the processing capabilities of all codec resources are quantified to obtain the total number of all codec resources, including:

[0095] During the initialization process, based on the model of the codec resource hardware, the number of reference media streams that can be processed simultaneously by all codec resources when using a preset reference encoding protocol or a preset reference decoding protocol is determined, and the total number of all codec resources is obtained, wherein the resolution of the above-mentioned reference media stream is the preset reference resolution, and the frame rate of the above-mentioned reference media stream is the preset reference frame rate.

[0096] During initialization, a baseline media stream is first set. For example, the H.265 1080P@30FPS decoding media stream is used as the baseline media stream. That is, the resources occupied by decoding one H.265 1080P@30FPS channel are set to KP (K Proportionality). Based on this baseline media stream, the number of baseline media streams that the codec resources are allowed to process simultaneously is determined to obtain the total number of codec resources. For example, the total codec capacity is x × KP. For example, if a device's VCP has a total decoding capacity of 16 H.265 1080P@30FPS videos, the total codec capacity is 16 × KP, meaning it can simultaneously decode 16 1080P@30FPS media streams. The setting of the baseline media stream can be based on actual conditions and is not limited here.

[0097] Based on the embodiment of FIG1 , referring to FIG2 , FIG2 is a schematic flow chart of a second method for allocating coding and decoding resources provided in an embodiment of the present application. The above step S130 can be implemented by the following steps:

[0098] S131: Determine a weight of the target protocol relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol based on the target protocol and the preset benchmark encoding protocol or the preset benchmark decoding protocol, thereby obtaining a first weight. The preset benchmark encoding protocol or the preset benchmark decoding protocol is the encoding protocol or the decoding protocol used when determining the total number of all encoding and decoding resources.

[0099] S132: Determine weights of the target resolution and the target frame rate relative to a reference media stream to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used when determining the total number of all codec resources;

[0100] S133, calculating the amount of codec resources required for the first type of operation based on the first weight and the second weight to obtain a to-be-allocated amount; wherein the to-be-allocated amount of codec resources required for the first type of operation may be the product of the first weight and the second weight.

[0101] After obtaining the media stream to be processed, if the first type of operation is performed on the media stream to be processed, the amount of encoding and decoding resources required for the first type of operation is related to the target protocol, resolution, and frame rate.

[0102] After setting the preset baseline encoding protocol or the preset baseline decoding protocol, the weight of the target protocol relative to the preset baseline encoding protocol or the preset baseline decoding protocol can be determined. For example, the H.265 1080P@30FPS decoding media stream is used as the baseline media stream, that is, the H.265 protocol is set as the baseline decoding protocol. Then, relative to the H.265 protocol, the weight of the target decoding protocol in the H.265 decoding scenario is 1. Because the encoding performance consumption will be higher than the decoding, the H.265 encoding can set different weights according to the measured performance consumption.

[0103] For H.264 encoding and decoding, because the performance consumption of H.264 is higher than that of H.265, and the encoding performance consumption is higher than that of decoding, the corresponding weights of H.264 encoding or decoding can be obtained through performance testing or theoretical calculation.

[0104] For characteristic bitstream processing, such as SVC (Scalable Video Coding) bitstream, since SVC bitstream mainly consists of time domain classification and spatial domain classification, its main function is to compress the media stream. The SVC bitstream in this application generally refers to time domain classification, that is, some frames in the media stream can be not decoded, reducing decoding performance consumption, that is, the reduction of video frames mainly for encoding and decoding will not change the original data stream. The first weight corresponding to its target protocol is determined based on the number of video frames after SVC classification. For example, if the number of video frames after SVC classification is 1 / 2 of the original (that is, the number of video frames involved in encoding and decoding is 1 / 2 of the original number of video frames), the first weight is 0.5; if the number of video frames is 1 / 4 of the original, the first weight is 0.25.

[0105] The second weight corresponding to the media stream to be processed is determined based on the resolution of the baseline media stream, the frame rate of the baseline media stream, the target resolution and the target frame rate. The second weight corresponding to the media stream to be processed can be calculated by dividing the product of the resolution and frame rate of the media stream to be processed by the product of the resolution and frame rate of the baseline media stream.

[0106] In a possible implementation, the above step S132 may be implemented by the following steps:

[0107] Calculating the product of the target resolution and the target frame rate to obtain a first product;

[0108] Calculating the product of the preset reference resolution and the preset reference frame rate to obtain a second product;

[0109] The ratio of the first product to the second product is calculated to obtain a second weight.

[0110] Similarly, using the H.265 1080P@30FPS decoded media stream as the benchmark media stream, and setting the resources occupied by decoding one channel of H.265 1080P@30FPS as KP, for the H.265 decoding scenario, for the to-be-processed media stream with CIF 10w pixels, 352×288 resolution, and 30 frame rate, the weight is 352×288×30 / (1920×1080×30)≈0.05.

[0111] The amount of codec resources consumed by the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated. Specifically, the amount of codec resources consumed by the first type of operation (i.e., the amount to be allocated) is the product of the first weight and the second weight.

[0112] For example, take the H.265 1080P@30FPS decoding media stream as the benchmark media stream, and set the resources occupied by decoding one channel of H.265 1080P@30FPS as KP. For a typical H.265 decoding scenario:

[0113] Among them, the decoding weight table (i.e., the second weight table) is shown in Table 1:

[0114] Table 1

[0115] For other H.265 decoding scenarios:

[0116] For H.265 atypical resolution and frame rate video, taking 1080P as the benchmark, the product of the resolution and frame rate of the to-be-processed media stream is divided by the product of the resolution and frame rate of the benchmark media stream, and the result is linearly calculated and rounded to the nearest integer. For example, for a 1000×1000@25FPS video, the second weight corresponding to the to-be-processed media stream is 1000×1000×25 / (1920×1080×30)≈0.4. Therefore, decoding the 1000×1000@25FPS video occupies 0.4KP of resources.

[0117] For H.264 encoding and decoding:

[0118] If the H.264 encoding or decoding meets the resolution and frame rate information in Table 1, the second weight in Table 1 can be directly multiplied by the first weight corresponding to the H.264 encoding or decoding; if it does not meet the resolution and frame rate information in Table 1, the product of the resolution and frame rate of the media stream to be processed is divided by the product of the resolution and frame rate of the reference media stream to obtain the second weight, and then the second weight is multiplied by the first weight corresponding to the corresponding H.264 encoding or decoding.

[0119] For example, if the second weight corresponding to H.264 / JPG is k1 and m is the first weight corresponding to the H.264 encoding protocol, then the resources occupied by H.264 / JPG decoding are KP2 = (m × K1)KP. The resources occupied by H.264 / JPG encoding are KP3 = n × KP2, where n is the ratio of encoding to decoding.

[0120] This application targets the characteristics of hardware units, unifies the management of encoding and decoding resources, rationally plans encoding and decoding resources, and designs weight values ​​based on various target protocols, resolutions, operating scenarios, etc., which can reduce resource calculation errors and thus achieve reasonable control and allocation of encoding and decoding resources.

[0121] Based on the embodiment of FIG. 1 , in a possible implementation, the above step S130 may be implemented by the following steps:

[0122] According to a predetermined first correspondence, the weight of the target protocol relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol is determined to obtain a first weight, wherein the first correspondence includes the weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol, and the preset benchmark encoding protocol or the preset benchmark decoding protocol is the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources; for example, since some encoding protocols or decoding protocols do not need to encode or decode all video frames, but only encode or decode some video frames, therefore, for a certain type of encoding protocol or decoding protocol, based on the number of video frames corresponding to the encoding protocol or decoding protocol of this type (i.e., the number of video frames per unit time), The first weight is determined by the ratio between the number of video frames involved in encoding or decoding (i.e., the number of all or part of all video frames in all video frames) and the number of video frames corresponding to the preset benchmark encoding protocol or preset benchmark decoding protocol (i.e., the number of video frames involved in encoding or decoding per unit time, i.e., the number of all or part of all video frames in all video frames). Alternatively, the first weight is determined by the ratio between the number of video frames involved in encoding or decoding (or the number of video frames not involved in encoding or decoding) of the encoding protocol or decoding protocol of that type and the total number of video frames, and the ratio between the number of video frames involved in encoding or decoding (or the number of video frames not involved in encoding or decoding) of the preset benchmark encoding protocol or preset benchmark decoding protocol and the total number of video frames. For example, if the ratio of the number of video frames involved in encoding or decoding of protocol type A to the total number of video frames is 100%, and the ratio of the number of video frames involved in encoding or decoding of the preset benchmark encoding protocol or preset benchmark decoding protocol to the total number of video frames is 50%, then the first weight is 2. For another example, if the ratio of the number of video frames involved in encoding or decoding for protocol type A to the total number of video frames is 25%, and the ratio of the number of video frames involved in encoding or decoding for the preset baseline encoding protocol or the preset baseline decoding protocol to the total number of video frames is 50%, then the first weight is 0.5. The purpose of the first weight is to clearly determine the encoding and decoding resources required for the frames that actually need to be encoded or decoded. For example, see Table 2, which shows an example of the first correspondence.

[0123] Table 2

[0124] Determining weights of the target resolution and the target frame rate relative to a reference media stream to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used when determining the total amount of all codec resources;

[0125] The amount of codec resources consumed by the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated. The amount of codec resources consumed by the first type of operation is the product of the first weight and the second weight.

[0126] The first correspondence between the weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol can be pre-stored. In this way, the weight of the target protocol relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol can be directly determined according to the predetermined first correspondence to obtain the first weight, thereby reducing the amount of calculation.

[0127] Specifically, the first correspondence can be stored as a first weight table (i.e., the first weight table and the first correspondence can be the same, as shown in Table 2), so that the weight of the target protocol relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol can be directly determined based on the first weight table to obtain the first weight. Specifically, the first weight table records the weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol. After obtaining the target protocol, the target protocol type is matched with the corresponding encoding protocol or decoding protocol type in the first weight table to determine the first weight corresponding to the target protocol.

[0128] Of course, the weights of the target resolution and target frame rate relative to the reference media stream can also be determined according to a predetermined second correspondence to obtain a second weight. Specifically, the second correspondence can be stored as a second weight table, so that the second weights of the target resolution and target frame rate relative to the reference media stream can be directly determined based on the second weight table. Specifically, the second weight table records the weights of each resolution and frame rate relative to the reference media stream. After obtaining the target resolution and target frame rate, the target resolution and target frame rate are matched with the corresponding resolution and frame rate in the second weight table to determine the second weights of the target resolution and target frame rate relative to the reference media stream.

[0129] Based on the embodiment of FIG1 , referring to FIG3 , FIG3 is a schematic diagram of a third flow chart of the encoding and decoding resource allocation method provided in an embodiment of the present application. The above step S130 can be implemented by the following steps:

[0130] S1310: Determine a first weight value of the target protocol relative to a preset reference encoding protocol or a preset reference decoding protocol according to a predetermined first weight table;

[0131] S1320, determining whether a second weight value of the target resolution and the target frame rate relative to the reference media stream exists in a predetermined second weight table;

[0132] S1330, if present, determining second weights of the target resolution and the target frame rate relative to the reference media stream based on the second weight table;

[0133] S1340: If the target media stream resolution and the target frame rate do not exist, divide the product of the target media stream resolution and the target frame rate by the product of the reference media stream resolution and the reference media stream frame rate to obtain a second weight.

[0134] The first weight table and the second weight table may be different tables or the same table, which is not limited here.

[0135] If the second weight table contains second weights for the target resolution and target frame rate relative to the baseline media stream, the second weight can be obtained directly from the second weight table. If the media stream to be processed is an unconventional media stream, the second weight can be obtained through linear calculation. In this way, the corresponding weights of the media stream to be processed with typical resolution and code stream can be directly obtained from the table, and the media stream to be processed with atypical resolution and code stream can be determined through calculations such as linear conversion. This can reduce resource calculation errors and thus formulate reasonable strategies for resource management.

[0136] In one possible implementation, determining the amount of codec resources required for the first type of operation based on the target protocol, target resolution, and target frame rate to obtain the amount to be allocated includes:

[0137] Determining whether the number of codec resources required to be consumed by the first type of operation exists in a predetermined codec resource table;

[0138] If so, determining the amount of codec resources required for the first type of operation based on a predetermined codec resource table;

[0139] If not, the amount of encoding and decoding resources required for the first type of operation is determined according to the method described in any of the above embodiments.

[0140] If the media stream to be processed is a regular one, the amount of codec resources required for the first type of operation can be directly obtained from a pre-determined codec resource table. If the media stream to be processed is an unconventional one, the required amount can be determined using a linear calculation method, which can reduce resource calculation errors and thus formulate reasonable resource management strategies.

[0141] Based on the above embodiments, the present application provides a method for allocating coding and decoding resources. As shown in FIG4 , FIG4 is a fourth flow chart of the method for allocating coding and decoding resources provided by the present application. In this method, the decoding operation is taken as an example. The method includes:

[0142] S401, obtaining a decoding task for a media stream to be processed;

[0143] S402, determining whether a preset situation is included; the preset situation includes special situations such as unsupported resolution and virtual decoding, where virtual decoding means that no decoding operation is actually performed and no decoding resources are consumed; if so, executing step S408, otherwise executing step S403;

[0144] S403, determining the number of encoding and decoding resources required for the decoding task, and obtaining the number to be allocated;

[0145] S404, determining whether the number of currently idle codec resources is not less than the number to be allocated; if so, executing step S405, otherwise executing step S408;

[0146] S405, allocating a waiting number of codec resources to the media stream to be processed;

[0147] S406, updating the number of currently idle codec resources;

[0148] S407, decoding;

[0149] S408, operations that do not require encoding and decoding resources;

[0150] In the above method, even if an operation requiring codec resources cannot be performed due to insufficient codec resources in the current system, other operations not requiring decoding resources can still be performed normally, thereby ensuring that codec-related operation processes proceed more smoothly.

[0151] In one possible implementation, when an instruction to stop performing a preset operation on a pending media stream is received, the preset operation is an operation requiring encoding and decoding resources, and the method includes:

[0152] Determining whether encoding and decoding resources have been allocated for the above-mentioned media stream to be processed;

[0153] In a case where codec resources are not allocated, performing a second type of operation on the to-be-processed media stream using processor resources, wherein the second type of operation is an operation that does not require codec resources;

[0154] In the case that codec resources have been allocated, the allocated codec resources are released and the currently idle codec resources are updated.

[0155] Specifically, as shown in FIG5 , FIG5 is a schematic diagram of a fifth flow chart of the encoding and decoding resource allocation method provided in an embodiment of the present application, the method comprising:

[0156] S501, receiving an instruction to stop decoding a media stream to be processed;

[0157] S502, determine whether the codec resources are occupied; if so, execute step S503, if not, execute step S505;

[0158] S503, releasing the occupied codec resources;

[0159] S504, updating currently idle codec resources;

[0160] S505: Utilize processor resources to perform a second type of operation on the media stream to be processed.

[0161] When you need to close the encoding or decoding resource operation, first determine whether the current channel has occupied encoding or decoding resources. If so, release the occupied encoding or decoding resources. If not, close the channel directly.

[0162] In a possible implementation, when the number of currently idle codec resources is less than the number to be allocated, after generating prompt information indicating insufficient codec resources, the method includes:

[0163] A prompt message indicating insufficient codec resources is displayed in the preview window corresponding to the media stream to be processed.

[0164] The NVR connects to digital cameras via a network, receives audio and video data from the cameras, and performs analysis, storage, forwarding, preview, and playback on the data. During preview, the NVR preview screen is related to the number of split screens configured on the NVR. If the number of split screens is 4, the preview screen windows will be 4, arranged in a 2*2 pattern. Furthermore, it is also related to the preview channels configured on the NVR. If channel 3 is configured in preview window 1, preview window 1 will see the media stream from channel 3.

[0165] For example, there are currently 4 preview windows, of which window 1 corresponds to the media stream of playback channel 1, window 2 corresponds to the media stream of playback channel 2, window 3 corresponds to the media stream of playback channel 3, and window 4 corresponds to the media stream of playback channel 4. When encoding and decoding the media stream corresponding to channel 3, if the number of currently idle encoding and decoding resources is less than the number to be allocated, it means that the current encoding and decoding resources cannot meet the requirements for encoding and decoding the media stream corresponding to channel 3. In this case, a prompt message of insufficient encoding and decoding resources will be displayed in the preview window corresponding to channel 3. The prompt message can be any information that can represent insufficient encoding and decoding resources, for example, the prompt message directly indicates insufficient decoding performance. Furthermore, it can also include information reminding you to modify parameters, as shown in Figure 5b.

[0166] Of course, when a channel cannot be opened, a system resource shortage window can be popped up directly on the NVR interface, as shown in Figure 5c. The specific form of the prompt message of insufficient codec resources and how to display the prompt message of insufficient codec resources can be set according to actual needs and are not limited here.

[0167] Based on the above method embodiment, an embodiment of the present application provides a coding and decoding resource allocation device, as shown in FIG6 , which is a schematic structural diagram of the coding and decoding resource allocation device provided in an embodiment of the present application. The device may include:

[0168] An acquisition module 610 is configured to acquire a first type of operation for a media stream to be processed;

[0169] A first determination module 620 is configured to determine a target protocol, a target resolution, and a target frame rate corresponding to the first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0170] A second determining module 630 is configured to determine the amount of codec resources required for the first type of operation based on the target protocol, the target resolution, and the target frame rate, and obtain an amount to be allocated;

[0171] An allocation module 640 is configured to allocate the number of codec resources to be allocated from the currently idle codec resources to the media stream to be processed when the number of currently idle codec resources is not less than the number to be allocated; and / or a generation module 650 is configured to generate a prompt message indicating insufficient codec resources when the number of currently idle codec resources is less than the number to be allocated.

[0172] In a possible implementation, the above device further includes:

[0173] The first processing module is configured to: not perform the first type of operation based on the codec resources when the number of currently idle codec resources is less than the number to be allocated; and perform the second type of operation on the media stream to be processed using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0174] In a possible implementation, the above device further includes:

[0175] The second processing module is configured to: perform the first type of operation not based on codec resources when the first type of operation does not require codec resources or the electronic device does not support processing the first type of operation; and utilize processor resources to perform the second type of operation on the media stream to be processed, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

[0176] In a possible implementation, the above device further includes:

[0177] The quantization module is used to quantify the processing capabilities of all codec resources during the initialization process to obtain the total number of all codec resources.

[0178] In a possible implementation, the quantization module is specifically configured to:

[0179] Based on the model of the codec resource hardware, determine the number of reference media streams that can be simultaneously processed by all codec resources when a preset reference encoding protocol or a preset reference decoding protocol is adopted, and use the obtained number as the total number of all codec resources, wherein the resolution of the reference media stream is the preset reference resolution, and the frame rate of the reference media stream is the preset reference frame rate.

[0180] In a possible implementation, the second determining module includes:

[0181] a first determination submodule, configured to determine, according to a predetermined first correspondence, a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, to obtain a first weight, wherein the first correspondence includes weights of various types of encoding protocols or decoding protocols relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, the preset benchmark encoding protocol or the preset benchmark decoding protocol being the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources;

[0182] a second determination submodule, configured to determine weights of the target resolution and the target frame rate relative to a reference media stream, and use the determined weights as second weights, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used when determining the total number of all codec resources;

[0183] A calculation submodule is used to calculate the amount of codec resources required for the first type of operation based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

[0184] In a possible implementation manner, the second determining submodule is specifically configured to:

[0185] Calculating the product of the target resolution and the target frame rate to obtain a first product;

[0186] Calculating the product of the preset reference resolution and the preset reference frame rate to obtain a second product;

[0187] The ratio of the first product to the second product is calculated to obtain a second weight.

[0188] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0189] The embodiment of the present application further provides an electronic device, as shown in FIG7 , including a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other via the communication bus 704.

[0190] Memory 703, used for storing computer programs;

[0191] The processor 701 is configured to execute the program stored in the memory 703, and implement the following steps:

[0192] Get the first type of operation for the media stream to be processed;

[0193] Determining a target protocol, target resolution, and target frame rate corresponding to the first type of operation;

[0194] Determining the amount of codec resources required for the first type of operation according to the target protocol, the target resolution, and the target frame rate, and obtaining an amount to be allocated;

[0195] When the number of currently idle codec resources is not less than the number to be allocated, allocating the number of codec resources to be allocated to the to-be-processed media stream from the currently idle codec resources;

[0196] When the number of currently idle codec resources is less than the number to be allocated, prompt information indicating that the codec resources are insufficient is generated.

[0197] Optionally, when the processor 701 is configured to execute a program stored in the memory 703 , it may further implement any of the above-mentioned encoding and decoding resource allocation methods.

[0198] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0199] The communication interface is used for communication between the above electronic device and other devices.

[0200] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0201] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0202] In another embodiment provided by the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned coding and decoding resource allocation methods are implemented.

[0203] In another embodiment provided by the present application, a computer program product including instructions is further provided, which, when executed on a computer, enables the computer to execute any one of the encoding and decoding resource allocation methods in the above embodiments.

Claims

1. A method for allocating coding and decoding resources, characterized in that: The method comprises: Get the first type of operation for the media stream to be processed; Determining a target protocol, a target resolution, and a target frame rate corresponding to the first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol; Determining the amount of codec resources required for the first type of operation according to the target protocol, the target resolution, and the target frame rate, and using the obtained amount as the amount to be allocated; When the number of currently idle codec resources is not less than the number to be allocated, allocating the number of codec resources to be allocated to the to-be-processed media stream from the currently idle codec resources; and / or, When the number of currently idle codec resources is less than the number to be allocated, prompt information indicating that the codec resources are insufficient is generated.

2. The method according to claim 1, characterized in that The method further comprises: When the number of currently idle codec resources is less than the number to be allocated, not performing the first type of operation based on the codec resources; A second type of operation is performed on the to-be-processed media stream using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

3. The method according to claim 1, characterized in that The method further comprises: If the first type of operation does not require consumption of codec resources, or the electronic device does not support processing the first type of operation, the first type of operation is not performed based on the codec resources; A second type of operation is performed on the to-be-processed media stream using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

4. The method according to claim 1, wherein The method further comprises: During the initialization process, the processing capabilities of all codec resources are quantified to obtain the total number of all codec resources.

5. The method according to claim 4, characterized in that The processing capabilities of all codec resources are quantified to obtain the total number of all codec resources, including: Based on the model of the codec resource hardware, determine the number of reference media streams that can be simultaneously processed by all codec resources when a preset reference encoding protocol or a preset reference decoding protocol is adopted, and use the obtained number as the total number of all codec resources, wherein the resolution of the reference media stream is the preset reference resolution, and the frame rate of the reference media stream is the preset reference frame rate.

6. The method according to claim 1, characterized in that The determining, based on the target protocol, the target resolution, and the target frame rate, the amount of codec resources required for the first type of operation to be consumed, and obtaining the amount to be allocated, includes: Determining, according to a predetermined first correspondence, a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol to obtain a first weight, wherein the first correspondence includes weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol, the preset benchmark encoding protocol or the preset benchmark decoding protocol being the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources; Determining weights of the target resolution and the target frame rate relative to a baseline media stream, and using the determined weights as second weights, wherein the resolution of the baseline media stream is a preset baseline resolution, the frame rate of the baseline media stream is a preset baseline frame rate, and the baseline media stream is the media stream used when determining the total amount of all codec resources; The amount of codec resources required for the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

7. The method according to claim 1, characterized in that The determining, based on the target protocol, the target resolution, and the target frame rate, the amount of codec resources required for the first type of operation to be consumed, and obtaining the amount to be allocated, includes: Determining a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol according to a predetermined first weight table to obtain a first weight; wherein the first weight table records the weights of various types of encoding protocols or decoding protocols relative to the preset benchmark encoding protocol or the preset benchmark decoding protocol; determining whether a second weight of the target resolution and the target frame rate relative to the reference media stream exists in a predetermined second weight table, wherein the second weight table records the weight of each resolution and frame rate relative to the reference media stream; if so, determining the second weight of the target resolution and the target frame rate relative to the reference media stream based on the second weight table; and / or, if not, dividing the product of the target resolution and the target frame rate by the product of the resolution and the frame rate of the reference media stream to obtain the second weight; The amount of codec resources required for the first type of operation is calculated based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

8. The method according to claim 6 or 7, characterized in that Determining the weights of the target resolution and the target frame rate relative to the reference media stream to obtain a second weight includes: Calculating a product of the target resolution and the target frame rate to obtain a first product; Calculating a product of the preset reference resolution and the preset reference frame rate to obtain a second product; A ratio of the first product to the second product is calculated to obtain a second weight.

9. The method according to claim 6 or 7, characterized in that The method further includes: determining a first weight corresponding to the target protocol based on the number of video frames after scalable video coding (SVC) scalability and the number of video frames before SVC scalability.

10. A coding and decoding resource allocation device, characterized in that: The device comprises: An acquisition module, used for acquiring a first type of operation for a media stream to be processed; A first determining module, configured to determine a target protocol, a target resolution, and a target frame rate corresponding to the first type of operation, wherein the target protocol is a target encoding protocol or a target decoding protocol; A second determining module is configured to determine the amount of codec resources required for the first type of operation according to the target protocol, the target resolution, and the target frame rate, and use the obtained amount as the amount to be allocated; An allocation module is configured to allocate the number of codec resources to be allocated from the currently idle codec resources to the media stream to be processed when the number of currently idle codec resources is not less than the number to be allocated; and / or a generation module is configured to generate a prompt message indicating insufficient codec resources when the number of currently idle codec resources is less than the number to be allocated.

11. The device according to claim 10, characterized in that The device further comprises: The first processing module is configured to: not perform the first type of operation based on the codec resources when the number of currently idle codec resources is less than the number to be allocated; and perform the second type of operation on the media stream to be processed using processor resources, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

12. The device according to claim 10, characterized in that The device further comprises: The second processing module is configured to: perform the first type of operation not based on codec resources when the first type of operation does not require codec resources or the electronic device does not support processing the first type of operation; and utilize processor resources to perform the second type of operation on the media stream to be processed, wherein the second type of operation is different from the first type of operation, and the processor resources and the codec resources are different data processing resources.

13. The device according to claim 10, characterized in that The device further comprises: The quantization module is used to quantify the processing capabilities of all codec resources during the initialization process to obtain the total number of all codec resources.

14. The device according to claim 13, characterized in that The quantization module is specifically used for: Based on the model of the codec resource hardware, determine the number of reference media streams that can be simultaneously processed by all codec resources when a preset reference encoding protocol or a preset reference decoding protocol is adopted, and use the obtained number as the total number of all codec resources, wherein the resolution of the reference media stream is the preset reference resolution, and the frame rate of the reference media stream is the preset reference frame rate.

15. The device according to claim 10, characterized in that The second determining module includes: a first determination submodule, configured to determine, according to a predetermined first correspondence, a weight of the target protocol relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, to obtain a first weight, wherein the first correspondence includes weights of various types of encoding protocols or decoding protocols relative to a preset benchmark encoding protocol or a preset benchmark decoding protocol, the preset benchmark encoding protocol or the preset benchmark decoding protocol being the encoding protocol or decoding protocol used when determining the total number of all encoding and decoding resources; a second determination submodule, configured to determine weights of the target resolution and the target frame rate relative to a reference media stream, and use the determined weights as second weights, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used when determining the total number of all codec resources; A calculation submodule is used to calculate the amount of codec resources required for the first type of operation based on the first weight and the second weight to obtain the amount to be allocated; wherein the amount of codec resources required for the first type of operation to be allocated is the product of the first weight and the second weight.

16. The device according to claim 15, characterized in that The second determining submodule is specifically configured to: Calculating a product of the target resolution and the target frame rate to obtain a first product; Calculating a product of the preset reference resolution and the preset reference frame rate to obtain a second product; A ratio of the first product to the second product is calculated to obtain a second weight.

17. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 9 when executing a program stored in a memory.