Coding resource allocation method, apparatus and electronic device

Abstract

Embodiments of the present application provide a method and device for coding resource allocation and an electronic device. A to-be-processed operation for a to-be-processed media stream is first acquired, and then a target protocol, a target resolution and a target frame rate corresponding to the to-be-processed operation are determined. The number of coding resources required by the to-be-processed operation is determined according to the target protocol, the target resolution and the target frame rate, and a to-be-allocated number is obtained. In a case where the number of currently idle coding resources is not less than the to-be-allocated number, the to-be-allocated number of coding resources is allocated to the to-be-processed media stream from the currently idle coding resources. In a case where the number of currently idle coding resources is less than the to-be-allocated number, prompt information indicating that the coding resources are insufficient is generated. By combining the code stream type, the resolution and the frame rate to determine the corresponding coding resources, the resource calculation error can be reduced, and the reasonable control and allocation of the coding resources can be implemented.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to encoding / decoding resource allocation methods, apparatuses, and electronic devices. Background Technology

[0002] Currently, in the process of image encoding, decoding, and transmission, some video recorders, such as NVRs (Network Video Recorders), use the same VCP (Video Codec Processor) to implement encoding and decoding functions. When the hardware unit has reached its bottleneck, adding operations requiring encoding and decoding resources, such as previewing, without control will affect the encoding and decoding performance and processing effect of the ongoing operation, causing problems such as preview stuttering, data anomalies, and data loss. Therefore, it is necessary to rationally control and allocate encoding and decoding resources according to the encoding and decoding capabilities of the hardware unit. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, and electronic device for allocating encoding and decoding resources in a reasonable manner. The specific technical solution is as follows:

[0004] This application provides a method for allocating encoding and decoding resources, the method comprising:

[0005] Retrieve pending operations for the media stream to be processed;

[0006] Determine the target protocol, target resolution, and target frame rate corresponding to the operation to be processed, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0007] Based on the target protocol, the target resolution, and the target frame rate, determine the amount of encoding and decoding resources required for the operation to be processed, and obtain the amount to be allocated;

[0008] If the number of currently available codec resources is not less than the number to be allocated, allocate the number of codec resources to be allocated for the media stream to be processed from the currently available codec resources;

[0009] If the number of currently available codec resources is less than the number to be allocated, a message indicating insufficient codec resources is generated.

[0010] In one possible implementation, the method further includes:

[0011] If the number of currently available codec resources is less than the number to be allocated, the pending operation will not be executed;

[0012] The processor receives the media stream to be processed and performs a specified operation on the media stream using processor resources, wherein the specified operation is an operation that does not require encoding / decoding resources.

[0013] In one possible implementation, the method further includes:

[0014] If the pending operation does not require the consumption of encoding / decoding resources, or if the processing of the pending operation is not supported, the pending operation will not be executed.

[0015] The processor receives the media stream to be processed and performs a specified operation on the media stream using processor resources, wherein the specified operation is an operation that does not require encoding / decoding resources.

[0016] In one possible implementation, the method further includes:

[0017] During initialization, the processing capabilities of all encoding and decoding resources are quantified to obtain the total number of all encoding and decoding resources.

[0018] In one possible implementation, the initialization process involves quantifying the processing capabilities of all codec resources to obtain the total number of all codec resources, including:

[0019] During initialization, based on the model of the encoding / decoding resource hardware, the number of reference media streams that can be processed simultaneously under the use of a preset reference encoding or decoding protocol is determined, thus obtaining the total number of all encoding / decoding resources. The resolution of the reference media stream is a preset reference resolution, and the frame rate of the reference media stream is a preset reference frame rate.

[0020] In one possible implementation, determining the amount of codec resources required by the operation to be processed, based on the target protocol, the target resolution, and the target frame rate, to obtain the amount to be allocated, includes:

[0021] According to a predetermined first correspondence, the weight of the target protocol relative to a preset benchmark encoding or decoding protocol is determined to obtain a first weight. The first correspondence includes the weight of each type of encoding or decoding protocol relative to the preset benchmark encoding or decoding protocol. The preset benchmark encoding or decoding protocol is the encoding or decoding protocol used to determine the total number of all encoding and decoding resources.

[0022] The target resolution and the target frame rate are determined relative to the reference media stream to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used to determine the total number of all codec resources;

[0023] Based on the first weight and the second weight, the amount of encoding and decoding resources required for the operation to be processed is calculated to obtain the amount to be allocated.

[0024] In one possible implementation, determining the weights of the target resolution and the target frame rate relative to the reference media stream to obtain the second weight includes:

[0025] Calculate the product of the target resolution and the target frame rate to obtain the first product;

[0026] Calculate the product of the preset baseline resolution and the preset baseline frame rate to obtain the second product;

[0027] Calculate the ratio of the first product to the second product to obtain the second weight.

[0028] This application embodiment also provides a codec resource allocation apparatus, the apparatus comprising:

[0029] The acquisition module is used to acquire pending operations for the media stream to be processed;

[0030] The first determining module is used to determine the target protocol, target resolution, and target frame rate corresponding to the operation to be processed, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0031] The second determining module is used to determine the amount of encoding and decoding resources required by the operation to be processed based on the target protocol, the target resolution and the target frame rate, and to obtain the amount to be allocated;

[0032] The allocation module is used to allocate the number of codec resources to be allocated to the media stream to be processed from the currently idle codec resources, provided that the number of currently idle codec resources is not less than the number to be allocated.

[0033] The generation module is used to generate a prompt message indicating insufficient codec resources when the number of currently available codec resources is less than the number to be allocated.

[0034] In one possible implementation, the device further includes:

[0035] The first processing module is configured to not execute the pending processing operation when the number of currently idle codec resources is less than the number to be allocated; receive the pending media stream and perform a specified operation on the pending media stream using processor resources, wherein the specified operation is an operation that does not require codec resources.

[0036] In one possible implementation, the device further includes:

[0037] The second processing module is configured to not execute the operation to be processed if the operation to be processed does not require the consumption of encoding / decoding resources or does not support the processing of the operation to be processed; receive the media stream to be processed and perform a specified operation on the media stream to be processed using processor resources, wherein the specified operation is an operation that does not require encoding / decoding resources.

[0038] In one possible implementation, the device further includes:

[0039] The quantization module is used to quantize the processing capabilities of all codec resources during the initialization process to obtain the total number of all codec resources.

[0040] In one possible implementation, the quantization module is specifically used for:

[0041] During initialization, based on the model of the encoding / decoding resource hardware, the number of reference media streams that can be processed simultaneously under the use of a preset reference encoding or decoding protocol is determined, thus obtaining the total number of all encoding / decoding resources. The resolution of the reference media stream is a preset reference resolution, and the frame rate of the reference media stream is a preset reference frame rate.

[0042] In one possible implementation, the second determining module includes:

[0043] The first determining submodule is used to determine the weight of the target protocol relative to the preset benchmark encoding or decoding protocol according to the predetermined first correspondence relationship to obtain the first weight value. The first correspondence relationship includes the weight of each type of encoding or decoding protocol relative to the preset benchmark encoding or decoding protocol. The preset benchmark encoding or decoding protocol is the encoding or decoding protocol used to determine the total number of all encoding and decoding resources.

[0044] The second determining submodule is used to determine the weights of the target resolution and the target frame rate relative to the reference media stream, and to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used to determine the total number of all codec resources;

[0045] The calculation submodule is used to calculate the amount of encoding and decoding resources required by the operation to be processed, based on the first weight and the second weight, to obtain the amount to be allocated.

[0046] In one possible implementation, the second determining submodule is specifically used for:

[0047] Calculate the product of the target resolution and the target frame rate to obtain the first product;

[0048] Calculate the product of the preset baseline resolution and the preset baseline frame rate to obtain the second product;

[0049] Calculate the ratio of the first product to the second product to obtain the second weight.

[0050] This application embodiment also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0051] Memory, used to store computer programs;

[0052] When a processor executes a program stored in memory, it implements any of the above-described encoding / decoding resource allocation methods.

[0053] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described encoding / decoding resource allocation methods.

[0054] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the above-described encoding / decoding resource allocation methods.

[0055] Beneficial effects of the embodiments in this application:

[0056] The encoding / decoding resource allocation method, apparatus, and electronic device provided in this application first obtain the operation to be processed for the media stream to be processed, then determine the target protocol, target resolution, and target frame rate corresponding to the operation to be processed, and determine the amount of encoding / decoding resources required by the operation to be processed based on the target protocol, target resolution, and target frame rate to obtain the amount to be allocated. If the amount of currently idle encoding / decoding resources is not less than the amount to be allocated, the amount of encoding / decoding resources to be allocated is allocated to the media stream to be processed from the currently idle encoding / decoding resources. If the amount of currently idle encoding / decoding resources is less than the amount to be allocated, a prompt message indicating insufficient encoding / decoding resources is generated. By combining the bitstream type, resolution, and frame rate to determine the encoding / decoding resources required for the operation to be processed on the bitstream, resource calculation errors can be reduced, and reasonable control and allocation of encoding / decoding resources can be achieved.

[0057] Furthermore, if the number of currently available codec resources is less than the number to be allocated, the pending processing operation is not executed. Instead, the pending media stream is received, and processor resources are used to perform operations on the pending media stream that do not require codec resources. Therefore, even if operations requiring codec resources cannot be performed due to insufficient codec resources in the current system, other operations that do not require decoding resources cannot proceed normally, thus ensuring a smoother codec-related operation process.

[0058] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0060] Figure 1 This is a schematic diagram of a first flowchart of the encoding / decoding resource allocation method provided in an embodiment of this application;

[0061] Figure 2 This is a second flowchart illustrating the encoding / decoding resource allocation method provided in the embodiments of this application;

[0062] Figure 3 This is a schematic diagram of a third method for allocating encoding and decoding resources provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of the fourth flowchart of the encoding / decoding resource allocation method provided in the embodiments of this application;

[0064] Figure 5a A schematic diagram of the fifth method for allocating encoding and decoding resources provided in the embodiments of this application;

[0065] Figure 5b This is a first schematic diagram illustrating the insufficient codec resource prompt information in the codec resource allocation method provided in the embodiments of this application;

[0066] Figure 5c This is a second schematic diagram illustrating the insufficient codec resource prompt information in the codec resource allocation method provided in the embodiments of this application;

[0067] Figure 6 This is a schematic diagram of a codec resource allocation device according to an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0070] In order to achieve reasonable control and allocation of encoding and decoding resources, embodiments of this application provide a method, apparatus, electronic device, storage medium, and computer program product containing instructions for encoding and decoding resource allocation.

[0071] The encoding / decoding resource allocation method provided in this application embodiment will be described below. This method is applied to electronic devices; specifically, the electronic device can be any type of electronic device that can provide media stream encoding / decoding resource allocation services, such as mobile phones, tablets, personal computers, servers, etc. The encoding / decoding resource allocation method provided in this application embodiment can be implemented in at least one of software, hardware circuits, and logic circuits within the electronic device.

[0072] like Figure 1 As shown, Figure 1 This is a first flowchart illustrating a codec resource allocation method provided in an embodiment of this application. The method includes:

[0073] S110, Obtain the pending operations for the media stream to be processed;

[0074] S120, determine the target protocol, target resolution, and target frame rate corresponding to the above-mentioned operation to be processed, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0075] S130, based on the above target protocol, the above target resolution and the above target frame rate, determine the amount of encoding and decoding resources required for the above-mentioned operation to be processed, and obtain the amount to be allocated;

[0076] S140, if the number of currently available codec resources is not less than the number to be allocated, allocate the number of codec resources to be allocated for the media stream to be processed from the currently available codec resources.

[0077] S150, if the number of currently available codec resources is less than the number to be allocated mentioned above, generate a prompt message indicating insufficient codec resources.

[0078] Media streams can be either video or audio. Because some operations related to media streams, such as parsing, storage, and forwarding, only require bitstream parsing, storage, and forwarding, and do not require operations that require encoding and decoding resources, such as previewing, when encoding and decoding resource-related operations are enabled, it is first determined whether the operation to be processed in the media stream actually requires the consumption of encoding and decoding resources. If it does require the consumption of encoding and decoding resources, the amount of encoding and decoding resources required needs to be calculated based on the target protocol, target resolution, and target frame rate. Then, it is determined whether the system has enough decoding resources available. If so, encoding and decoding resources are allocated. Furthermore, after allocating encoding and decoding resources, the currently available encoding and decoding resources need to be updated. If decoding resources are insufficient, a prompt message indicating insufficient encoding and decoding resources is generated.

[0079] By combining the bitstream type, resolution, and frame rate to determine the encoding and decoding resources required for operations on the bitstream to be processed, resource calculation errors can be reduced, and reasonable control and allocation of encoding and decoding resources can be achieved.

[0080] In one possible implementation, the above method further includes:

[0081] If the number of currently available codec resources is less than the number to be allocated, the above-mentioned pending operation is not performed; the above-mentioned pending media stream is received, and the specified operation is performed on the above-mentioned pending media stream using processor resources, wherein the specified operation is an operation that does not require codec resources.

[0082] When the number of currently available codec resources is less than the number to be allocated, it means that there are not enough codec resources available. Since operations such as parsing, storing, and forwarding media streams do not require codec resources, no encoding or decoding operations are performed at this time. Instead, the processor receives the media stream to be processed and uses the processor resources to perform operations on the media stream that do not require codec resources, such as parsing, storing, and forwarding. In this way, even when codec resources are insufficient, the processor resources can still be used reasonably to maintain the parsing, storage, and forwarding of the bitstream.

[0083] In one possible implementation, the above method further includes:

[0084] If the above-mentioned pending operation does not require the consumption of encoding / decoding resources, or does not support the processing of the above-mentioned pending operation, the above-mentioned pending operation is not executed; the above-mentioned pending media stream is received, and a specified operation is performed on the above-mentioned pending media stream using processor resources, wherein the specified operation is an operation that does not require encoding / decoding resources.

[0085] If the operation to be processed does not consume encoding / decoding resources or the electronic device does not support processing the operation, then the operation is not executed. Instead, the processor directly receives the media stream and uses its resources to perform operations on the media stream that do not require encoding / decoding resources, such as parsing, storing, and forwarding. Alternatively, if the electronic device cannot support processing the operation, for example, if it only supports encoding / decoding operations for H.264 and H.265 protocols, and the operation to be processed involves encoding / decoding operations for protocols other than H.264 and H.265, then the electronic device does not support processing that operation. After receiving the media stream, the electronic device uses its processor resources to perform operations on the media stream that do not require encoding / decoding resources. In this way, even when the operation to be processed does not consume encoding / decoding resources or the electronic device does not support processing the operation, processor resources can still be used efficiently to maintain the parsing, storage, and forwarding of the bitstream.

[0086] In one possible implementation, the above method further includes:

[0087] During initialization, the processing capabilities of all encoding and decoding resources are quantified to obtain the total number of all encoding and decoding resources.

[0088] After initial startup, the processing capabilities of all codec resources need to be quantified to obtain the total number of all codec resources. When quantifying the processing capabilities of all codec resources, a baseline media stream can be set first, and the processing capabilities of all codec resources can be quantified based on this baseline media stream. For clarity and layout, another embodiment will be used as an example for detailed explanation below.

[0089] In one possible implementation, the initialization process described above quantifies the processing capabilities of all codec resources to obtain the total number of all codec resources, including:

[0090] During initialization, based on the model of the encoding / decoding resource hardware, the number of reference media streams that can be processed simultaneously under the use of a preset reference encoding or decoding protocol is determined, thus obtaining the total number of all encoding / decoding resources. The resolution of the aforementioned reference media stream is a preset reference resolution, and the frame rate of the aforementioned reference media stream is a preset reference frame rate.

[0091] During initialization, a baseline media stream is first set. For example, a common media stream, H.265 1080P@30FPS, is used as the baseline media stream. That is, the resources required to decode one H.265 1080P@30FPS stream are set to KP (K Proportionality). Based on this baseline media stream, the number of baseline media streams that the hardware can simultaneously process is determined to obtain the total number of all codec resources. For example, the total codec capability of the device is x×KP. For instance, if a device's VCP has a total decoding capability of 16 H.265 1080P@30FPS video streams, then the total codec capability of the device is 16×KP, meaning that this chip can simultaneously decode 16 1080P@30FPS media streams. The setting of the baseline media stream can be based on actual conditions and is not limited here.

[0092] based on Figure 1 See the embodiments. Figure 2 , Figure 2 This is a second flowchart illustrating the encoding / decoding resource allocation method provided in this application embodiment. Step S130 described above can be implemented through the following steps:

[0093] S131, based on the target protocol and the preset reference encoding or decoding protocol, determine the weight of the target protocol relative to the preset reference encoding or decoding protocol to obtain a first weight value, wherein the preset reference encoding or decoding protocol is the encoding or decoding protocol used to determine the total number of all encoding and decoding resources;

[0094] S132, determine the weights of the target resolution and the target frame rate relative to the reference media stream to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used to determine the total number of all codec resources.

[0095] S133, based on the first weight and the second weight, calculate the amount of encoding and decoding resources required for the operation to be processed to obtain the amount to be allocated.

[0096] After acquiring the media stream to be processed, if processing operations are to be performed on the media stream, the amount of encoding and decoding resources required for the processing operations depends on the target protocol, camera resolution, and frame rate.

[0097] After setting a preset baseline encoding or decoding protocol, the weight of the target protocol relative to the preset baseline encoding or decoding protocol can be determined. For example, if a common media stream H.265 1080P@30FPS decoding media stream is used as the baseline media stream, that is, if the H.265 protocol is set as the baseline decoding protocol, then for the H.265 protocol, the weight of the target decoding protocol in the H.265 decoding scenario is equivalent to 1. Since the encoding performance consumption is higher than that of decoding, H.265 encoding can be set with different weights based on the actual measured performance consumption.

[0098] For H.264 encoding and decoding, because H.264 consumes more resources than H.265, and encoding consumes more resources than decoding, the weights corresponding to H.264 encoding or decoding can be obtained through performance testing and theoretical calculation.

[0099] For characteristic bitstream processing, such as SVC (Scalable Video Coding) bitstreams, since SVC bitstreams are mainly composed of temporal and spatial scalars, their main function is to compress the media stream. In this application, SVC bitstream generally refers to temporal scalars, meaning that some frames in the media stream do not need to be decoded, reducing decoding performance consumption. That is, for video frames that are mainly encoded and decoded, the reduction will not change the original data stream. The first weight corresponding to the target protocol is determined based on the number of video frames after SVC scalarization. For example, if the number of video frames after SVC scalarization is 1 / 2 of the original, the first weight is 0.5; if the number of video frames is 1 / 4 of the original, the first weight is 0.25.

[0100] The second weight corresponding to the media stream to be processed is determined based on the resolution of the reference media stream, the frame rate of the reference media stream, the target resolution, and the target frame rate. The calculation method for the second weight corresponding to the media stream to be processed can be: dividing the product of the resolution and frame rate of the media stream to be processed by the product of the resolution and frame rate of the reference media stream.

[0101] In one possible implementation, step S132 can be achieved through the following steps:

[0102] Calculate the product of the target resolution and the target frame rate to obtain the first product;

[0103] Calculate the product of the above-mentioned preset baseline resolution and the above-mentioned preset baseline frame rate to obtain the second product;

[0104] Calculate the ratio of the first product to the second product to obtain the second weight.

[0105] Similarly, taking the common H.265 1080P@30FPS decoding media stream as the benchmark media stream, and setting the resource occupied by decoding one H.265 1080P@30FPS stream as KP, then for the H.265 decoding scenario, the weight of the media stream to be processed with CIF 10w pixels, a resolution of 352×288, and a frame rate of 30 is 352×288×30 / (1920×1080×30)≈0.05.

[0106] The number of encoding and decoding resources required for the operation to be processed is calculated based on the first weight and the second weight to obtain the number to be allocated. Specifically, the number of encoding and decoding resources required for the operation to be processed is the product of the first weight and the second weight of the number to be allocated.

[0107] For example, taking a common H.265 1080P@30FPS media stream as the baseline media stream, and setting the resource usage for decoding one H.265 1080P@30FPS stream as KP, consider the typical H.265 decoding scenario:

[0108] Table 1 shows a typical camera decoding weight table:

[0109] Table 1

[0110]

[0111] For other H.265 decoding scenarios:

[0112] For H.265 non-typical resolution and frame rate videos, taking 1080P as the baseline, the product of the resolution and frame rate of the media stream to be processed is linearly calculated by dividing by the product of the resolution and frame rate of the baseline media stream and then rounding down. For example, for a 1000×1000@25FPS video, the second weight corresponding to the media stream to be processed is 1000×1000×25 / (1920×1080×30)≈0.4. Therefore, the resources occupied by decoding a 1000×1000@25FPS video are 0.4KP.

[0113] For H.264 encoding and decoding:

[0114] If the H.264 encoding or decoding conforms to the resolution and frame rate information in Table 1, the second weight in the table can be directly multiplied by the first weight corresponding to the H.264 encoding or decoding. If it does not conform to the resolution and frame rate information in the table, the product of the resolution and frame rate of the media stream to be processed is divided by the product of the resolution and frame rate of the reference media stream to obtain the second weight, and then the second weight is multiplied by the first weight corresponding to the H.264 encoding or decoding.

[0115] For example, if the second weight for H.264 / JPG is k1, and m is the first weight for the H.264 encoding protocol, then the resources occupied by H.264 / JPG decoding are KP2 = (m × K1)KP. The resources occupied by H.264 / JPG encoding are KP3 = n × KP2, where n is the ratio coefficient between encoding and decoding.

[0116] This application addresses the characteristics of hardware units by unifying the management of encoding and decoding resources, rationally planning encoding and decoding resources, and designing weight values ​​based on various bitstream types, resolutions, and operating scenarios. This reduces resource calculation errors and enables reasonable control and allocation of encoding and decoding resources.

[0117] based on Figure 1 In one possible implementation, step S130 described above can be achieved through the following steps:

[0118] According to a predetermined first correspondence, the weight of the target protocol relative to the preset benchmark encoding or decoding protocol is determined to obtain a first weight. The first correspondence includes the weight of each type of encoding or decoding protocol relative to the preset benchmark encoding or decoding protocol. The preset benchmark encoding or decoding protocol is the encoding or decoding protocol used to determine the total number of all encoding and decoding resources.

[0119] The weights of the target resolution and the target frame rate relative to the reference media stream are determined to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used to determine the total number of all codec resources.

[0120] Based on the first weight and the second weight mentioned above, the amount of encoding and decoding resources required for the above-mentioned operation to be processed is calculated to obtain the amount to be allocated.

[0121] The first correspondence between the weights of various types of encoding or decoding protocols and the preset benchmark encoding or decoding protocol can be stored in advance. In this way, the weight of the target protocol relative to the preset benchmark encoding or decoding protocol can be determined directly according to the predetermined first correspondence, thereby reducing the amount of computation.

[0122] Specifically, the first correspondence can be stored as a first weight table. This allows for the direct determination of the target protocol's weight relative to a preset baseline encoding or decoding protocol, thus obtaining the first weight. Specifically, the first weight table records the weights of each type of encoding or decoding protocol relative to the preset baseline encoding or decoding protocol. After obtaining the target protocol, the target protocol type is matched against the corresponding encoding or decoding protocol type in the first weight table to determine the first weight corresponding to the target protocol.

[0123] Alternatively, the weights of the target resolution and target frame rate relative to the baseline media stream can be determined according to a pre-determined second correspondence, thus obtaining the second weights. Specifically, the second correspondence can be stored as a second weight table, allowing the second weights of the target resolution and target frame rate relative to the baseline media stream to be determined directly. Specifically, the second weight table records the weights of each resolution and frame rate relative to the baseline media stream. After obtaining the target resolution and target frame rate, they are matched with the corresponding resolutions and frame rates in the second weight table to determine the second weights of the target resolution and target frame rate relative to the baseline media stream.

[0124] based on Figure 1 See the embodiments. Figure 3 , Figure 3 This is a third flowchart illustrating the encoding / decoding resource allocation method provided in this application embodiment. Step S130 described above can be implemented through the following steps:

[0125] S1310, according to the predetermined first weight table, determine the weight of the target protocol relative to the preset benchmark encoding or decoding protocol to obtain the first weight;

[0126] S1320, Determine whether there is a second weight value for the target resolution and target frame rate relative to the reference media stream in the pre-determined second weight table;

[0127] S1330, if it exists, then determine the second weight of the target resolution and target frame rate relative to the reference media stream based on the second weight table;

[0128] S1340, if not present, then divide the product of the target media stream resolution and the target frame rate by the product of the base media stream resolution and the base media stream frame rate to obtain the second weight.

[0129] The first weight table and the second weight table can be different tables or the same table. No restriction is imposed here.

[0130] If the second weight table contains second weights for the target resolution and target frame rate relative to the baseline media stream, these second weights can be directly obtained from the table. If the media stream to be processed is an unconventional stream, the second weights can be obtained through linear calculation. This allows media streams with typical resolutions and bitrates to directly obtain their corresponding weights from the table, while unconventional resolutions and bitrates are determined through linear conversion and other calculations, thereby reducing resource calculation errors. This enables the development of reasonable resource management strategies.

[0131] In one possible implementation, the amount of codec resources required for the operation to be processed is determined based on the target protocol, the target resolution, and the target frame rate, to obtain the amount to be allocated, including:

[0132] Determine whether the pre-determined codec resource table contains the amount of codec resources required by the pending operation;

[0133] If it exists, the amount of encoding and decoding resources required for the operation to be processed is determined based on the pre-determined encoding and decoding resource table.

[0134] If not, the amount of encoding / decoding resources required for the operation to be processed is determined according to the method described in any of the above embodiments.

[0135] If the media stream to be processed is a regular media stream, the amount of codec resources required for its processing can be directly obtained from a pre-defined codec resource table. If the media stream to be processed is an unconventional media stream, it can be determined through linear calculation, thereby reducing resource calculation errors. This allows for the development of reasonable resource management strategies.

[0136] Based on the above embodiments, this application provides a method for allocating encoding and decoding resources, such as... Figure 4 As shown, Figure 4 This is a fourth flowchart illustrating the encoding / decoding resource allocation method provided in this application embodiment. Taking decoding as an example, the method includes:

[0137] S401, Obtain the decoding task for the media stream to be processed;

[0138] S402, determine whether the preset situation is included; the preset situation includes special cases such as resolution not supported and virtual decoding. Virtual decoding means that no decoding operation is actually performed and no decoding resources are consumed; if yes, then proceed to step S408, otherwise proceed to step S403.

[0139] S403, determine the amount of encoding and decoding resources required by the decoding task, and obtain the amount to be allocated;

[0140] S404, determine whether the number of currently available codec resources is not less than the number to be allocated; if yes, proceed to step S405, otherwise proceed to step S408.

[0141] S405, allocates the number of codec resources to be allocated for the media stream to be processed;

[0142] S406, Update the number of currently available codec resources;

[0143] S407, Decode;

[0144] S408 does not require operations on encoding / decoding resources;

[0145] Even if operations requiring encoding / decoding resources cannot be performed due to insufficient encoding / decoding resources in the current system, the above method will not cause other operations that do not require decoding resources to proceed normally, thus ensuring that the encoding / decoding related operation process proceeds smoothly.

[0146] In one possible implementation, when an instruction is received to stop performing a preset operation on the media stream to be processed, where the preset operation is an operation requiring encoding / decoding resources, the method includes:

[0147] Determine whether encoding / decoding resources have been allocated for the aforementioned media streams to be processed;

[0148] Without allocating codec resources, the processor resources are used to perform a specified operation on the media stream to be processed, wherein the specified operation is an operation that does not require codec resources;

[0149] If codec resources have been allocated, release the allocated codec resources and update the currently available codec resources.

[0150] Specifically, such as Figure 5a As shown, Figure 5a This is a fifth flowchart illustrating the encoding / decoding resource allocation method provided in the embodiments of this application. The method includes:

[0151] S501, received an instruction to stop performing decoding operations on the media stream to be processed;

[0152] S502, determine whether encoding / decoding resources are occupied; if yes, proceed to step S503; if no, proceed to step S505.

[0153] S503 releases the occupied encoding and decoding resources;

[0154] S504, Update currently available codec resources;

[0155] The S505 utilizes processor resources to perform specified operations on the media stream to be processed.

[0156] When it is necessary to close encoding or decoding resource operations, first determine whether the current channel is occupying encoding or decoding resources. If it is, release the occupied encoding or decoding resources; otherwise, close the channel directly.

[0157] In one possible implementation, after generating a message indicating insufficient codec resources when the number of currently available codec resources is less than the number to be allocated, the method includes:

[0158] A message indicating insufficient codec resources is displayed in the preview window corresponding to the media stream to be processed.

[0159] The NVR connects to the digital camera via a network, receives audio and video data from the camera, and parses, stores, forwards, and previews the data. During previewing, the NVR's preview screen size depends on the number of split screens configured in the NVR. If the number of split screens is configured to 4, then there will be 4 preview windows arranged in a 2x2 pattern. Additionally, it also depends on the preview channels configured in the NVR. For example, if preview window 1 is configured with channel 3, then preview window 1 will display the media stream from channel 3.

[0160] For example, there are currently four preview windows: window 1 corresponds to the media stream playing channel 1, window 2 corresponds to the media stream playing channel 2, window 3 corresponds to the media stream playing channel 3, and window 4 corresponds to the media stream playing channel 4. When the electronic device is encoding and decoding the media stream corresponding to channel 3, if the number of currently available encoding and decoding resources is less than the number to be allocated, it means that the current encoding and decoding resources cannot meet the requirements for encoding and decoding the media stream corresponding to channel 3. In this case, a message indicating insufficient encoding and decoding resources will be displayed in the preview window corresponding to channel 3. This message can be any information that can indicate insufficient encoding and decoding resources, such as: the message can directly indicate insufficient decoding performance; furthermore, it can also include information reminding users to modify parameters, specifically as follows: Figure 5b As shown.

[0161] Of course, if none of the channels can be opened, a system resource shortage window can be displayed directly on the NVR interface, as shown below. Figure 5c As shown. The specific format of the insufficient codec resource prompt and how to display the insufficient codec resource prompt can be set according to actual needs, and is not limited here.

[0162] Based on the above method embodiments, this application provides a codec resource allocation apparatus, such as... Figure 6 As shown, Figure 6 A schematic diagram of a codec resource allocation device provided in an embodiment of this application is shown. The device may include:

[0163] The acquisition module 610 is used to acquire the pending operations for the media stream to be processed;

[0164] The first determining module 620 is used to determine the target protocol, target resolution and target frame rate corresponding to the above-mentioned operation to be processed, wherein the target protocol is a target encoding protocol or a target decoding protocol;

[0165] The second determining module 630 is used to determine the amount of encoding and decoding resources required for the operation to be processed based on the target protocol, the target resolution and the target frame rate, and to obtain the amount to be allocated.

[0166] The allocation module 640 is used to allocate the number of codec resources to be allocated to the media stream to be processed from the currently idle codec resources, provided that the number of currently idle codec resources is not less than the number to be allocated.

[0167] The generation module 650 is used to generate a prompt message indicating insufficient codec resources when the number of currently available codec resources is less than the number to be allocated mentioned above.

[0168] In one possible implementation, the above-described apparatus further includes:

[0169] The first processing module is configured to not perform the above-mentioned pending processing operation when the number of currently available codec resources is less than the number to be allocated; receive the above-mentioned pending media stream and perform a specified operation on the above-mentioned pending media stream using processor resources, wherein the specified operation is an operation that does not require codec resources.

[0170] In one possible implementation, the above-described apparatus further includes:

[0171] The second processing module is configured to not execute the above-mentioned pending operation if it does not require the consumption of encoding / decoding resources or does not support the above-mentioned pending operation; receive the above-mentioned pending media stream and perform a specified operation on the above-mentioned pending media stream using processor resources, wherein the specified operation is an operation that does not require encoding / decoding resources.

[0172] In one possible implementation, the above-described apparatus further includes:

[0173] The quantization module is used to quantize the processing capabilities of all codec resources during the initialization process to obtain the total number of all codec resources.

[0174] In one possible implementation, the quantization module described above is specifically used for:

[0175] During initialization, based on the model of the encoding / decoding resource hardware, the number of reference media streams that can be processed simultaneously under the use of a preset reference encoding or decoding protocol is determined, thus obtaining the total number of all encoding / decoding resources. The resolution of the aforementioned reference media stream is a preset reference resolution, and the frame rate of the aforementioned reference media stream is a preset reference frame rate.

[0176] In one possible implementation, the second determining module includes:

[0177] The first determining submodule is used to determine the weight of the target protocol relative to the preset benchmark encoding or decoding protocol according to the predetermined first correspondence relationship to obtain the first weight value. The first correspondence relationship includes the weight of each type of encoding or decoding protocol relative to the preset benchmark encoding or decoding protocol. The preset benchmark encoding or decoding protocol is the encoding or decoding protocol used to determine the total number of all encoding and decoding resources.

[0178] The second determining submodule is used to determine the weights of the target resolution and the target frame rate relative to the reference media stream to obtain a second weight, wherein the resolution of the reference media stream is a preset reference resolution, the frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is the media stream used to determine the total number of all codec resources.

[0179] The calculation submodule is used to calculate the amount of encoding and decoding resources required for the above-mentioned operation to be processed, based on the first weight and the second weight, to obtain the amount to be allocated.

[0180] In one possible implementation, the second determining submodule is specifically used for:

[0181] Calculate the product of the target resolution and the target frame rate to obtain the first product;

[0182] Calculate the product of the above-mentioned preset baseline resolution and the above-mentioned preset baseline frame rate to obtain the second product;

[0183] Calculate the ratio of the first product to the second product to obtain the second weight.

[0184] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0185] This application also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0186] Memory 703 is used to store computer programs;

[0187] When processor 701 executes a program stored in memory 703, it performs the following steps:

[0188] Retrieve pending operations for the media stream to be processed;

[0189] Determine the target protocol, target resolution, and target frame rate corresponding to the operation to be processed;

[0190] Based on the target protocol, the target resolution, and the target frame rate, determine the amount of encoding and decoding resources required for the operation to be processed, and obtain the amount to be allocated;

[0191] If the number of currently available codec resources is not less than the number to be allocated, allocate the number of codec resources to be allocated for the media stream to be processed from the currently available codec resources;

[0192] If the number of currently available codec resources is less than the number to be allocated, a message indicating insufficient codec resources is generated.

[0193] Optionally, when the processor 701 executes the program stored in the memory 703, it can also implement any of the above-mentioned encoding / decoding resource allocation methods.

[0194] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0195] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0196] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0197] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0198] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described encoding / decoding resource allocation methods.

[0199] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the encoding / decoding resource allocation methods described above.

[0200] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0201] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0202] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and computer-readable storage media are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0203] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method of coding resource allocation, characterized by, The method comprises: obtaining a to-be-processed operation for a to-be-processed media stream; determining a target protocol, a target resolution and a target frame rate corresponding to the to-be-processed operation, wherein the target protocol is a target encoding protocol or a target decoding protocol; determining a weight of the target protocol relative to a preset reference encoding or decoding protocol according to a predetermined first correspondence relationship, to obtain a first weight, wherein the first correspondence relationship comprises weights of each type of encoding or decoding protocol relative to the preset reference encoding or decoding protocol, and the preset reference encoding or decoding protocol is an encoding or decoding protocol used when determining a total number of all codec resources; determining weights of the target resolution and the target frame rate relative to a reference media stream, to obtain a second weight, wherein a resolution of the reference media stream is a preset reference resolution, a frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is a media stream used when determining the total number of all codec resources; calculating a number of codec resources required by the to-be-processed operation according to the first weight and the second weight, to obtain a to-be-allocated number; in a case where a number of currently idle codec resources is not less than the to-be-allocated number, allocating the to-be-allocated number of codec resources to the to-be-processed media stream from the currently idle codec resources; in a case where the number of currently idle codec resources is less than the to-be-allocated number, generating prompt information indicating that codec resources are insufficient.

2. The method of claim 1, wherein, The method further comprises: in a case where the number of currently idle codec resources is less than the to-be-allocated number, not performing the to-be-processed operation; receiving the to-be-processed media stream and performing a specified operation on the to-be-processed media stream using a processor resource, wherein the specified operation is an operation that does not require codec resources.

3. The method of claim 1, wherein, The method further comprises: in a case where the to-be-processed operation does not require consumption of codec resources or does not support processing of the to-be-processed operation, not performing the to-be-processed operation; receiving the to-be-processed media stream and performing a specified operation on the to-be-processed media stream using a processor resource, wherein the specified operation is an operation that does not require codec resources.

4. The method of claim 1, wherein, The method further comprises: in an initialization process, quantifying processing capabilities of all codec resources to obtain a total number of all codec resources.

5. The method of claim 4, wherein, The quantifying, in the initialization process, of the processing capabilities of all codec resources to obtain the total number of all codec resources comprises: in the initialization process, determining, according to a model of codec resource hardware, a number of reference media streams that all codec resources are allowed to process simultaneously in a case where the preset reference encoding or decoding protocol is used, to obtain the total number of all codec resources, wherein a resolution of the reference media stream is a preset reference resolution and a frame rate of the reference media stream is a preset reference frame rate.

6. The method of claim 1, wherein, The determining of the weights of the target resolution and the target frame rate relative to the reference media stream to obtain the second weight comprises: calculating a product of the target resolution and the target frame rate to obtain a first product; calculating a product of the preset reference resolution and the preset reference frame rate to obtain a second product; calculating a ratio of the first product and the second product to obtain a second weight.

7. An apparatus for coding resource allocation, the apparatus comprising: The apparatus comprises: an acquisition module, configured to acquire a to-be-processed operation for a to-be-processed media stream; a first determination module, configured to determine a target protocol, a target resolution, and a target frame rate corresponding to the to-be-processed operation, wherein the target protocol is a target encoding protocol or a target decoding protocol; a second determination module, comprising: a first determination submodule, a second determination submodule, and a calculation submodule; the first determination submodule is configured to determine a first weight of the target protocol relative to a preset reference encoding or decoding protocol according to a predetermined first correspondence relationship, wherein the first correspondence relationship comprises weights of each type of encoding or decoding protocol relative to the preset reference encoding or decoding protocol, and the preset reference encoding or decoding protocol is an encoding or decoding protocol used when determining a total quantity of all encoding and decoding resources; the second determination submodule is configured to determine a second weight of the target resolution and the target frame rate relative to a reference media stream, wherein a resolution of the reference media stream is a preset reference resolution, a frame rate of the reference media stream is a preset reference frame rate, and the reference media stream is a media stream used when determining the total quantity of all encoding and decoding resources; and the calculation submodule is configured to calculate a quantity of encoding and decoding resources consumed by the to-be-processed operation according to the first weight and the second weight to obtain a to-be-allocated quantity. an allocation module, configured to, in a case where a quantity of currently idle encoding and decoding resources is not less than the to-be-allocated quantity, allocate the to-be-allocated quantity of encoding and decoding resources to the to-be-processed media stream from the currently idle encoding and decoding resources. a generation module, configured to, in a case where the quantity of currently idle encoding and decoding resources is less than the to-be-allocated quantity, generate prompt information indicating that encoding and decoding resources are insufficient.

8. The apparatus of claim 7, wherein, The apparatus further comprises: a first processing module, configured to, in the case where the quantity of currently idle encoding and decoding resources is less than the to-be-allocated quantity, not perform the to-be-processed operation, receive the to-be-processed media stream, and perform a specified operation on the to-be-processed media stream using a processor resource, wherein the specified operation is an operation that does not require encoding and decoding resources.

9. The apparatus of claim 7, wherein, The apparatus further comprises: a second processing module, configured to, in a case where the to-be-processed operation does not consume encoding and decoding resources or does not support processing the to-be-processed operation, not perform the to-be-processed operation, receive the to-be-processed media stream, and perform a specified operation on the to-be-processed media stream using a processor resource, wherein the specified operation is an operation that does not require encoding and decoding resources.

10. The apparatus of claim 7, wherein, The apparatus further comprises: a quantization module, configured to, in an initialization process, quantize processing capabilities of all encoding and decoding resources to obtain a total quantity of all encoding and decoding resources.

11. The apparatus of claim 10, wherein, The quantization module is specifically configured to: In the initialization process, according to the type of codec resource hardware, the total number of the codec resources is determined, and the number of reference media streams allowed to be processed simultaneously under the preset reference encoding or decoding protocol is obtained, wherein the resolution of the reference media stream is a preset reference resolution, and the frame rate of the reference media stream is a preset reference frame rate.

12. The apparatus of claim 7, wherein, The second determining sub-module is specifically configured to: calculate the product of the target resolution and the target frame rate to obtain a first product; calculate the product of the preset reference resolution and the preset reference frame rate to obtain a second product; calculate the ratio of the first product to the second product to obtain a second weight.

13. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used for storing a computer program; The processor is used for executing the program stored on the memory to realize the method steps of any one of claims 1-6.

Images