Image coding control method, device, equipment and computer readable storage medium

By calculating and calculating the network delay during the image encoding process in real time, and judging and discarding image frames that have not started encoding, the problem of excessive image encoding delay is solved, and image transmission efficiency and CPU resource utilization are improved.

CN112087635BActive Publication Date: 2024-11-19XIAN WANXIANG ELECTRONICS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010771938.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-04
Publication Date
2024-11-19
Estimated Expiration
2040-08-04

AI Technical Summary

Technical Problem

During the image encoding process, the image encoding delay is long, resulting in waste of CPU resources and increased network pressure.

Method used

By counting the network delay of the preset number of frame images in real time, calculate the average value and determine whether it is greater than the preset threshold. If it is greater, the number of frames dropped is determined according to the preset rule and the continuous images that have not started encoding are discarded.

Benefits of technology

It shortens the image encoding delay, alleviates network pressure, and reduces the encoding load of the CPU.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112087635B_ABST
    Figure CN112087635B_ABST
Patent Text Reader

Abstract

The present application provides an image coding control method, device, equipment and computer-readable storage medium, which relates to the field of image processing technology and can solve the problem of long image coding delay. The specific technical solution is: counting the network delay of each frame of a preset number of frames; calculating the average network delay of a preset number of frames according to the network delay of each frame; judging whether the average network delay is greater than a preset threshold; when the average network delay is greater than the preset threshold, determining the number of lost frames M according to preset rules; discarding the continuous M frames of images that are not currently encoded. The ability to determine the number of lost frames before encoding and discard a corresponding number of images before encoding can relieve network pressure, shorten image encoding delay, and reduce the encoding load of the CPU.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image coding control method, apparatus, device and computer-readable storage medium. Background Art

[0002] At present, in the process of encoding and decoding graphics images, images are usually collected at a fixed frame rate, and after the collected images are encoded, they are stored in the cache waiting for transmission. During the transmission process, it is determined whether the encoded images in the cache need to be discarded based on the network delay.

[0003] For all images, since it is determined whether to discard them after encoding is completed, the image encoding delay is increased and the encoding resources of the CPU (Central Processing Unit) are wasted. Summary of the invention

[0004] The embodiments of the present application provide an image coding control method, apparatus, device and computer-readable storage medium, which can solve the problem of long image coding delay. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present application, there is provided an image encoding control method, the method comprising:

[0006] Count the network delay of each frame of a preset number of frames;

[0007] Calculate the average value of the network delay of the preset number of frames of image according to the network delay of each frame of image;

[0008] Determine whether the average value of the network delay is greater than a preset threshold;

[0009] When the average value of the network delay is greater than the preset threshold, determining the number of lost frames M according to a preset rule, where M is an integer;

[0010] The continuous M frames of images that have not yet started to be encoded are discarded.

[0011] In one embodiment, determining the number of lost frames M according to a preset rule includes:

[0012] The number of lost frames M is determined according to the difference between the average value of the network delay and the preset threshold.

[0013] In one embodiment, determining the number of lost frames M according to the difference between the average value of the network delay and the preset threshold value includes:

[0014] The number of lost frames M is determined according to a first formula, wherein the first formula is: M=[(T 平均 -A) / (t采集 +t 编码 )];

[0015] Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0016] In one embodiment, determining the number of lost frames M according to the difference between the average value of the network delay and the preset threshold value includes:

[0017] The number of lost frames M is determined according to a second formula, where the second formula is:

[0018] M=[(T 平均 -A) / (t 采集 +t 编码 )]+1;

[0019] Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0020] In one embodiment, the method further comprises:

[0021] detecting a setting operation for setting a preset quantity;

[0022] The preset number is determined according to the setting operation.

[0023] The image encoding control method provided by the embodiment of the present application counts the network delay of each frame of a preset number of frames in real time during the image transmission process and calculates the average value of the network delay of the preset number of frames. When the average value is greater than a preset threshold, the number of lost frames M is determined according to a preset rule, and the continuous M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding and the corresponding number of images can be discarded before encoding, thereby alleviating network pressure, shortening image encoding delay, and reducing the encoding load of the CPU.

[0024] According to a second aspect of an embodiment of the present application, there is provided an image encoding control device, including:

[0025] A statistics module, used to count the network delay of each frame of a preset number of frames;

[0026] A calculation module, used for calculating an average value of the network delay of the preset number of frames of images according to the network delay of each frame of image;

[0027] A first determination module, configured to determine whether the average value of the network delay is greater than a preset threshold value, and when the average value of the network delay is greater than the preset threshold value, determine the number of lost frames M according to a preset rule, where M is an integer;

[0028] The frame discarding module is used to discard the continuous M frames of images that have not yet started to be encoded.

[0029] In one embodiment, the first determination module is specifically configured to determine the number of lost frames M according to a difference between an average value of the network delay and the preset threshold.

[0030] In one embodiment, the first determining module is specifically used to determine the number of lost frames M according to a first formula, and the first formula is: M=[(T 平均 -A) / (t 采集 +t 编码 )];

[0031] Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0032] In one embodiment, the first determination module is specifically used to determine the number of lost frames M according to a second formula, and the second formula is:

[0033] M=[(T 平均 -A) / (t 采集 +t 编码 )]+1;

[0034] Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0035] In one embodiment, the apparatus further comprises:

[0036] A detection module, used for detecting a setting operation for setting a preset quantity;

[0037] The second determining module is used to determine the preset number according to the setting operation.

[0038] The image encoding control device provided by the embodiment of the present application uses a statistical module to count the network delay of each frame of a preset number of frames of images, and a calculation module calculates the average value of the network delay of the preset number of frames of images. When the first determination module determines that the average value is greater than a preset threshold, it determines the number of lost frames M according to a preset rule, and the frame discarding module discards the continuous M frames of images that are not currently encoded. The number of lost frames can be determined before encoding and the corresponding number of images can be discarded before encoding, thereby alleviating network pressure, shortening image encoding delay, and reducing the encoding load of the CPU.

[0039] According to a third aspect of an embodiment of the present application, an image coding control device is provided, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the image coding control method as described above.

[0040] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein at least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the steps performed in the image encoding control method as described above.

[0041] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] Figure 1 is a flow chart of an image encoding control method provided by an embodiment of the present application;

[0044] Figure 2 is a flowchart of a specific image encoding control method provided by an embodiment of the present application;

[0045] Figure 3 is a structural schematic diagram of an image encoding control device provided in an embodiment of the present application;

[0046] Figure 4 is a structural schematic diagram of another image encoding control device provided in an embodiment of the present application;

[0047] Figure 5 It is a structural schematic diagram of an image encoding control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] The present application embodiment provides an image encoding control method, such as Figure 1 As shown, the image encoding control method may include the following steps:

[0050] Step 101: Count the network delays of a preset number of image frames.

[0051] During the image transmission process, the network delay of each frame of the preset number of frames of images that have recently completed transmission is counted in real time. Specifically, the network delay of a frame of image is equal to the difference between the timestamp when the receiving end receives the image and the timestamp when the image is sent by the sending end.

[0052] Step 102: Calculate the average value of the network delay of a preset number of image frames.

[0053] After the network delay of each of the preset number of frames of images that have recently completed transmission is obtained, the average network delay of the preset number of frames of images is calculated based on the network delay of each frame of images obtained by statistics. For example, if the preset number is N, the network delay of each of the N frames of images is continuously counted, and then the average network delay of the N frames of images is calculated.

[0054] Step 103: Determine whether the average value is greater than a preset threshold.

[0055] After calculating the average value of the network delay of a preset number of frame images, it is determined whether the average value of the network delay is greater than a preset threshold. If so, step 104 is executed, otherwise, the process goes to step 101 to continue statistics.

[0056] Step 104: Determine the number of lost frames M according to a preset rule.

[0057] When the average value of the network delay of a preset number of frames of images is greater than a preset threshold, the number of lost frames M is determined according to a preset rule, where M is an integer. For example, the number of lost frames M can be determined according to the difference between the average value of the network delay and the preset threshold. In one embodiment, the number of lost frames M can be obtained by using the difference and the ratio of the average time required to capture one frame of image to the sum of the average time required to encode one frame of image.

[0058] Step 105: discard the M consecutive image frames that have not yet started to be encoded.

[0059] After determining the number of lost frames M, the continuous M frames of images that have not yet started to be encoded are directly discarded, and then the process goes to step 101 to continue counting the network delay. In this step, the images that have already started to be transmitted or encoded will not be discarded, and subsequent processing will still be performed normally. Only the continuous M frames of images that have not yet started to be encoded are discarded.

[0060] The image encoding control method provided by the embodiment of the present application counts the network delay of each frame of a preset number of frames in real time during the image transmission process and calculates the average value of the network delay of the preset number of frames. When the average value is greater than a preset threshold, the number of lost frames M is determined according to a preset rule, and the continuous M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding and the corresponding number of images can be discarded before encoding, thereby alleviating network pressure, shortening image encoding delay, and reducing the encoding load of the CPU.

[0061] Based on the above Figure 1 Corresponding to the image coding control method provided in the embodiment, another embodiment of the present application provides a specific image coding control method, which can be applied to a video image coding terminal device, for example, the video image coding terminal device can be a terminal device or a network server, etc. Figure 2 As shown, the image encoding control method provided in this embodiment may include the following steps:

[0062] Step 201: Count the network delays of a preset number of image frames.

[0063] During the image transmission process, the average network delay of each frame of the preset number of frames that have been transmitted recently is counted in real time. The preset number can be set according to actual needs, for example, it can be set to 10 frames, 20 frames, 30 frames, etc. The network delay of a frame of image refers to the difference between the timestamp of the image received by the receiving end and the timestamp of the image sent by the sending end.

[0064] In one embodiment, a preset number setting function may be provided, which may be implemented by the following method: detecting a setting operation for setting a preset number; and determining the preset number according to the setting operation. For example, a setting interface for setting a preset number may be provided to the user, and the user may enter the preset number according to the interface prompt, such as setting it to 10 frames, 20 frames, 30 frames, etc.; or, the preset number may be set by using a preset number shortcut key.

[0065] Step 202: Calculate the average value of the network delay of a preset number of image frames.

[0066] After obtaining the network delay of each of the preset number of frames of images that have recently completed transmission, the average network delay of the preset number of frames of images is calculated based on the network delay of each frame of image. For example, if the preset number is N, the network delay of each of the N frames of images is continuously counted, and then the average network delay of the N frames of images is calculated.

[0067] Specifically, the average network delay T of N consecutive frames of images can be calculated by the following formula: 平均 :

[0068]

[0069] Among them, ▽t x is the network delay of the x-th frame image, x is an integer greater than 0 and less than or equal to N, Among them, Rt x Indicates the timestamp of the x-th frame image transmitted to the receiving end, St x Indicates the timestamp when the x-th frame image is sent from the sender.

[0070] Step 203: Determine whether the average value is greater than a preset threshold.

[0071] It is determined whether the average value of the network delay of the preset number of frame images obtained by statistics is greater than a preset threshold value. If it is, step 204 is executed; otherwise, the process goes to step 201 to continue statistics.

[0072] In this step, the calculated average value of the network delay of the preset number of frame images is compared with the preset threshold to determine whether the average value is greater than the preset threshold. The preset threshold is a preset value that can be set based on experience, such as 100ms, 90ms, 80ms, etc., or other values. The higher the preset threshold, the higher the tolerance for delay. Therefore, the setting of the preset threshold can depend on the usage scenario.

[0073] In one embodiment, a preset threshold setting function may be provided, which may be implemented by the following method: detecting a setting operation for setting the preset threshold; and determining the preset threshold according to the setting operation. For example, a setting interface for setting the preset threshold may be provided to the user, and the user may input the preset threshold according to the interface prompt, such as setting it to 100ms, 90ms, 80ms, etc.; or, the preset threshold may be set by using a shortcut key for the preset threshold.

[0074] Step 204: Determine the number of lost frames M according to the difference between the average value of the network delay and a preset threshold.

[0075] Assume that the average network delay obtained by statistics is T 平均, then the number of lost frames M can be calculated by the following first formula or the following second formula.

[0076] The first formula is: M = [(T 平均 -A) / (t 采集 +t 编码 )];

[0077] The second formula is: M = [(T 平均 -A) / (t 采集 +t 编码 )]+1;

[0078] In the above formula, T 平均 represents the average value of network delay, A represents the preset threshold, t 采集 is the average time required to collect one frame of image, t 编码 is the average time required to encode a frame of image, t 采集 and t 编码 All can be set according to experience. The symbol [] indicates the rounding symbol, that is, [(T 平均 -A) / (t 采集 +t 编码 )] is equal to (T 平均 -A) / (t 采集 +t 编码 ) is the integer part of the result.

[0079] Step 205: discard the M consecutive image frames that have not yet started to be encoded.

[0080] After determining the number of lost frames M, the continuous M frames of images that are not currently encoded are directly discarded, and then the process goes to step 101 to continue counting the network delay.

[0081] In this step, the continuous M frames of images that have not yet started to be encoded are directly discarded. At the same time, the images that have already started to be transmitted or encoded will not be discarded and will still be processed normally. The discarded images are the continuous M frames of images that have not yet started to be encoded.

[0082] The image encoding control method provided by the embodiment of the present application counts the network delay of each frame of a preset number of frames in real time during the image transmission process and calculates the average value of the network delay of the preset number of frames. When the average value is greater than the preset threshold, the number of lost frames M is determined according to the difference between the average value of the network delay and the preset threshold, and then the continuous M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding and the corresponding number of images can be discarded before encoding, thereby alleviating network pressure, shortening image encoding delay, and reducing the encoding load of the CPU.

[0083] Based on the above Figure 1 and Figure 2 The image encoding control method described in the corresponding embodiment is as follows: an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application.

[0084] The present application embodiment provides an image encoding control device, such as Figure 3 As shown, the image coding control device 30 includes: a statistical module 301, a calculation module 302, a first determination module 303 and a frame discarding module 304.

[0085] The statistical module 301 is used to count the network delay of each frame of a preset number of frames of images;

[0086] The calculation module 302 is used to calculate the average value of the network delay of a preset number of frames of images according to the network delay of each frame of image obtained by the statistical module 301;

[0087] The first determination module 303 is used to determine whether the average value of the network delay calculated by the calculation module 302 is greater than a preset threshold value. When the average value of the network delay is greater than the preset threshold value, the number of lost frames M is determined according to a preset rule, where M is an integer;

[0088] The frame discarding module 304 is used to discard the continuous M frames of images that have not yet started to be encoded.

[0089] In one embodiment, the first determination module 303 is specifically configured to determine the number of lost frames M according to a difference between an average value of the network delay and the preset threshold.

[0090] In one embodiment, the first determination module 303 is specifically configured to determine the number of lost frames M according to a first formula, where M=[(T 平均 -A) / (t 采集 +t 编码 )];

[0091] Among them, T 平均 is the average value of network delay, A is the preset threshold, t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0092] In one embodiment, the first determination module 303 is specifically configured to determine the number of lost frames M according to a second formula, where the second formula is:

[0093] M=[(T 平均 -A) / (t 采集 +t 编码 )]+1;

[0094] Among them, T 平均is the average value of network delay, A is the preset threshold, t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

[0095] In one embodiment, Figure 4 As shown, the image coding control device further includes a detection module 305 and a second determination module 306.

[0096] The detection module 305 is used to detect a setting operation for setting a preset quantity;

[0097] The second determining module 306 is used to determine the preset number according to the setting operation detected by the detecting module 305 .

[0098] The image encoding control device provided by the embodiment of the present application uses a statistical module to count the network delay of each frame of a preset number of frames in real time during image transmission, and a calculation module calculates the average value of the network delay of the preset number of frames. When the first determination module determines that the average value is greater than a preset threshold, it determines the number of lost frames M according to a preset rule, and the frame discarding module discards the continuous M frames of images that are not currently encoded. The number of lost frames can be determined before encoding and the corresponding number of images can be discarded before encoding, thereby alleviating network pressure, shortening image encoding delay, and reducing the encoding load of the CPU.

[0099] refer to Figure 5 As shown, the embodiment of the present application further provides an image coding control device 50, the image coding control device 50 includes a memory 501 and a processor 502, the memory 501 stores at least one computer instruction, and the processor 502 is used to load and execute at least one computer instruction to implement the above Figure 1 or Figure 2 The image encoding control method described in the corresponding embodiment.

[0100] Based on the above Figure 1 and Figure 2 In accordance with the image coding control method described in the embodiment, the present application also provides a computer-readable storage medium. For example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, or an optical data storage device. The storage medium stores computer instructions for executing the above Figure 1 and Figure 2 The image encoding control method described in the corresponding embodiment will not be repeated here.

[0101] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The description and embodiments are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

[0102] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for controlling image coding, It is characterized in that The method comprises: Count the network delay of each frame of a preset number of frames; Calculate the average value of the network delay of the preset number of frames of image according to the network delay of each frame of image; Determine whether the average value of the network delay is greater than a preset threshold; When the average value of the network delay is greater than the preset threshold, determining the number of lost frames M according to the difference between the average value of the network delay and the preset threshold, where M is an integer; Discard the M consecutive frames of images that have not yet started to be encoded; The determining the number of lost frames M according to the difference between the average value of the network delay and the preset threshold value comprises: The number of lost frames M is determined according to a first formula, wherein the first formula is: M=[(T 平均 -A) / (t 采集 +t 编码 )]; Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image; Alternatively, the number of lost frames M is determined according to a second formula, where the second formula is: M=[(T 平均 -A) / (t 采集 +t 编码 )]+1; Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

2. The method according to claim 1, It is characterized in that The method further comprises: detecting a setting operation for setting a preset quantity; The preset number is determined according to the setting operation.

3. An image coding control device, It is characterized in that include: A statistics module, used to count the network delay of each frame of a preset number of frames; A calculation module, used for calculating an average value of the network delay of the preset number of frames of images according to the network delay of each frame of image; a first determination module, configured to determine whether the average value of the network delay is greater than a preset threshold value, and when the average value of the network delay is greater than the preset threshold value, determine the number of lost frames M according to the difference between the average value of the network delay and the preset threshold value, where M is an integer; A frame discarding module is used to discard the M consecutive frames of images that have not yet started to be encoded; The first determination module is specifically used to determine the number of lost frames M according to a first formula, wherein the first formula is: M=[(T 平均 -A) / (t 采集 +t 编码 )]; Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image; Alternatively, the number of lost frames M is determined according to a second formula, where the second formula is: M=[(T 平均 -A) / (t 采集 +t 编码 )]+1; Among them, the T 平均 is the average value of network delay, A is the preset threshold, and t 采集 is the average time required to collect one frame of image, t 编码 The average time required to encode a frame of image.

4. The device according to claim 3, It is characterized in that The device also includes: A detection module, used for detecting a setting operation for setting a preset quantity; The second determining module is used to determine the preset number according to the setting operation.

5. An image encoding control device, It is characterized in that The image coding control device comprises a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the image coding control method according to claim 1 or 2.

6. A computer-readable storage medium, It is characterized in that The storage medium stores at least one computer instruction, which is loaded and executed by the processor to implement the steps performed in the image encoding control method according to claim 1 or 2.