Video frame encoding method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a video frame encoding method, device, electronic device and computer-readable medium. A specific implementation of the method includes: obtaining a video frame; encoding and segmenting the video frame to obtain a set of encoding units; filtering the set of encoding units to generate a first threshold and a second threshold; based on the first threshold and the second threshold, performing multi-level region division processing on the video frame to generate multi-level regions, wherein the multi-level regions include an area of ​​interest group, a possible area group and an area of ​​no interest group; encoding the area of ​​interest group, the possible area group and the area of ​​no interest group included in the multi-level region respectively to obtain multiple encoded video frames. This implementation solves the problem of region division of video frames, reduces the probability of missed detection of the overall target object, and improves encoding efficiency.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a video frame encoding method, device, electronic device, and computer-readable medium. Background Art

[0002] Image coding refers to the technology of representing an image or the information contained in an image with fewer bits. Commonly used image coding technologies often involve first reducing the information redundancy of video frames that have undergone high-precision analog-to-digital conversion. Then, within a certain loss range, the video frames with reduced information redundancy are re-encoded.

[0003] However, when the above method is used to encode video frames, the following technical problems often occur:

[0004] First, it is impossible to accurately divide the video frame into regions, resulting in missing information in the video frame when encoding each region included in the video frame;

[0005] Second, it is impossible to accurately divide the video frames into regions, which results in the inability to accurately classify and encode regions of different levels, resulting in low encoding efficiency of the video frames. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a video frame encoding method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a video frame encoding method, the method comprising: acquiring a video frame; encoding and segmenting the video frame to obtain a set of encoding units; filtering the set of encoding units to generate a first threshold and a second threshold; based on the first threshold and the second threshold, performing multi-level region division processing on the video frame to generate multi-level regions, wherein the multi-level regions include an area of ​​interest group, a possible area group, and a non-interest area group; encoding the area of ​​interest group, the possible area group, and the non-interest area group included in the above-mentioned multi-level regions respectively to obtain multiple encoded video frames.

[0009] In a second aspect, some embodiments of the present disclosure provide a video frame encoding method, the method comprising: an acquisition unit, acquiring a video frame; a coding segmentation unit, configured to perform coding segmentation on the video frame to obtain a set of coding units; a filtering processing unit, configured to perform filtering processing on the set of coding units to generate a first threshold and a second threshold; a division processing unit, configured to perform multi-level region division processing on the video frame based on the first threshold and the second threshold to generate multi-level regions, wherein the multi-level regions include an area of ​​interest group, a possible area group and a non-interest area group; and an encoding unit, configured to perform encoding processing on the area of ​​interest group, the possible area group and the non-interest area group included in the above-mentioned multi-level regions, respectively, to obtain multiple encoded video frames.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method as described in any implementation manner in the first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the multiple coded video frames obtained by the video frame coding method of some embodiments of the present disclosure improve the accuracy of region division, thereby reducing the overall probability of misjudgment and missed detection of target objects, and the video frame coding efficiency is improved. Specifically, the reason for the missed detection of the relevant target objects is that the traditional method cannot accurately divide the region, resulting in the inability to completely detect the target object. Based on this, in the multiple coded video frames obtained by the video frame coding method of some embodiments of the present disclosure. First, obtain a video frame. Secondly, encode and segment the above-mentioned video frame to obtain a set of coding units. By encoding and segmenting the above-mentioned video frame to generate a set of coding units, the video frame can be processed in blocks, which can improve the processing efficiency of the video frame. Then, the above-mentioned set of coding units is filtered to generate a first threshold and a second threshold. By filtering the above-mentioned set of coding units, the threshold of the coding unit significance is determined, so as to prepare for the next step of region division. Afterwards, based on the above-mentioned first threshold and the above-mentioned second threshold, the above-mentioned video frame can be subjected to multi-level region division processing to generate a multi-level region. By comparing the coding unit significance with the first threshold and the second threshold, the video frame is accurately divided into the multi-level regions, so that the importance of different regions of the video frame can be distinguished. Finally, the region of interest group, possible region group and non-interest region group included in the multi-level region are encoded respectively to obtain a multi-encoded video frame. By encoding the multi-level region, a multi-encoded video frame is obtained, which can improve the coding efficiency of the video. The multi-encoded video frame includes not only the region of interest group and the non-interest region group, but also the possible region group. Because of the participation of the possible region group, the target object (such as a vehicle) in the video frame can be effectively detected, reducing the probability of missed detection of the target object. Thereby improving the accuracy of target object detection and the coding efficiency of the video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0014] Figure 1 is a schematic diagram of an application scenario of a video frame encoding method of some embodiments of the present disclosure;

[0015] Figure 2 is a flowchart of some embodiments of the video frame encoding method according to the present disclosure;

[0016] Figure 3 is a schematic structural diagram of some embodiments of a video frame encoding device according to the present disclosure;

[0017] Figure 4 It is a structural schematic diagram of an electronic device according to the video frame encoding method disclosed in the present invention. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 It is a schematic diagram of an application scenario of the video frame encoding method of some embodiments of the present disclosure.

[0025] exist Figure 1In the application scenario, first, the computing device 101 can obtain a video frame 102. Then, the computing device 101 can encode and segment the above video frame 102 to obtain a coding unit set 103. After that, the computing device 101 can filter the above coding unit set 103 to generate a first threshold 104 and a second threshold 105. Then, the computing device 101 can perform a multi-level region division process on the above video frame 102 based on the above first threshold 104 and second threshold 105 to generate a multi-level region 106, wherein the multi-level region 106 includes an area of ​​interest group, a possible area group, and a non-interest area group. Finally, the computing device 101 can encode the area of ​​interest group, the possible area group, and the non-interest area group included in the multi-level area 106 respectively to obtain a multi-encoded video frame 107.

[0026] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

[0027] It should be understood that Figure 1 The number of computing devices in the embodiment is only illustrative. Any number of computing devices may be provided according to implementation requirements.

[0028] Continue to refer Figure 2 , shows a process 200 of some embodiments of the video frame encoding method according to the present disclosure. The video frame encoding method comprises the following steps:

[0029] Step 201, obtaining a video frame.

[0030] In some embodiments, the execution subject of the video frame encoding method (eg Figure 1 The computing device 101 shown in the figure can obtain the above video frame through a wired connection mode or a wireless connection mode. The above video frame can be any picture in the target video. For example, the above video frame can be any picture in a movie video. The above video frame can also be any picture in a TV video.

[0031] Step 202: Encode and segment the video frame to obtain a set of encoding units.

[0032] In some embodiments, the execution subject may perform coding segmentation on the video frame to obtain a set of coding units, which may include the following steps:

[0033] The first step is to encode the video frame to obtain a video information sequence, wherein the video information sequence may be an information sequence consisting of 0s and 1s.

[0034] The second step is to evenly divide the video information sequence into a predetermined number of sub-video information sequences to generate the coding unit set.

[0035] As an example, the predetermined number may be 4.

[0036] Step 203: filter the coding unit set to generate a first threshold and a second threshold.

[0037] In some embodiments, the execution subject of the video frame encoding method may perform filtering processing on the above coding unit set to generate a first threshold and a second threshold, wherein the above first threshold is a region critical value used to divide the non-interest region group and the possible region group, and the above second threshold is a region critical value used to divide the possible region group and the interesting region group. The above filtering processing may be a motion smoothing filtering processing. The above filtering processing may also be a limiting filtering processing.

[0038] In some optional implementations of some embodiments, processing the encoding unit to generate the first threshold and the second threshold may include the following steps:

[0039] In the first step, each coding unit in the above coding unit set is subjected to discrete cosine transform processing to generate a frequency component distribution matrix, thereby obtaining a frequency component distribution matrix group. The discrete cosine transform processing is to transform the coding unit signal into a frequency component including a DC component and a cosine component. The frequency component is used to characterize the magnitude (amplitude) of the frequency vibration of the DC component and the cosine component.

[0040] As an example, the distribution matrix corresponding to the above frequency components may be:

[0041]

[0042] Among them, F(0,0) is used to represent the frequency component value at the (0,0) position.

[0043] The second step is to filter each frequency component distribution matrix in the above frequency component distribution matrix group to generate a filtered frequency component distribution matrix as a filtered frequency component distribution matrix to obtain a filtered frequency component distribution matrix group. Wherein, each frequency component distribution matrix is ​​filtered. When the normalized frequency value corresponding to the frequency component distribution matrix corresponding to the above frequency component is within [0.03, 0.25], the above frequency component passes the filtering process. When the normalized frequency value corresponding to the frequency component distribution matrix corresponding to the above frequency component is not within [0.03, 0.25], the above frequency component does not pass the filtering process.

[0044] As an example, the above filtering frequency component distribution matrix group may be:

[0045]

[0046] Among them, F(0,0) is used to represent the frequency component value at the (0,0) position.

[0047] In the third step, each element included in each filter frequency component distribution matrix in the filter frequency component distribution matrix group is accumulated to generate a filter frequency component significance as a coding unit significance, thereby obtaining a coding unit significance group, wherein the coding unit significance indicates the ability to distinguish between coding units.

[0048] The fourth step is to generate a first threshold and a second threshold based on the coding unit significance group.

[0049] As an example, the execution entity may generate the first threshold and the second threshold in various ways based on the coding unit significance group.

[0050] Optionally, based on the above coding unit significance group, generating the first threshold and the second threshold may further include the following sub-steps:

[0051] In the first sub-step, a plane rectangular coordinate system is constructed with the coding unit significance group as the horizontal axis and the number of the coding unit sets as the vertical axis.

[0052] The second sub-step is to divide the horizontal axis into intervals based on the above plane rectangular coordinate system using a preset interval length to obtain a predetermined number of intervals.

[0053] The third sub-step is to sequentially scan the significance of each coding unit in the above coding unit set to obtain a coding unit significance group.

[0054] The fourth sub-step is to construct a coding unit histogram on the predetermined number of intervals based on the coding unit significance group.

[0055] The fifth sub-step is to scan the coding unit histogram to generate a first threshold and a second threshold, including:

[0056] The first and second thresholds are generated by the following formula:

[0057]

[0058] Among them, TH1 represents the first threshold. TH2 represents the second threshold. X0 represents the starting coordinate of the horizontal axis of the coding unit histogram. N(i) represents the number of coding units in the i-th interval of the above coding unit histogram corresponding to the significance of any coding unit in the above coding unit significance group, and the value range of i is [1, M], M represents the predetermined number of intervals; m represents the interval number corresponding to the maximum value of N(i). k represents the interval interval of the above predetermined number of intervals; l represents the interval interval of the above predetermined number of intervals; m+k represents scanning from the m-th interval to the direction of increasing i, and after k intervals, find the first interval where N(i) is less than δ1N(m) as the first threshold interval, and continue to scan the values ​​of N(m+k+1) and N(m+k+2) are both less than δ1N(m), and the interval number corresponding to the above first threshold interval is m+k. m+k+l means scanning from the m+kth interval to the direction of increasing i. After l intervals, find the first interval where N(i) is less than δ2N(m) as the second threshold interval, and continue to scan until the values ​​of N(m+k+l+1) and N(m+k+l+2) are both less than δ2N(m). The interval number corresponding to the second threshold interval is m+k+l. δ1 represents the first threshold predetermined coefficient, which is 0.8. δ2 represents the second threshold predetermined coefficient, which is 0.2. Δx represents the length of the predetermined number of intervals divided by the horizontal axis of the above coding unit histogram.

[0059] As an inventive point of the embodiment of the present disclosure, the above formula solves the second technical problem mentioned in the background technology, "it is impossible to accurately divide the video frame into regions, resulting in the inability to accurately classify and encode regions of different levels, resulting in low encoding efficiency of the video frame". The factors that lead to inaccurate threshold setting are often as follows: the relationship between the saliency of each coding unit is not considered, the threshold of the coding unit cannot be accurately set, and then the region cannot be accurately divided. If the above factors are solved, accurate regional division of the video frame can be achieved. In order to achieve this effect, first, the above formula scans the saliency of each coding unit in the above coding unit set in turn, and obtains the saliency interval corresponding to the maximum number of coding units, and the number of coding units corresponding to the above saliency interval is the maximum value. Afterwards, continue scanning in the direction of increasing saliency, find a saliency interval that meets the first formula condition, and the adjacent and continuous next two intervals also meet the first formula condition. The above first threshold can be obtained, and the non-interest region is divided to obtain a non-interest region group, so that the non-interest region group does not contain useful information, which is conducive to more accurate detection of the target object in the subsequent process. Scan again in the direction of increasing saliency to find a saliency interval that satisfies the second formula condition, and the next two adjacent and continuous intervals also satisfy the second formula condition. The second threshold can be obtained, and the region of interest is divided to obtain a region of interest group. Two thresholds are obtained on the coordinate axis, so that the video frame is divided into three non-intersecting regions, and the video frame is divided into regions more accurately. Therefore, while ensuring that the probability of missing detection of the overall target object is reduced, the coding efficiency is improved.

[0060] Step 204 : Based on the first threshold and the second threshold, perform multi-level region division processing on the video frame to generate multi-level regions.

[0061] In some embodiments, based on the first threshold and the second threshold, the video frame is subjected to a multi-level region division process to generate a multi-level region. The multi-level region may include an area of ​​interest group, a possible area group and a non-interest area group. The multi-level region division may be an image processing technology, the purpose of which is to divide at least one area of ​​an object from a video frame, that is, to find a set of pixels corresponding to an object or an object surface, which appear as a two-dimensional mass, which is one of the basic shape characteristics of the region. The multi-level region division process may be performed by comparing the coding unit saliency with a threshold, thereby obtaining a multi-level region including an area of ​​interest group, a possible area group and a non-interest area group. The area of ​​interest is an area where the target object is likely to appear. The possible area is an area where the target object is likely to appear. The non-interest area is an area where there is no possible target object.

[0062] In some optional implementations of some embodiments, the execution subject performs multi-level region division processing on the video frame based on the first threshold and the second threshold to generate the multi-level region, which may include the following steps:

[0063] The first step is to determine the coding unit significance that is less than the first threshold in the coding unit significance group as the first coding unit significance, thereby obtaining the first coding unit significance group;

[0064] Step 2: determining the region corresponding to each first coding unit saliency in the first coding unit saliency group as a non-interest region to obtain a non-interest region group;

[0065] Step 3: Determine the coding unit significance of the coding unit significance group that is greater than or equal to the first threshold and less than the second threshold as the second coding unit significance, thereby obtaining the second coding unit significance group;

[0066] Step 4: determine the region corresponding to each second coding unit significance in the second coding unit significance group as a possible region, and obtain a possible region group;

[0067] Step 5: determining the coding unit significance that is greater than or equal to the second threshold in the coding unit significance group as the third coding unit significance, thereby obtaining a third coding unit significance group;

[0068] In the sixth step, the region corresponding to each third coding unit significance in the third coding unit significance group is determined as a region of interest, so as to obtain a region of interest group.

[0069] Step 205 , encoding is performed on the region of interest group, the possible region group and the region of no interest group included in the multi-level region to obtain multiple encoded video frames.

[0070] In some embodiments, the execution subject performs encoding processing on the interest region group, the possible region group and the non-interest region group included in the multi-level region respectively to obtain multiple encoded video frames.

[0071] In some optional implementations of some embodiments, the execution subject performs encoding processing on the interest region group, the possible region group, and the non-interest region group included in the multi-level region to obtain multiple encoded video frames, which may include the following steps:

[0072] In the first step, each region of interest in the region of interest group is subjected to a first quantization parameter encoding process to obtain a first-level encoded video frame, wherein the first quantization parameter encoding process may be a Huffman encoding process.

[0073] The second step is to perform a second quantization parameter encoding process on each possible region in the possible region group to obtain a secondary encoded video frame. The second quantization parameter encoding process may be a Huffman encoding process.

[0074] The third step is to perform a third quantization parameter encoding process on each non-interest region in the non-interest region group to obtain a three-level encoded video frame. The first quantization parameter encoding process may be a Huffman encoding process.

[0075] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the multiple coded video frames obtained by the video frame coding method of some embodiments of the present disclosure improve the accuracy of region division, thereby reducing the overall probability of misjudgment and missed detection of target objects, and the video frame coding efficiency is improved. Specifically, the reason for the missed detection of the relevant target objects is that the traditional method cannot accurately divide the region, resulting in the inability to completely detect the target object. Based on this, in the multiple coded video frames obtained by the video frame coding method of some embodiments of the present disclosure. First, obtain a video frame. Secondly, encode and segment the above-mentioned video frame to obtain a set of coding units. By encoding and segmenting the above-mentioned video frame to generate a set of coding units, the video frame can be processed in blocks, which can improve the processing efficiency of the video frame. Then, the above-mentioned set of coding units is filtered to generate a first threshold and a second threshold. By filtering the above-mentioned set of coding units, the threshold of the coding unit significance is determined, so as to prepare for the next step of region division. Afterwards, based on the above-mentioned first threshold and the above-mentioned second threshold, the above-mentioned video frame can be subjected to multi-level region division processing to generate a multi-level region. By comparing the coding unit significance with the first threshold and the second threshold, the video frame is accurately divided into the multi-level regions, so that the importance of different regions of the video frame can be distinguished. Finally, the region of interest group, possible region group and non-interest region group included in the multi-level region are encoded respectively to obtain a multi-encoded video frame. By encoding the multi-level region, a multi-encoded video frame is obtained, which can improve the coding efficiency of the video. The multi-encoded video frame includes not only the region of interest group and the non-interest region group, but also the possible region group. Because of the participation of the possible region group, the target object (such as a vehicle) in the video frame can be effectively detected, reducing the probability of missed detection of the target object. Thereby improving the accuracy of target object detection and the coding efficiency of the video frame.

[0076] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a video frame encoding device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0077] like Figure 3 As shown, the apparatus 300 for video frame encoding in some embodiments includes: an acquisition unit 301, an encoding segmentation unit 302, a filtering unit 303, a division unit 304, and an encoding unit 305. The acquisition unit 301 is configured to acquire a video frame; the encoding segmentation unit 302 is configured to perform encoding segmentation on the video frame to obtain a set of encoding units; the filtering unit 303 is configured to perform filtering processing on the set of encoding units to generate a first threshold and a second threshold; the division unit 304 is configured to perform multi-level region division processing on the video frame based on the first threshold and the second threshold to generate multi-level regions, wherein the multi-level regions include an area of ​​interest group, a possible area group, and a non-interest area group; the encoding unit 305 is configured to perform encoding processing on the area of ​​interest group, the possible area group, and the non-interest area group included in the multi-level regions, respectively, to obtain multiple encoded video frames.

[0078] It is understood that the units described in the device 300 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 300 and the units contained therein, and will not be described in detail here.

[0079] Reference below Figure 4 , which shows an electronic device suitable for implementing some embodiments of the present disclosure (e.g., Figure 1 Schematic diagram of the structure of the computing device 101)400. Figure 4 The server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0080] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 404 is also connected to the bus 404.

[0081] Typically, the following devices may be connected to the I / O interface 404: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication devices 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0082] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0083] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0084] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0085] The computer-readable medium may be included in the device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a video frame; performs coding segmentation on the video frame to obtain a set of coding units; performs filtering processing on the set of coding units to generate a first threshold and a second threshold; performs multi-level region division processing on the video frame based on the first threshold and the second threshold to generate multi-level regions, wherein the multi-level regions include an area of ​​interest group, a possible area group, and an area of ​​non-interest group; and performs coding processing on the area of ​​interest group, the possible area group, and the area of ​​non-interest group included in the multi-level region, respectively, to obtain multiple coded video frames.

[0086] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0087] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, an encoding segmentation unit, a filtering processing unit, a partitioning processing unit, and an encoding unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring video frames in a video".

[0089] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0090] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A video frame encoding method, comprising: Get video frames; Encoding and segmenting the video frame to obtain a set of encoding units; Performing filtering on the coding unit set to generate a first threshold and a second threshold; Based on the first threshold and the second threshold, performing a multi-level region division process on the video frame to generate a multi-level region, wherein the multi-level region includes an area of ​​interest group, a possible area group, and a non-interest area group; Performing encoding processing on the interest region group, possible region group and non-interest region group included in the multi-level region respectively to obtain multiple encoded video frames; The filtering process on the coding unit set to generate a first threshold and a second threshold comprises: Performing discrete cosine transform processing on each coding unit in the coding unit set to generate a frequency component distribution matrix, thereby obtaining a frequency component distribution matrix group; Performing filtering processing on each frequency component distribution matrix in the frequency component distribution matrix group to generate a filtered frequency component distribution matrix as a filtered frequency component distribution matrix, thereby obtaining a filtered frequency component distribution matrix group; Performing accumulation processing on each element included in each filter frequency component distribution matrix in the filter frequency component distribution matrix group to generate filter frequency component significance as coding unit significance, and obtaining a coding unit significance group; Based on the coding unit significance group, a first threshold and a second threshold are generated.

2. The method according to claim 1, wherein: The generating the first threshold and the second threshold comprises: Constructing a plane rectangular coordinate system with the coding unit significance group as the horizontal axis and the number of the coding unit sets as the vertical axis; Based on the plane rectangular coordinate system, the horizontal axis is divided into intervals using a preset interval length to obtain a predetermined number of intervals; Scanning each coding unit significance in the coding unit set in sequence to obtain a coding unit significance group; Based on the coding unit significance group, constructing a coding unit histogram over the predetermined number of intervals; The coding unit histogram is scanned to generate a first threshold and a second threshold.

3. The method according to claim 2, wherein: The scanning process of the coding unit histogram to generate a first threshold and a second threshold comprises: The first and second thresholds are generated by the following formula: Among them, TH1 represents the first threshold; TH2 represents the second threshold; X0 represents the starting coordinate of the horizontal axis of the coding unit histogram; N(i) represents the number of coding units in the i-th interval of the coding unit histogram corresponding to any coding unit significance in the coding unit significance group, the value range of i is [1, M], M represents a predetermined number of intervals; m represents the interval number corresponding to the maximum value of N(i); k represents the interval interval of the predetermined number of intervals; l represents the interval interval of the predetermined number of intervals; m+k represents scanning in the direction of increasing from the m-th interval to i, and after k intervals, find the first interval where N(i) is less than δ1N(m) as the first threshold interval, and continue to scan N The values ​​of N(m+k+1) and N(m+k+2) are both less than δ1N(m), and the interval number corresponding to the first threshold interval is m+k; m+k+l means scanning from the m+kth interval in the direction of increasing i, and after l intervals, find the first interval in which N(i) is less than δ2N(m) as the second threshold interval, and continue scanning until the values ​​of N(m+k+l+1) and N(m+k+l+2) are both less than δ2N(m), and the interval number corresponding to the second threshold interval is m+k+l; δ1 represents the first threshold predetermined coefficient, which is 0.8; δ2 represents the second threshold predetermined coefficient, which is 0.2; Δx represents the length of the predetermined number of intervals divided by the horizontal axis of the histogram of the coding unit.

4. The method according to claim 3, wherein: The performing multi-level region division processing on the video frame based on the first threshold and the second threshold to generate multi-level regions includes: Determine the coding unit significance that is less than the first threshold in the coding unit significance group as the first coding unit significance, to obtain a first coding unit significance group; Determine a region corresponding to each first coding unit significance in the first coding unit significance group as a non-interest region, to obtain a non-interest region group; Determine the coding unit significance of the coding unit significance group that is greater than or equal to the first threshold and less than the second threshold as the second coding unit significance, to obtain a second coding unit significance group; Determine a region corresponding to each second coding unit significance in the second coding unit significance group as a possible region, to obtain a possible region group; Determine the coding unit significance that is greater than or equal to the second threshold in the coding unit significance group as a third coding unit significance, to obtain a third coding unit significance group; The region corresponding to each third coding unit significance in the third coding unit significance group is determined as a region of interest to obtain a region of interest group.

5. The method according to claim 4, wherein: The encoding process is performed on the interest region group, possible region group and non-interest region group included in the multi-level region to obtain multiple encoded video frames, including: Performing a first quantization parameter encoding process on each region of interest in the region of interest group to obtain a primary encoded video frame; Performing a second quantization parameter encoding process on each possible region in the possible region group to obtain a secondary encoded video frame; A third quantization parameter encoding process is performed on each non-interest region in the non-interest region group to obtain a three-level encoded video frame.

6. A video frame encoding device, comprising: An acquisition unit, configured to acquire a video frame; A coding segmentation unit, configured to perform coding segmentation on the video frame to obtain a set of coding units; A filtering processing unit, configured to perform filtering processing on the set of coding units to generate a first threshold and a second threshold; a division processing unit configured to perform a multi-level region division process on the video frame based on the first threshold and the second threshold to generate a multi-level region, wherein the multi-level region includes an area of ​​interest group, a possible area group, and a non-interest area group; A coding processing unit is configured to perform coding processing on the interest region group, the possible region group and the non-interest region group included in the multi-level region, respectively, to obtain multiple coded video frames; Wherein, the filtering processing unit is further configured to: Performing discrete cosine transform processing on each coding unit in the coding unit set to generate a frequency component distribution matrix, thereby obtaining a frequency component distribution matrix group; Performing filtering processing on each frequency component distribution matrix in the frequency component distribution matrix group to generate a filtered frequency component distribution matrix as a filtered frequency component distribution matrix, thereby obtaining a filtered frequency component distribution matrix group; Performing accumulation processing on each element included in each filter frequency component distribution matrix in the filter frequency component distribution matrix group to generate filter frequency component significance as coding unit significance, and obtaining a coding unit significance group; Based on the coding unit significance group, a first threshold and a second threshold are generated.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images