Video coding and decoding method and device

Abstract

The invention belongs to the technical field of video processing, and particularly discloses a video encoding and decoding method and device. According to the method and the device, the interested areas are comprehensively judged in multiple aspects by combining an interested area intelligent detection algorithm of manual annotation and correction, the video is divided into a plurality of interested areas, and the interested area grade table is established; according to the grade of each region-of-interest in the region-of-interest grade table corresponding to the video and the output code rate range of the video, coding resources and decoding resources are comprehensively allocated, the video code rate resource configuration is optimized, the code rate consumption of a non-region-of-interest is reduced, and meanwhile, the decoding effect is improved, so that the problem that in a complex and changing network environment, the decoding efficiency is greatly improved is solved. The problem that the real-time video transmission quality is poor due to insufficient network bandwidth is solved, and better image overall quality is provided for users under the condition that the overall coding output code rate meets the requirements of the users.

Description

Technical Field

[0001] The present application belongs to the field of video processing technology, and more specifically, to a video encoding and decoding method and device. Background Art

[0002] Since the advent of the Internet era, multimedia technology and communication technology have developed rapidly, and high-definition 1080P and ultra-high-definition 4K, 8K resolution video images are gradually entering our work and life. Compared with text, pictures and sound, video signals have a larger amount of data, especially in the case of limited bandwidth of wireless transmission networks such as satellite communications, 4G, and 5G, the research on high-compression technology of video is very important.

[0003] At present, the direct and effective solution to the above problems is to reduce the resolution and frame rate of video encoding. There are also some more complex processing methods, which are to compress the video image using the region of interest (ROI) compression technology, realize the identification of the region of interest in the image through a certain algorithm, and then send the region coordinates as parameters to the ROI compression module. The compression module adjusts the bit rate allocation of the region of interest and the non-region of interest according to the parameters, improves the image quality of the region of interest, improves the user's visual perception effect, and reduces the compression bit rate of the overall video image. This approach will lead to excessive requirements for a single algorithm. There is no unified standard for the selection of regions of interest by different algorithms. If the algorithm has low accuracy, or the video changes exceed the large model requirement range of the algorithm, it will seriously reduce the selection of the region of interest and affect the compression effect. At the same time, indiscriminate decoding at the decoding end will also affect the decoding effect of key areas and increase research and development, and cannot fully utilize decoding resources.

[0004] How to solve the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment is an urgent problem that needs to be solved. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of this application is to provide a video encoding and decoding method and device, aiming to solve the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment in the prior art.

[0006] To achieve the above objectives, in a first aspect, the present application provides a video encoding and decoding method, comprising:

[0007] According to the focus condition instruction received by the region of interest detection module, at least one region of interest sub-detection module in the region of interest detection module is called to detect the video to obtain at least one region of interest;

[0008] Calling the region of interest judgment module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and combining the region of interest judgment strategy to obtain a region of interest level table corresponding to the video;

[0009] According to the level of each ROI in the ROI level table, encoding resources and decoding resources are allocated to perform encoding and decoding processing on the video.

[0010] In some embodiments, according to the focus condition instruction received by the region of interest detection module, calling at least one region of interest sub-detection module in the region of interest detection module to detect the video and obtain at least one region of interest includes:

[0011] Determine at least one region of interest sub-detection module to be called according to the focus condition corresponding to each region of interest sub-detection module in the focus condition instruction received by the region of interest detection module;

[0012] At least one region of interest sub-detection module is called to detect the video according to the ignorable area in the focus condition instruction to obtain at least one region of interest.

[0013] In some embodiments, the region of interest determination module is called to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and combined with the region of interest determination strategy, a region of interest level table is obtained, including:

[0014] Calling an area of ​​interest determination module to receive at least one area of ​​interest, a focus area in a focus condition instruction, and a priority level corresponding to the focus area;

[0015] In the case where there are overlapping areas among the regions of interest, the region of interest having more overlapping areas with the other regions of interest has a higher rank;

[0016] The level of the region of interest overlapping with the focused area of ​​interest is added to the priority level corresponding to the focused area of ​​interest to obtain the level of the region of interest overlapping with the focused area of ​​interest.

[0017] In some embodiments, according to the level of each region of interest in the region of interest level table, encoding resources and decoding resources are allocated to encode and decode the video, including:

[0018] According to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, the encoding parameter of each region of interest is determined, and the output resolution and output frame rate are adjusted according to the encoding resource allocation strategy, and different encoding resources are allocated to regions of interest of different levels to encode the video and obtain encoded data;

[0019] Decoding parameters of each ROI are determined according to the level and coded data of each ROI, and more decoding resources are allocated to ROIs with higher levels and less decoding resources are allocated to ROIs with lower levels, so as to decode the coded data.

[0020] In some embodiments, encoding a resource allocation strategy includes:

[0021] When the input resolution and input frame rate of the video take different values, the output resolution and output frame rate are determined according to the output bit rate range, and more coding resources are allocated to higher-level regions of interest, and fewer coding resources are allocated to lower-level regions of interest.

[0022] In some embodiments, the network protocol for transmitting the encoded data is the real-time transport protocol RTP or the real-time streaming protocol RTSP, and the network protocol for transmitting the region of interest level table is the user datagram protocol UDP or the transmission control protocol TCP.

[0023] In a second aspect, the present application provides a video encoding and decoding device, including:

[0024] A first acquisition module is used to call at least one region of interest sub-detection module in the region of interest detection module to detect the video and acquire at least one region of interest according to the focus condition instruction received by the region of interest detection module;

[0025] A second acquisition module is used to call the region of interest judgment module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and obtain a region of interest level table corresponding to the video in combination with the region of interest judgment strategy;

[0026] The encoding and decoding module is used to allocate encoding resources and decoding resources according to the level of each region of interest in the region of interest level table to perform encoding and decoding processing on the video.

[0027] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the video encoding and decoding method described in the first aspect or any embodiments of the first aspect.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the video encoding and decoding method described in the first aspect or any embodiments of the first aspect.

[0029] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the video encoding and decoding method described in the first aspect or any embodiments of the first aspect.

[0030] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0031] The video encoding and decoding method and device provided by the present application, combined with manual annotation and modified region of interest intelligent detection algorithm, comprehensively judge the region of interest from multiple aspects, divide the video into several regions of interest and establish a region of interest level table, comprehensively allocate encoding resources and decoding resources according to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, optimize the video bit rate resource configuration, reduce the bit rate consumption of non-interested areas, and at the same time, improve the decoding effect, so as to solve the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment, and ensure that the overall encoding output bit rate meets the user's requirements and provides users with better overall image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of a video encoding and decoding method provided in an embodiment of the present application;

[0033] Figure 2 It is a schematic diagram of the comprehensive detection compression and decompression process provided by the embodiment of the present application;

[0034] Figure 3 This is a schematic diagram of the format of the key attention condition instruction sent by the client provided in the embodiment of the present application;

[0035] Figure 4 It is a schematic diagram of the process flow of the region of interest determination strategy provided in an embodiment of the present application;

[0036] Figure 5 The level representation of the region of interest provided in the embodiment of the present application is intended;

[0037] Figure 6 is a schematic diagram of the structure of a video encoding and decoding device provided in an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0041] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more than two.

[0043] The video encoding and decoding method and device provided in the embodiment of the present application, by combining manual labeling and a modified ROI intelligent detection algorithm, detects the region of interest of the current video screen and implements multi-level division, thereby achieving real-time video high compression adaptation, reducing the bandwidth occupied by compression, and making the video quality more in line with the visual characteristics of the human eye.

[0044] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0045] See also Figure 1 , an embodiment of the present application provides a video encoding and decoding method, which may include: step 110, step 120 and step 130.

[0046] Step 110: According to the focus condition instruction received by the region of interest detection module, at least one region of interest sub-detection module in the region of interest detection module is called to detect the video and obtain at least one region of interest;

[0047] Step 120 calls the region of interest judgment module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and obtains a region of interest level table corresponding to the video in combination with the region of interest judgment strategy;

[0048] Step 130 allocates encoding resources and decoding resources according to the level of each ROI in the ROI level table to perform encoding and decoding processing on the video.

[0049] Please see further Figure 2In the embodiment of the present application, after the video acquisition module at the sending end acquires the video, the acquired video is sent to the region of interest detection module in parallel. The region of interest detection module receives the focus condition instruction sent by the user through the client, calls at least one region of interest sub-detection module inside it to perform comprehensive detection and judgment on the video, and then outputs one or more corresponding regions of interest (specifically, the coordinates corresponding to one or more regions of interest), and sends them to the region of interest judgment module.

[0050] The region of interest judgment module receives the focus condition instruction sent by the user through the client in real time, integrates one or more regions of interest of the region of interest detection module, and determines the coordinates corresponding to the final region of interest according to the region of interest judgment strategy to form a region of interest level table.

[0051] In the embodiment of the present application, the interest region determination strategy is to form an interest region level table according to the user's focus condition instruction and the coordinates corresponding to one or more interest regions sent by the interest region detection module.

[0052] In an embodiment of the present application, the region of interest sub-detection module included in the region of interest detection module can be specifically composed of a face recognition module (for identifying the face area in the video), a human figure detection module (for identifying the area where the person is located in the video), a moving frame detection module (for identifying the area where the moving object is located in the video), a target tracking module (for real-time tracking of a specific target in the video), an abnormal behavior detection module (for identifying the abnormal behavior area of ​​the target in the video), a traffic flow monitoring module (for detecting areas with large traffic flow in the video), an intrusion monitoring module (for identifying whether the protected area in the video is invaded), a relic monitoring module (for identifying whether there are items left in a specific area in the video), and other expandable and deletable sub-detection modules.

[0053] The region of interest determination module receives the user's focus condition instruction and decides to start one or more region of interest sub-detection modules. If there are multiple region of interest sub-detection modules, each region of interest sub-detection module completes the recognition of the video image in parallel.

[0054] Please see further Figure 3 In the embodiment of the present application, the focus condition instruction sent by the client includes information such as the output bit rate upper limit, focus conditions, focus areas, the number of focus areas, and the priority corresponding to each area of ​​interest, and the ignorable area.

[0055] Among them, the output bit rate upper limit is the maximum output bit rate of the sender set by the user according to the actual bandwidth situation. The key attention condition has two bytes of 16 bits in total, which correspond to the region of interest detection module one by one. Each bit corresponds to a region of interest sub-detection module in the region of interest detection module. If the bit value is 1, the corresponding region of interest sub-detection module is turned on, and the bit value is 0, the corresponding region of interest sub-detection module is turned off.

[0056] The focus area is the focus area drawn by the user based on the real-time picture output by the client decoding output module, and the number of focus areas is the number of focus areas drawn by the user based on the real-time picture output by the client decoding output module. The higher the priority, the higher the focus level of the region of interest. The upper left coordinates and the width and height of the region of interest refer to the image range of this region of interest. The ignorable area is the area range that the user draws that does not need to be detected. All the sub-detection modules of the region of interest do not need to detect these areas, and these areas can occupy as little encoding output bandwidth as possible. The message header, message tail and flag bit are fixed values.

[0057] The video encoding and decoding method provided in the embodiment of the present application combines manual annotation and a modified region of interest intelligent detection algorithm to comprehensively determine the region of interest from multiple aspects, divide the video into a number of regions of interest and establish a region of interest level table, and comprehensively allocate encoding resources and decoding resources according to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, optimize the video bit rate resource configuration, reduce the bit rate consumption of non-interested regions, and at the same time, improve the decoding effect, so as to solve the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment, and ensure that the overall encoding output bit rate meets the user's requirements and provides users with better overall image quality.

[0058] Further, in some embodiments, in step 110, according to the focus condition instruction received by the region of interest detection module, calling at least one region of interest sub-detection module in the region of interest detection module to detect the video and obtain at least one region of interest may include:

[0059] Determine at least one region of interest sub-detection module to be called according to the focus condition corresponding to each region of interest sub-detection module in the focus condition instruction received by the region of interest detection module;

[0060] At least one region of interest sub-detection module is called to detect the video according to the ignorable area in the focus condition instruction to obtain at least one region of interest.

[0061] Please see further Figure 4In the embodiment of the present application, the region of interest detection module obtains the key focus condition instruction sent by the user through the client, and obtains the key focus condition, key focus area, ignorable area and other information in the key focus condition instruction. According to the key focus condition, several corresponding region of interest sub-detection modules in the region of interest detection module are turned on or off. Each region of interest sub-detection module detects the screen range of the video screen without the ignorable area according to the user's requirements and outputs the detection result. The coordinates of all regions of interest are uniformly sorted and sent to the region of interest judgment module to form a region of interest level table.

[0062] Further, in some embodiments, in step 120, calling the region of interest determination module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and combining the region of interest determination strategy to obtain the region of interest level table may include:

[0063] Calling an area of ​​interest determination module to receive at least one area of ​​interest, a focus area in a focus condition instruction, and a priority level corresponding to the focus area;

[0064] In the case where there are overlapping areas among the regions of interest, the region of interest having more overlapping areas with the other regions of interest has a higher rank;

[0065] The level of the region of interest overlapping with the focused area of ​​interest is added to the priority level corresponding to the focused area of ​​interest to obtain the level of the region of interest overlapping with the focused area of ​​interest.

[0066] In the embodiment of the present application, the region of interest judgment module receives the coordinates corresponding to one or more regions of interest sent by the region of interest detection module, the focus region in the focus condition instruction, and the priority corresponding to the focus region. When the coordinates corresponding to each region of interest overlap, the region of interest with more overlapping areas with other regions of interest has a higher level of the screen. When the region of interest overlaps with the focus region in the focus condition instruction sent by the client, the level of the screen in this region of interest is superimposed on the priority of the focus region in the focus condition instruction as its corresponding level, and the region of interest level table is sorted out according to the calculated level of the region of interest and the coordinates of the corresponding region of interest.

[0067] Further, in some embodiments, in step 130, allocating encoding resources and decoding resources according to the level of each region of interest in the region of interest level table to perform encoding and decoding processing on the video may include:

[0068] According to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, the encoding parameter of each region of interest is determined, and the output resolution and output frame rate are adjusted according to the encoding resource allocation strategy, and different encoding resources are allocated to regions of interest of different levels to encode the video and obtain encoded data;

[0069] Decoding parameters of each ROI are determined according to the level and coded data of each ROI, and more decoding resources are allocated to ROIs with higher levels and less decoding resources are allocated to ROIs with lower levels, so as to decode the coded data.

[0070] Please continue to see Figure 2 The ROI level table contains the coordinates of one or more ROIs and their corresponding levels. A higher level indicates a higher image quality requirement for the corresponding ROI. The output bitrate range is determined by the output bitrate upper limit set by the user.

[0071] The encoding module completes video encoding according to the region of interest table corresponding to the video and obtains encoded data. Specifically, the encoding module determines the encoding parameters of each region of interest through the region of interest level table corresponding to the video and the output bit rate range given by the user, adjusts the output resolution and output frame rate according to the encoding parameters and the encoding resource allocation strategy, and allocates different encoding resources to regions of interest of different levels, ensuring that better overall image quality is provided to users while meeting the user's output bit rate limit.

[0072] The video encoding and decoding method provided in the embodiment of the present application determines the encoding parameters of each region of interest through the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, so as to comprehensively allocate encoding resources, optimize the video bit rate resource configuration, and reduce the bit rate consumption of non-interested regions. At the same time, when decoding the encoded data, different decoding parameters are allocated to each region of interest according to its level to improve the decoding effect, so as to solve the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment, and ensure that the overall encoding output bit rate meets the user's requirements and provides users with better overall image quality.

[0073] Furthermore, in some embodiments, in the above steps, the encoding resource allocation strategy may include:

[0074] When the input resolution and input frame rate of the video take different values, the output resolution and output frame rate are determined according to the output bit rate range, and more coding resources are allocated to higher-level regions of interest, and fewer coding resources are allocated to lower-level regions of interest.

[0075] In the embodiment of the present application, the coding resource allocation strategy means that when the video input resolution and input frame rate take different values, the coding module will determine the output resolution and output frame rate according to the output bit rate upper limit set by the user, and allocate different coding resources to the regions of interest at different levels, and reduce the quantization parameter value for the regions of interest with high levels, such as: allocating a lower quantization step to the regions of interest with high levels, thereby allocating a higher bit rate to make them have better picture quality; and increasing the quantization parameter value for the regions of interest with low levels, thereby allocating less bit rate. The overall coding output bit rate provides users with better overall image quality under the condition that the output bit rate requirements of the user are met.

[0076] In an embodiment of the present application, the decoding module obtains the level table of the region of interest corresponding to the video and parses the content in the level table of the region of interest to obtain the level of each region of interest. In combination with the obtained encoded data, different decoding parameters are assigned to several regions of interest corresponding to different levels of the video screen, more decoding resources are assigned to regions of interest with high levels, and fewer decoding resources are assigned to regions of interest with low levels, so as to decode the encoded data, thereby ensuring the picture quality of the user's area of ​​interest under limited bandwidth while reducing the decoding delay.

[0077] In the embodiment of the present application, the level of each region of interest can be set according to the user's requirements for the image quality of each region of interest. A region of interest with a high level has a high image quality requirement, and a region of interest with a low level has a low image quality requirement.

[0078] Furthermore, in some embodiments, the network protocol for transmitting the encoded data is the real-time transport protocol RTP or the real-time streaming protocol RTSP, and the network protocol for transmitting the region of interest level table is the user datagram protocol UDP or the transmission control protocol TCP.

[0079] Please continue to see Figure 2 In an embodiment of the present application, the network sending module sends the encoded data and the area of ​​interest level table to the network.

[0080] The decoding module receives the encoded data and the region of interest level table sent to the network, parses the content in the region of interest level table, assigns different decoding parameters to several regions of interest in the video screen, decodes the encoded data, outputs the decoded data through the client decoding output module, and displays the decoded data on the client.

[0081] In the embodiment of the present application, the encoded data transmission network protocol is generally the Real-time Transport Protocol (RTP) or the Real-time Streaming Protocol (RTSP), and the transmission network protocol of the area of ​​interest level table is the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP).

[0082] Please see further Figure 5 The region of interest level table formed in the embodiment of the present application is composed of information such as the number of regions of interest, the level of the regions of interest, and the corresponding coordinates of the regions of interest. The number of regions of interest is the number of ROI images finally output by the region of interest judgment strategy. The higher the level, the more coding resources should be allocated to the region of interest. The header and the tail of the message are fixed values.

[0083] As shown in Table 1, the coding resource allocation strategy means that when the video input resolution and input frame rate take different values, the output resolution and output frame rate values ​​will be determined according to the output bit rate upper limit set by the user, and different quantization step parameters will be allocated to different levels of regions of interest, so that the overall coding output bit rate can provide users with better overall image quality while meeting user requirements.

[0084] Table 1

[0085]

[0086] When the output bitrate is capped at 64Mb / s, 32Mb / s, and 16Mb / s, the output resolution and output frame rate are consistent with the input resolution and input frame rate. When the output bitrate is capped at 8Mb / s and 4Mb / s, the output frame rate remains unchanged and the output resolution is 1920*1080. If the input resolution is lower than 1920*1080 at this time, the output resolution remains consistent with the input resolution. When the output bitrate is capped at 2Mb / s, the output frame rate is 30 frames and the output resolution is 1024*768. If the input resolution is lower than 1024*768 at this time, the output resolution remains consistent with the input resolution. If the input frame rate is lower than 30 at this time, the output frame rate remains consistent with the input frame rate. When the output bitrate is capped at 1Mb / s, the output frame rate is 25 frames and the output resolution is 720*576. If the input resolution is lower than 720*576, the output resolution will remain the same as the input resolution. If the input frame rate is lower than 25, the output frame rate will remain the same as the input frame rate. When the output bitrate is capped at 512kb / s, the output frame rate is 20 frames and the output resolution is 640*480. If the input resolution is lower than 640*480, the output resolution will remain the same as the input resolution. If the input frame rate is lower than 20, the output frame rate will remain the same as the input frame rate. When the output bitrate is capped at 256kb / s, the output frame rate is 20 frames and the output resolution is 320*240. If the input resolution is lower than 320*240, the output resolution will remain the same as the input resolution. If the input frame rate is lower than 20, the output frame rate will remain the same as the input frame rate.

[0087] While reducing the output resolution and output frame rate, the encoding parameters are reallocated for the regions of interest in the region of interest level table sent by the region of interest judgment module, and the quantization step parameters are divided into 9 levels, from Q1 to Q9, and gradually increased. The smaller the quantization step, the greater the encoding output bit rate of this region of interest. All pictures in non-regions of interest are allocated to Q9, and the pictures in the lowest-level region of interest in the region of interest level table are allocated to Q8, and so on, and the pictures in the highest-level region of interest are allocated to Q1. In this way, the parameter allocation of the region of interest is completed.

[0088] The video encoding and decoding method provided in the embodiment of the present application solves the problem of poor quality of real-time video transmission due to insufficient network bandwidth in a complex and changing network environment. It combines the resolution frame rate adjustment method, as well as the manually labeled and corrected ROI intelligent detection algorithm to realize functions such as automatic detection of video application scenarios and adaptive adjustment of compression parameters. The video bit rate resource configuration is optimized to further reduce the amount of bit stream data. At the same time, at the decoding and playback end, an intelligent perception algorithm matched with the intelligent ROI detection algorithm is used to assign different decoding parameters to several interest-level areas of the video screen, thereby improving the decoding effect and the overall image quality under low bandwidth constraints. Different from the traditional encoding and transmission method under low bandwidth, the present application compresses the terminal area more purposefully to meet the user's demand for stable and smooth video stream playback under low bandwidth conditions.

[0089] The video coding and decoding device provided by the present application is described below. The video coding and decoding device described below and the video coding and decoding method described above can refer to each other.

[0090] See also Figure 6 , an embodiment of the present application provides a video encoding and decoding device, which may include: a first acquisition module 610, a second acquisition module 620 and a encoding and decoding module 630.

[0091] A first acquisition module 610 is configured to call at least one region of interest sub-detection module in the region of interest detection module to detect the video and acquire at least one region of interest according to the focus condition instruction received by the region of interest detection module;

[0092] The second acquisition module 620 is used to call the region of interest judgment module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and obtain a region of interest level table corresponding to the video in combination with the region of interest judgment strategy;

[0093] The encoding and decoding module 630 is used to allocate encoding resources and decoding resources according to the level of each ROI in the ROI level table to perform encoding and decoding processing on the video.

[0094] The video encoding and decoding device provided in the embodiment of the present application combines manual annotation and modified region of interest intelligent detection algorithm, comprehensively judges the region of interest from multiple aspects, divides the video into several regions of interest and establishes a region of interest level table, comprehensively allocates encoding resources and decoding resources according to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, optimizes the video bit rate resource configuration, reduces the bit rate consumption of non-interested areas, and improves the decoding effect at the same time, so as to solve the problem of poor real-time video transmission quality due to insufficient network bandwidth in a complex and changing network environment, and ensure that the overall encoding output bit rate meets the user's requirements and provides users with better overall image quality.

[0095] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0096] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0097] Based on the method in the above embodiment, the present application embodiment provides an electronic device, see Figure 7 The electronic device may include: a processor (Processor) 710, a communication interface (CommunicationsInterface) 720, a memory (Memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the method in the above embodiment.

[0098] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0099] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0100] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0101] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0102] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0104] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0105] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A video encoding and decoding method, characterized in that: include: According to the focus condition instruction received by the region of interest detection module, calling at least one region of interest sub-detection module in the region of interest detection module to detect the video and obtain at least one region of interest; Calling an area of ​​interest judgment module to receive the focus condition instruction and at least one area of ​​interest sent by the area of ​​interest detection module, and combining an area of ​​interest judgment strategy to obtain an area of ​​interest level table corresponding to the video; According to the level of each ROI in the ROI level table, encoding resources and decoding resources are allocated to perform encoding and decoding processing on the video.

2. The video encoding and decoding method according to claim 1, characterized in that: The step of calling at least one region of interest sub-detection module in the region of interest detection module to detect the video according to the focus condition instruction received by the region of interest detection module to obtain at least one region of interest includes: Determine at least one region of interest sub-detection module to be called according to the key focus condition corresponding to each region of interest sub-detection module in the key focus condition instruction received by the region of interest detection module; The at least one region of interest sub-detection module is called to detect the video according to the ignorable area in the focus condition instruction to obtain the at least one region of interest.

3. The video encoding and decoding method according to claim 1, characterized in that: The calling of the region of interest judgment module receives the focus condition instruction and at least one region of interest sent by the region of interest detection module, and obtains the region of interest level table in combination with the region of interest judgment strategy, including: Calling the region of interest determination module to receive the at least one region of interest, the focus region in the focus condition instruction, and the priority corresponding to the focus region; In the case where there are overlapping areas among the regions of interest, the region of interest having more overlapping areas with the other regions of interest has a higher rank; The level of the region of interest overlapping with the focused region of interest is added to the priority level corresponding to the focused region of interest to obtain the level of the region of interest overlapping with the focused region of interest.

4. The video encoding and decoding method according to claim 1, characterized in that: The allocating encoding resources and decoding resources according to the level of each region of interest in the region of interest level table to perform encoding and decoding processing on the video includes: Determine the encoding parameters of each region of interest according to the level of each region of interest in the region of interest level table corresponding to the video and the output bit rate range of the video, adjust the output resolution and output frame rate according to the encoding resource allocation strategy, allocate different encoding resources to regions of interest of different levels, so as to encode the video and obtain encoded data; Determine the decoding parameters of each region of interest according to the levels of the respective regions of interest and the coded data, allocate more decoding resources to regions of interest with higher levels, and allocate fewer decoding resources to regions of interest with lower levels, so as to decode the coded data.

5. The video encoding and decoding method according to claim 4, characterized in that: The coding resource allocation strategy includes: When the input resolution and input frame rate of the video take different values ​​respectively, the output resolution and output frame rate are determined according to the output bit rate range, and more coding resources are allocated to higher-level regions of interest, and fewer coding resources are allocated to lower-level regions of interest.

6. The video encoding and decoding method according to claim 4, characterized in that: The network protocol for transmitting the encoded data is the real-time transport protocol RTP or the real-time streaming protocol RTSP, and the network protocol for transmitting the region of interest level table is the user datagram protocol UDP or the transmission control protocol TCP.

7. A video encoding and decoding device, characterized in that: include: A first acquisition module is used to call at least one region of interest sub-detection module in the region of interest detection module to detect the video and acquire at least one region of interest according to the focus condition instruction received by the region of interest detection module; A second acquisition module is used to call the region of interest judgment module to receive the focus condition instruction and at least one region of interest sent by the region of interest detection module, and obtain a region of interest level table corresponding to the video in combination with the region of interest judgment strategy; The encoding and decoding module is used to allocate encoding resources and decoding resources according to the level of each region of interest in the region of interest level table to perform encoding and decoding processing on the video.

8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the video encoding and decoding method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is enabled to execute the video encoding and decoding method according to any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is enabled to execute the video encoding and decoding method according to any one of claims 1 to 6.

Images