Regional high-efficiency video coding method for mine low-bit-rate video transmission

Abstract

The invention provides a regional efficient video coding method for mine low-bit-rate video transmission, and relates to the technical field of video processing. The method comprises the steps of firstly obtaining each initial image frame corresponding to a first video stream, determining first metadata corresponding to each initial image frame, then determining a first image region and a second image region in each initial image frame based on the first metadata for each initial image frame, and determining a first coding strategy corresponding to the first image area in each initial image frame and a second coding strategy corresponding to the second image area in each initial image frame, and finally coding the initial image frames based on the first coding strategies and the second coding strategies to obtain a coded second video stream. Therefore, the video coding quality and effect are improved, and the video coding efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a regional efficient video encoding method for low-bitrate video transmission in mines. Background Art

[0002] Nowadays, with the widespread use of high-definition and high-bitrate cameras, the image quality of videos in mines is getting higher and higher. However, increasingly high-definition and high-resolution videos bring huge challenges to storage and transmission, so video coding and compression are particularly important. However, the existing methods for encoding videos in mines cannot guarantee the quality of video coding, which affects the effect of video coding. Summary of the invention

[0003] The present invention provides a regional efficient video encoding method for low bit rate video transmission in mines.

[0004] According to one aspect of the present disclosure, a regional efficient video encoding method for low bit rate video transmission in a mine is provided, comprising: Obtain each initial image frame corresponding to the first video stream; Determine first metadata corresponding to each of the initial image frames, wherein the first metadata is data used to describe a region of interest in the initial image frame; For each initial image frame, determining a first image region and a second image region in the initial image frame based on the first metadata, wherein the first image region is a region of interest, and the second image region is other image regions in the initial image frame except the first image region; Determining a first encoding strategy corresponding to a first image region in each of the initial image frames, and a second encoding strategy corresponding to a second image region; Based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain an encoded second video stream.

[0005] The present disclosure provides a regional efficient video encoding method for low-bitrate video transmission in mines. First, each initial image frame corresponding to the first video stream is obtained, and the first metadata corresponding to each initial image frame is determined. Then, for each initial image frame, the first image area and the second image area in the initial image frame are determined based on the first metadata. Then, the first encoding strategy corresponding to the first image area in each initial image frame and the second encoding strategy corresponding to the second image area are determined. Finally, based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain the encoded second video stream. Thus, based on the metadata of the area of ​​interest corresponding to each initial image frame in the video stream, the area of ​​interest and the area of ​​non-interest are determined in each initial image frame, and the area of ​​interest is encoded with high quality, and the area of ​​non-interest is encoded with high compression to obtain the encoded video stream, thereby ensuring the image quality of the area of ​​interest, improving the quality and effect of video encoding, and improving the efficiency of video encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0007] Figure 1 A flowchart of a method for efficient video encoding in different regions for low bit rate video transmission in mines provided by an embodiment of the present disclosure; Figure 2 A flowchart of a method for efficient video encoding in different regions for low bit rate video transmission in mines provided by an embodiment of the present disclosure; Figure 3 A flowchart of a method for efficient video encoding in different regions for low bit rate video transmission in mines provided by an embodiment of the present disclosure; Figure 4 A flowchart of a method for efficient video encoding in different regions for low bit rate video transmission in mines provided by an embodiment of the present disclosure; Figure 5 A flowchart of a method for efficient video encoding in different regions for low bit rate video transmission in mines provided by an embodiment of the present disclosure; Figure 6 A schematic diagram of the effect of the regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0008] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0009] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0010] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0011] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0012] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0013] The following describes in detail the regional efficient video encoding method for low bit rate video transmission in mines according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0014] Figure 1 A flowchart of a regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0015] like Figure 1 As shown, the method includes: Step 101: Acquire each initial image frame corresponding to the first video stream.

[0016] The first video stream may be a video stream to be encoded, which may be a video stream of any scene. For example, the first video stream may be a video stream of a scene in a coal mine, which is not limited in the present disclosure.

[0017] It should be noted that the first video stream may be obtained in any manner. For example, when the first video stream is a video stream of a coal mine underground scene, it may be obtained through a camera in the coal mine underground scene, and the present disclosure does not limit this.

[0018] It should be noted that the number of initial image frames corresponding to the first video stream can be any number, and the present disclosure does not limit this.

[0019] In the present disclosure, when obtaining the first video stream through a camera, since the video stream output by the camera is pre-encoded, the first video stream must first be decoded to obtain the decoded first video stream, and then based on the decoded first video stream, the initial image frame corresponding to the first video stream is obtained.

[0020] It should be noted that after obtaining the initial image frame corresponding to the first video stream, the size of the initial image frame can also be determined, and then the format of the initial image frame is converted into a preset format, for example, converted into a YUV420 format, which is not limited in the present disclosure.

[0021] Among them, YUV420 is a color encoding method. "Y" can represent brightness (Luminance or Luma), that is, the grayscale value, "U" can represent chrominance (Chrominance), and "V" can represent concentration (Chroma).

[0022] In the present disclosure, after obtaining each initial image frame corresponding to the first video stream, the spatial, temporal, visual and statistical redundant data existing in the initial image frame can be eliminated through intra-frame prediction, inter-frame prediction, conversion, quantization and entropy coding, thereby improving the efficiency of video coding.

[0023] Step 102: determine first metadata corresponding to each initial image frame, wherein the first metadata is data used to describe a region of interest in the initial image frame.

[0024] It should be noted that the first metadata may include size information of the width and length of the region of interest ROI in the corresponding initial image frame, coordinate information of the region of interest in the initial image frame, and other arbitrary information, which is not limited in the present disclosure.

[0025] Among them, ROI is the abbreviation of Region of Interest (ROI).

[0026] It should be noted that the regions of interest included in each initial image frame may be different or the same, and the number of regions of interest included may be different or the same, which is not limited in the present disclosure.

[0027] In the present disclosure, the first video stream can be input into a preset artificial intelligence (AI) edge computing box, and then the first video stream can be decoded to obtain each corresponding initial image frame, and then the first metadata of each initial image frame can be determined based on the neural network model preset in the AI ​​edge computing box, wherein the neural network model can be a model of any type and structure for determining the first metadata, and the present disclosure does not limit this.

[0028] Step 103, for each initial image frame, based on the first metadata, determine a first image area and a second image area in the initial image frame, wherein the first image area is the region of interest, and the second image area is other image areas in the initial image frame except the first image area.

[0029] In the present disclosure, after determining the first metadata of each initial image frame, the first image area and the second image area in the corresponding initial image frame can be determined based on the first metadata corresponding to each initial image frame, thereby improving the efficiency of determining the first image area in the initial image frame.

[0030] Step 104: Determine a first encoding strategy corresponding to a first image region in each initial image frame, and a second encoding strategy corresponding to a second image region.

[0031] The first encoding strategy may be a high-quality encoding strategy for the first image region. For example, the first image region is a region of interest in the initial image frame. In order to ensure the image quality of the first image region, a high-quality encoding strategy may be used to encode the first image region, which is not limited in the present disclosure.

[0032] The second encoding strategy may be a high-compression encoding strategy for the second image region. For example, the second image region is an image region other than the first image region in the initial image frame, that is, a non-interest region. In order to reduce bandwidth overhead while ensuring the image quality of the first image region, a high-compression encoding strategy may be used for encoding the second image region, and the present disclosure does not limit this.

[0033] It should be noted that, for different second image regions, the corresponding second encoding strategies may be determined to be different, and the present disclosure does not limit this.

[0034] It should be noted that the first coding strategy and the second coding strategy may be different. The first coding strategy and the second coding strategy may use the same coding protocol, such as the H.264 coding protocol or the H.265 coding protocol, but the video coding compression rate of the first coding strategy and the second coding strategy is different, which is not limited in this disclosure.

[0035] Among them, H.264 can also be called Advanced Video Coding (AVC), and H.265 can also be called High Efficiency Video Coding (HEVC).

[0036] In the present disclosure, after determining the first image area and the second image area in each initial image frame, in order to improve the encoding quality of the first video stream and improve resource utilization, the first encoding strategy corresponding to the first image area in each initial image frame and the second encoding strategy corresponding to the second image area can be determined respectively.

[0037] Step 105 : Encode the initial image frame based on the first encoding strategy and the second encoding strategy to obtain an encoded second video stream.

[0038] In the present disclosure, after determining the first encoding strategy corresponding to the first image area in each initial image frame and the second encoding strategy corresponding to the second image area, the corresponding initial image frame can be encoded based on the first encoding strategy and the second encoding strategy. After encoding of each initial image frame of the first video stream is completed, the encoded second video stream can be obtained.

[0039] In the disclosed embodiment, each initial image frame corresponding to the first video stream is first obtained, and the first metadata corresponding to each initial image frame is determined. Then, for each initial image frame, based on the first metadata, the first image area and the second image area in the initial image frame are determined. Then, the first encoding strategy corresponding to the first image area in each initial image frame and the second encoding strategy corresponding to the second image area are determined. Finally, based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain the encoded second video stream. Thus, by determining the region of interest and the region of non-interest in each initial image frame based on the region of interest metadata corresponding to each initial image frame in the video stream, high-quality encoding is performed on the region of interest, and high-compression encoding is performed on the region of non-interest, to obtain the encoded video stream, thereby ensuring the image quality of the region of interest, improving the quality and effect of video encoding, and improving the efficiency of video encoding.

[0040] Figure 2 A flowchart of a regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0041] like Figure 2 As shown, the method includes: Step 201: Acquire each initial image frame corresponding to the first video stream.

[0042] Step 202: determine first metadata corresponding to each initial image frame, wherein the first metadata is data used to describe a region of interest in the initial image frame.

[0043] The specific implementation forms of step 201 to step 202 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0044] Step 203: for each initial image frame, obtain a foreground mask and a background mask corresponding to the initial image frame.

[0045] The foreground mask can be used to identify the foreground area in the initial image frame, and the background mask can be used to identify the background area in the initial image frame.

[0046] It should be noted that the foreground mask and the background mask corresponding to the initial image frame can both be masks in the form of a two-dimensional array that is as large as the corresponding initial image frame, wherein the number corresponding to the foreground area position in the foreground mask can be 1, and the number corresponding to the background area position can be 0. The background mask is opposite to the foreground mask, and the present disclosure does not limit this.

[0047] It should be noted that different initial image frames may have different corresponding foreground masks, which is not limited in the present disclosure.

[0048] In the present disclosure, after determining the first metadata corresponding to each initial image frame, in order to improve the accuracy of the determined first image region in the initial image frame, the corresponding foreground mask and background mask may be determined for each initial image frame.

[0049] Step 204 : determining a first image region in the initial image frame based on the first metadata and the foreground mask.

[0050] In the present disclosure, after obtaining the foreground mask corresponding to the initial image frame, the first image area in the initial image frame can be determined based on the first metadata corresponding to the initial image frame and the foreground mask, and the first image area can be identified by a bounding box, thereby improving the accuracy and reliability of the determined first image area in the initial image frame.

[0051] It should be noted that the boundary box of the first image area can be determined by the size and coordinates of the first image area included in the corresponding first metadata, and the present disclosure does not limit this.

[0052] Step 205 : determining a second image region in the initial image frame based on the background mask and the first image region.

[0053] In the present disclosure, after determining the first image area in the initial image frame, other image areas in the background area identified by the background mask except the first image area can be determined as the second image area in the initial image frame, thereby improving the accuracy of the determined second image area.

[0054] Step 206: Determine a first encoding strategy corresponding to the first image region in each initial image frame, and a second encoding strategy corresponding to the second image region.

[0055] Step 207: Encode the initial image frame based on the first encoding strategy and the second encoding strategy to obtain an encoded second video stream.

[0056] The specific implementation forms of step 206 to step 207 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0057] In the disclosed embodiment, each initial image frame corresponding to the first video stream is first obtained, and the first metadata corresponding to each initial image frame is determined. Then, for each initial image frame, the foreground mask and the background mask corresponding to the initial image frame are obtained. Then, based on the first metadata and the foreground mask, the first image area in the initial image frame is determined, and based on the background mask and the first image area, the second image area in the initial image frame is determined. Finally, the first encoding strategy corresponding to the first image area in each initial image frame and the second encoding strategy corresponding to the second image area are determined, and the initial image frame is encoded based on the first encoding strategy and the second encoding strategy to obtain the encoded second video stream. Thus, by determining the region of interest of the initial image frame based on the region of interest metadata and the foreground mask corresponding to each initial image frame of the video stream, the other image areas except the region of interest in the background area identified by the background mask corresponding to the initial image frame are determined as non-regions of interest, the region of interest is encoded with high quality, and the non-region of interest is encoded with high compression, thereby improving the reliability and efficiency of the region-based efficient video encoding method.

[0058] Figure 3 A flowchart of a regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0059] like Figure 3 As shown, the method includes: Step 301: Acquire each initial image frame corresponding to the first video stream.

[0060] Step 302: determine first metadata corresponding to each initial image frame, wherein the first metadata is data used to describe a region of interest in the initial image frame.

[0061] The specific implementation forms of step 301 to step 302 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0062] Step 303: Acquire the previous initial image frame adjacent to the initial image frame, and the motion vector between the initial image frame and the previous initial image frame.

[0063] The motion vector may be a vector for detecting a moving object in the initial image frame, which may be obtained by any method. For example, the motion vector may be obtained when decoding the first video stream, which is not limited in the present disclosure.

[0064] In the present disclosure, when the initial image frame is the first initial image frame of the first video stream, the initial image frame may be used as a background image frame to obtain a motion vector between the next adjacent initial image frame and the initial image frame.

[0065] In the present disclosure, after determining the first metadata corresponding to each initial image frame, the previous initial image frame adjacent to the corresponding initial image frame and the motion vector between the initial image frame and the previous initial image frame can be obtained, thereby providing conditions for improving the accuracy of the foreground mask of the determined initial image frame.

[0066] Step 304: Determine the pixel value difference between the initial image frame and the corresponding pixel point of the previous initial image frame.

[0067] In the present disclosure, when the initial image frame is the first initial image frame of the first video stream, the initial image frame can be used as a background image frame, and the pixel value difference of the corresponding pixel point is obtained by subtracting the pixel value of the next initial image frame adjacent to it from the pixel value of the initial image frame.

[0068] In the present disclosure, by determining the pixel value difference between the corresponding pixel points of the initial image frame and the previous initial image frame, a condition is provided for improving the accuracy of the foreground mask of the determined initial image frame.

[0069] Step 305: perform edge detection on the object in the initial image frame to obtain a first detection result.

[0070] The first detection result may be an edge detection result of the object in the initial image frame, which may be a contour of the object in the initial image frame.

[0071] In the present disclosure, any edge detection method may be used to perform edge detection on the object in the initial image frame to obtain the first detection result. For example, the Sobel operator may be used to perform edge detection on the object in the initial image frame, and the present disclosure does not limit this.

[0072] Step 306: Perform motion detection on the object in the initial image frame based on the motion vector to obtain a second detection result.

[0073] The second detection result may be a motion detection result of the object in the initial image frame.

[0074] In the present disclosure, after obtaining the motion vector between the initial image frame and the previous initial image frame, motion detection can be performed on the object in the corresponding initial image frame based on the motion vector, and the image content can be divided according to the motion mode to obtain a second detection result, wherein the motion mode can be still or moving, and the present disclosure does not limit this.

[0075] Step 307: determining a foreground mask based on the pixel value difference, the first detection result, and the second detection result.

[0076] In the present disclosure, after obtaining the pixel value difference, the first detection result and the second detection result of the initial image frame, the pixel value difference, the first detection result and the second detection result of the initial image frame can be used as indication information of the foreground extraction algorithm, and the foreground mask of the initial image frame is determined through the foreground extraction algorithm, wherein the foreground extraction algorithm can be any foreground extraction algorithm, for example, the foreground extraction algorithm can be the GrabCut algorithm, etc., and the present disclosure does not limit this.

[0077] Among them, GrabCut is an image segmentation algorithm based on Graph Cut, which can be used to extract and segment the foreground of objects in the image.

[0078] It should be noted that after obtaining the foreground mask of the initial image frame, the foreground mask may be multiplied by the initial image frame, so as to extract the foreground in the initial image frame and obtain a foreground image.

[0079] Step 308: Determine a background mask based on the foreground mask and the initial image frame.

[0080] In the present disclosure, after the foreground mask of the initial image frame is determined, the background mask of the initial image frame may be obtained by subtracting the initial image frame from the foreground mask corresponding to the initial image frame.

[0081] Step 309 : determining a first image region in the initial image frame based on the first metadata and the foreground mask.

[0082] Step 310: Determine a second image region in the initial image frame based on the background mask and the first image region.

[0083] Step 311: determine a first encoding strategy corresponding to a first image region in each initial image frame, and a second encoding strategy corresponding to a second image region.

[0084] Step 312: Encode the initial image frame based on the first encoding strategy and the second encoding strategy to obtain an encoded second video stream.

[0085] The specific implementation forms of steps 309 to 312 may refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0086] In the disclosed embodiment, each initial image frame corresponding to the first video stream is first obtained, and the first metadata corresponding to each initial image frame is determined. Then, for each initial image frame, the previous initial image frame adjacent to the initial image frame and the motion vector between the initial image frame and the previous initial image frame are obtained, and the pixel value difference between the corresponding pixels of the initial image frame and the previous initial image frame is determined. Edge detection is performed on the object in the initial image frame to obtain a first detection result. Based on the motion vector, motion detection is performed on the object in the initial image frame to obtain a second detection result. Then, based on the pixel value difference, the first detection result and the second detection result, a foreground mask is determined. Based on the foreground mask and the initial image frame, a background mask is determined. Then, based on the first metadata and the foreground mask, a first image area in the initial image frame is determined. Based on the background mask and the first image area, a second image area in the initial image frame is determined. Finally, a first encoding strategy corresponding to the first image area in each initial image frame and a second encoding strategy corresponding to the second image area are determined. Based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain an encoded second video stream. Therefore, the foreground mask of the initial image frame is determined through the pixel value difference between the initial image frame and the previous initial image frame of the video stream, the motion detection result and the edge detection result, and based on the foreground mask, the background mask is determined, and based on the foreground mask and the background mask, the region of interest and the region of no interest of the initial image frame are determined, and the encoded video stream is obtained by performing regional encoding on the region of interest and the region of no interest of each initial image frame, thereby improving the reliability of the regional efficient video encoding method and improving the effect of video encoding.

[0087] Figure 4 A flowchart of a regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0088] like Figure 4 As shown, the method includes: Step 401: Acquire each initial image frame corresponding to the first video stream.

[0089] Step 402: determine first metadata corresponding to each initial image frame, wherein the first metadata is data used to describe a region of interest in the initial image frame.

[0090] Step 403, for each initial image frame, based on the first metadata, determine a first image area and a second image area in the initial image frame, wherein the first image area is the region of interest, and the second image area is other image areas in the initial image frame except the first image area.

[0091] The specific implementation forms of steps 401 to 403 may refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0092] Step 404: determine a first quantization parameter corresponding to a first image region in each initial image frame.

[0093] It should be noted that the quantization parameter may also be referred to as a "quantization step parameter" or "QP", etc., which is not limited in the present disclosure.

[0094] Among them, QP is the abbreviation of Quantizer Parameter (QP).

[0095] In the present disclosure, after determining the first image area in the initial image frame, the first quantization parameter corresponding to the first image area can be determined through a preset first image area parameter configuration interface based on the user's requirement for the image quality of the first image area. Since the user's requirement for the image quality of the first image area may be high, at this time, the first quantization parameter corresponding to the first image area can be lowered on the basis of the global quantization parameter, thereby enhancing the image quality of the first image area. The present disclosure does not limit this.

[0096] It should be noted that after determining the first image area in the initial image frame, the first image area can also be cropped to obtain a new image containing the first image area, and then the new image is used as input for the encoding stage, which is not limited in the present disclosure.

[0097] Step 405: Determine a first encoding strategy for a corresponding first image region based on the first quantization parameter.

[0098] In the present disclosure, after determining the first quantization parameter corresponding to the first image region in each initial image frame, a first encoding strategy for the corresponding first image region may be determined based on the first quantization parameter.

[0099] Step 406: Determine a second encoding strategy corresponding to the second image region in each initial image frame.

[0100] Step 407: Encode the initial image frame based on the first encoding strategy and the second encoding strategy to obtain an encoded second video stream.

[0101] The specific implementation forms of step 406 to step 407 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0102] In the disclosed embodiment, each initial image frame corresponding to the first video stream is first obtained, and the first metadata corresponding to each initial image frame is determined. For each initial image frame, the first image area and the second image area in the initial image frame are determined based on the first metadata. Then, the first quantization parameter corresponding to the first image area in each initial image frame is determined, and based on the first quantization parameter, the first encoding strategy of the corresponding first image area is determined. Then, the second encoding strategy corresponding to the second image area in each initial image frame is determined. Finally, based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain the encoded second video stream. Thus, the region of interest and the non-region of interest of the corresponding initial image frame are determined through the metadata of the region of interest corresponding to each initial image frame corresponding to the video stream, and the encoding strategy corresponding to the region of interest is determined based on the quantization parameter corresponding to the region of interest. Then, the encoding strategy of the non-region of interest is determined, and the video stream is encoded by region, thereby improving the effect and efficiency of video encoding on the basis of ensuring the image quality of the region of interest.

[0103] Figure 5 A flowchart of a regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure.

[0104] like Figure 5 As shown, the method includes: Step 501: Acquire each initial image frame corresponding to the first video stream.

[0105] Step 502: determine first metadata corresponding to each initial image frame, wherein the first metadata is data used to describe a region of interest in the initial image frame.

[0106] Step 503, for each initial image frame, based on the first metadata, determine a first image area and a second image area in the initial image frame, wherein the first image area is the region of interest, and the second image area is other image areas in the initial image frame except the first image area.

[0107] Step 504: determine a first encoding strategy corresponding to a first image region in each initial image frame.

[0108] The specific implementation forms of steps 501 to 504 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0109] Step 505: Obtain the image complexity of the second image region in each initial image frame.

[0110] It should be noted that different second image regions may have different corresponding image complexities, which is not limited in the present disclosure.

[0111] In the present disclosure, when determining the second encoding strategy for the second image area in each initial image frame, the image complexity of each second image area can be obtained first, which provides conditions for realizing layered high-compression encoding of the second image area and improving the accuracy of the determined second encoding strategy.

[0112] It should be noted that the method for obtaining the image complexity of the second image region in each initial image frame may be any method, for example, the image complexity may be determined by the grayscale distribution of the second image region, etc., which is not limited in the present disclosure.

[0113] Optionally, when determining the image complexity of the second image area, the grayscale histogram and the grayscale co-occurrence matrix of the second image area can be determined first, and then based on the grayscale histogram and the grayscale co-occurrence matrix, the occurrence frequency and spatial distribution of each grayscale level of the second image area can be determined respectively. Finally, based on the occurrence frequency and spatial distribution of each grayscale level, the image complexity of the second image area can be determined, thereby improving the accuracy and reliability of the determined image complexity of the second image area.

[0114] Step 506: Determine a complexity level corresponding to the image complexity based on a preset first complexity threshold and a second complexity threshold, wherein the second complexity threshold is greater than the first complexity threshold.

[0115] The first complexity threshold and the second complexity threshold may be complexity critical values ​​for determining the complexity level of the image complexity of the second image region. The first complexity threshold and the second complexity threshold may be preset through experiments, and the present disclosure does not limit this.

[0116] For example, the complexity levels are the first level, the second level, and the third level, wherein the first level is the lowest, the second level is the second, and the third level is the highest. When the image complexity of the second image area is less than the first complexity threshold, the complexity level corresponding to the image complexity can be determined to be the first level, when the image complexity is greater than or equal to the first complexity threshold and less than the second complexity threshold, the complexity level corresponding to the image complexity can be determined to be the second level, and when the image complexity is greater than or equal to the second complexity threshold, the complexity level corresponding to the image complexity can be determined to be the third level, and the present disclosure does not limit this.

[0117] Step 507: Based on the complexity level, determine a filtering parameter and a second quantization parameter corresponding to the complexity level, wherein the second quantization parameter is greater than the first quantization parameter.

[0118] The filtering parameter may be a low-pass filtering parameter used to eliminate redundant details and complex information in the second image region, which may reduce the image complexity of the second image region.

[0119] It should be noted that, the determined filtering parameters and the second quantization parameters may be different for different complexity levels, and the present disclosure does not limit this.

[0120] In the present disclosure, since the user's requirement for the image quality of the second image area may be lower than that of the first image area, in order to reduce the amount of data in the second image area, the encoding compression rate of the second image area can be improved by adopting a second quantization parameter that is larger than the first quantization parameter, thereby achieving flexible and efficient encoding of the second image area.

[0121] It should be noted that the quantization parameter is inversely proportional to the image quality and directly proportional to the coding compression rate. For example, when the image quality is low or the coding compression rate is high, a larger quantization parameter can be used, and when the image quality is high or the coding compression rate is low, a smaller quantization parameter can be used, and the present disclosure does not limit this.

[0122] It should be noted that the first quantization parameter and the second quantization parameter can be determined by periodically detecting the content of the image frame through the preset multimedia processing framework Gstreamer, thereby realizing region-based quantization coding, which is not limited in the present disclosure.

[0123] Step 508: Determine a second encoding strategy for the corresponding second image region based on the filtering parameter and the second quantization parameter.

[0124] It should be noted that, if the filtering parameters and the second quantization parameters are different, the determined second encoding strategy may be different, and the present disclosure does not limit this.

[0125] In the present disclosure, after determining the filtering parameters and the second quantization parameters corresponding to the second image area in the initial image frame, the second encoding strategy corresponding to the second image area can be determined based on the filtering parameters and the second quantization parameters to achieve high-compression encoding of the second image area and improve the flexibility of encoding the second image area.

[0126] Step 509: Encode the initial image frame based on the first encoding strategy and the second encoding strategy to obtain an encoded second video stream.

[0127] The specific implementation of step 509 can refer to the detailed description in other embodiments of the present disclosure, and will not be described in detail here.

[0128] In the present disclosure, after obtaining the encoded second video stream, the effect of the regional efficient video encoding method for low-bitrate video transmission in mines provided by the present disclosure can also be analyzed. First, the second video stream is decoded and filtered and optimized to eliminate the artifact problem caused by regional encoding, and then the reconstructed image frame f(i) is obtained. Then the image frame f(i) is compared with the initial image frame F(i) of the corresponding first video stream. By comparing the peak signal-to-noise ratio or the structural similarity measurement distortion index, the distortion rate of the video encoding method provided by the present disclosure can be obtained. The encoding compression rate of the encoding method provided by the present disclosure is determined by the ratio of the data amount corresponding to the first video stream to the data amount corresponding to the second video stream. The rate-distortion curve of the encoding method provided by the present disclosure is determined based on the distortion rate and the encoding compression rate, and finally compared with the rate-distortion curve of the general encoding method. Figure 6 As shown, Figure 6 A schematic diagram of the effect of the regional efficient video encoding method for low-bitrate video transmission in mines provided by an embodiment of the present disclosure. Figure 6 In the example, taking the peak signal-to-noise ratio as the distortion index, curve 1 is the rate-distortion curve of the regional efficient video coding method for low-bitrate video transmission in mines provided by the present disclosure, and curve 2 is the rate-distortion curve of the general coding method. Figure 6 It can be seen that, at the same bit rate, compared with the general encoding method, the reconstructed video based on the encoding method provided by the present disclosure has higher image quality and better effect, wherein, Figure 6 This is just an example and the present disclosure is not limited to this.

[0129] It should be noted that in order to facilitate the analysis of the effects before and after video encoding, a bypass output can be added before the system outputs the second video stream to lead out the video data of the first video stream to record the size and image quality of the video itself. This disclosure does not limit this.

[0130] In the disclosed embodiment, each initial image frame corresponding to the first video stream is first obtained, and the first metadata corresponding to each initial image frame is determined. Then, for each initial image frame, based on the first metadata, the first image area and the second image area in the initial image frame are determined, and the first encoding strategy corresponding to the first image area in each initial image frame is determined. Then, the image complexity of the second image area in each initial image frame is obtained, and based on a preset first complexity threshold and a second complexity threshold, the complexity level corresponding to the image complexity is determined. Based on the complexity level, the filtering parameters and the second quantization parameters corresponding to the complexity level are determined. Based on the filtering parameters and the second quantization parameters, the second encoding strategy is determined. Finally, based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain the encoded second video stream. Therefore, by determining the region of interest and the region of non-interest of the corresponding initial image frame based on the region of interest metadata of each initial image frame corresponding to the video stream, determining the encoding strategy of the region of interest, and then determining the corresponding filtering parameters and quantization parameters based on the image complexity of the region of non-interest, and determining the encoding strategy of the region of non-interest based on the filtering parameters and the quantization parameters, the video stream is encoded, thereby effectively solving the contradiction between the encoding compression ratio and the image quality on the basis of ensuring the image quality of the region of interest, reducing resource overhead, and improving the flexibility and effect of video encoding.

[0131] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0132] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0133] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0134] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0135] It should be understood that the various parts of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0136] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0137] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0138] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0139] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0140] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A regional efficient video encoding method for low bit rate video transmission in mines, comprising: Obtain each initial image frame corresponding to the first video stream; Determine first metadata corresponding to each of the initial image frames, wherein the first metadata is data used to describe a region of interest in the initial image frame; For each initial image frame, determining a first image region and a second image region in the initial image frame based on the first metadata, wherein the first image region is a region of interest, and the second image region is other image regions in the initial image frame except the first image region; Determining a first encoding strategy corresponding to a first image region in each of the initial image frames, and a second encoding strategy corresponding to a second image region; Based on the first encoding strategy and the second encoding strategy, the initial image frame is encoded to obtain an encoded second video stream.

2. The method of claim 1, wherein: The determining, based on the first metadata, a first image region and a second image region in the initial image frame includes: Obtaining a foreground mask and a background mask corresponding to the initial image frame; Determining a first image region in the initial image frame based on the first metadata and the foreground mask; Based on the background mask and the first image region, a second image region in the initial image frame is determined.

3. The method of claim 2, wherein: The obtaining of a foreground mask and a background mask corresponding to the initial image frame includes: Acquire a previous initial image frame adjacent to the initial image frame, and a motion vector between the initial image frame and the previous initial image frame; Determine a pixel value difference between corresponding pixels of the initial image frame and the previous initial image frame; Performing edge detection on the object in the initial image frame to obtain a first detection result; Based on the motion vector, performing motion detection on the object in the initial image frame to obtain a second detection result; Determining the foreground mask based on the pixel value difference, the first detection result, and the second detection result; The background mask is determined based on the foreground mask and the initial image frame.

4. The method of claim 1, wherein: The process of determining the first encoding strategy includes: determining a first quantization parameter corresponding to the first image area; Based on the first quantization parameter, the first encoding strategy is determined.

5. The method of claim 1, wherein: The process of determining the second encoding strategy includes: Acquire the image complexity of the second image area; Determining a complexity level corresponding to the image complexity based on a preset first complexity threshold and a second complexity threshold, wherein the second complexity threshold is greater than the first complexity threshold; Based on the complexity level, determining a filtering parameter and a second quantization parameter corresponding to the complexity level, wherein the second quantization parameter is greater than the first quantization parameter; The second encoding strategy is determined based on the filtering parameter and the second quantization parameter.

6. The method of claim 5, wherein: The obtaining the image complexity of the second image area includes: Determining a grayscale histogram and a grayscale co-occurrence matrix of the second image region; Based on the grayscale histogram and the grayscale co-occurrence matrix, respectively determining the occurrence frequency and spatial distribution of each grayscale level of the second image area; Based on the occurrence frequency and spatial distribution of the gray levels, the image complexity of the second image region is determined.

Images