Video network transmission method and device, video monitoring system, and storage medium

Abstract

The present invention provides a video network transmission method and device, a video monitoring system, and a storage medium. The method includes: dividing the image to be transmitted in the video into coding macroblocks, and for each divided macroblock, determining whether it is a target macroblock according to whether its motion vector reaches a predetermined level; for each determined target macroblock, respectively composing it with adjacent macroblocks into a macroblock set, and determining whether the macroblock set is a target macroblock set according to whether the mean value of the motion vector of the macroblock in the macroblock set reaches a predetermined level; for each obtained target macroblock set, obtaining the corresponding target area in the image; using each obtained target area as a foreground area, and using the remaining area in the image except the target area as a background area; encoding and compressing the foreground area and the background area respectively according to the bandwidth of the network; and transmitting the data obtained by encoding and compressing the image through the network. The present invention is conducive to achieving high-quality network monitoring.

Description

Technical Field

[0001] The present invention relates to the field of video compression, and in particular to a video network transmission method and device, a video monitoring system, and a storage medium. Background Art

[0002] At present, video surveillance systems have spread all over the world. In recent years, some new video surveillance products have emerged, such as video doorbells, elevator monitoring, pet monitoring, etc. The cameras of these monitoring products will generate a large amount of video data, but these video data are usually not used immediately, but need to be stored for future use and viewing. Due to the large amount of video data generated, the network bandwidth is also tight in some cases, so it is usually necessary to process the transmitted images and videos accordingly. The main factors that need to be considered at present include: the time to save the video, the quality of the video, and the storage cost. Among them, in order to save storage costs, the main method currently used is to compress the stored video to obtain low-quality video, but these low-quality videos often result in the loss of important details.

[0003] The patent document with publication number CN101945287A provides a region of interest ROI encoding method, including: a video surveillance codec system obtains the foreground area and background area selected by the user in the image display window, and the foreground area image quality level selected by the user; the video surveillance codec system determines the bit rate allocation ratio of the foreground area and the background area according to the foreground area image quality level, and allocates the bit rate of the foreground area and the background area according to the bit rate allocation ratio; the video surveillance codec system performs FMO encoding on the foreground area and the background area respectively according to the bit rates of the foreground area and the background area, so that the clarity of the foreground area can be higher than that of the background area, and the display clarity of the foreground and background can be flexibly set as needed. However, this prior art requires more manual operations by the user, such as the user not only needs to select the foreground area in the image display window, but also needs to select the foreground area image quality level, which makes it difficult to achieve high-quality network monitoring. Summary of the invention

[0004] Based on the above situation, the main purpose of the present invention is to provide a video network transmission method and device, a video monitoring system, and a storage medium, which are conducive to achieving high-quality network monitoring.

[0005] To achieve the above object, the technical solution of the present invention provides a video network transmission method, comprising:

[0006] Step 100: Divide the image to be transmitted in the video into coding macroblocks, and determine whether each divided macroblock is a target macroblock according to whether its motion vector reaches a predetermined level;

[0007] Step 200: for each determined target macroblock, respectively form a macroblock set with adjacent macroblocks, and determine whether the macroblock set is a target macroblock set according to whether the mean value of the motion vectors of the macroblocks in the macroblock set reaches a predetermined level;

[0008] Step 300: for each target macroblock set obtained, obtaining the target area corresponding to the target macroblock set in the image, wherein the target area corresponding to each target macroblock set is a rectangular area obtained by expanding outwards by a preset value with the target macroblock set as the center;

[0009] Step 400: taking each obtained target area as a foreground area, and taking the remaining area in the image except the target area as a background area;

[0010] Step 500: Obtain the bandwidth of the network currently transmitting the video, and perform encoding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to achieve encoding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the encoding compression degree of the foreground area is reduced to increase the amount of data obtained by encoding and compressing the image, and if the bandwidth of the network decreases from the first value to a third value, the encoding compression degree of the background area is increased to reduce the amount of data obtained by encoding and compressing the image;

[0011] Step 600: Transmitting the data obtained by encoding and compressing the image through the network.

[0012] Furthermore, in step 100, for each divided macroblock, determining whether it is a target macroblock is determined according to whether its motion vector reaches a predetermined level, including:

[0013] Step 101: for each macroblock divided from the image, determine whether there is a sub-macroblock divided therein, and if so, select the sub-macroblock divided therein, wherein the sub-macroblock is a sub-block further divided from the macroblock;

[0014] Step 102: averaging the horizontal motion vectors and vertical motion vectors of all selected sub-macroblocks to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY;

[0015] Step 103: For each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, then the macroblock is determined to be a target macroblock.

[0016] Further, step 200 includes:

[0017] For each determined target macroblock, it is respectively combined with the four adjacent macroblocks above, below, left and right to form a macroblock set, and the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set are averaged to obtain the horizontal motion vector mean VDX and the vertical motion vector mean VDY. If the horizontal motion vector mean VDX is greater than the horizontal motion vector mean VAX, or the vertical motion vector mean VDY is greater than the vertical motion vector mean VAY, then the macroblock set is determined to be the target macroblock set.

[0018] Further, step 300 includes:

[0019] For each obtained target macroblock set, the set is taken as the center and expanded upward, downward, leftward and rightward by a number of macroblocks respectively, so as to obtain the corresponding target area in the image.

[0020] Further, step 500 includes:

[0021] If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed using a coding compression coefficient n1, and the background area is coded and compressed using a coding compression coefficient m1, wherein n1 is less than m1;

[0022] If the bandwidth of the network is within the second interval [K2, K1), the foreground area is coded and compressed using a coding compression coefficient n2, and the background area is coded and compressed using a coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1;

[0023] If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2;

[0024] If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is deleted;

[0025] The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

[0026] Furthermore, the encoding compression method includes H.264 encoding method;

[0027] The K1 is any value between 500 kbps and 520 kbps, the K2 is any value between 390 kbps and 410 kbps, and the K3 is any value between 190 kbps and 210 kbps.

[0028] To achieve the above object, the technical solution of the present invention further provides a video network transmission device, comprising:

[0029] The macroblock processing module is used to divide the image to be transmitted in the video into coding macroblocks, and determine whether each divided macroblock is a target macroblock according to whether its motion vector reaches a predetermined level;

[0030] A macroblock set processing module, for respectively combining each determined target macroblock with adjacent macroblocks to form a macroblock set, and determining whether the macroblock set is a target macroblock set according to whether the mean value of the motion vectors of the macroblocks in the macroblock set reaches a predetermined level;

[0031] A target region processing module is used to obtain the target region corresponding to each target macroblock set in the image, wherein the target region corresponding to each target macroblock set is a rectangular region obtained by expanding outwards by a preset value with the target macroblock set as the center;

[0032] A partitioning module, used to use each obtained target area as a foreground area, and use the remaining area of ​​the image except the target area as a background area;

[0033] A coding compression module, used for obtaining the bandwidth of the network currently transmitting the video, and performing coding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to realize coding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the coding compression degree of the foreground area is reduced to increase the amount of data obtained by coding compression of the image, and if the bandwidth of the network decreases from the first value to a third value, the coding compression degree of the background area is increased to reduce the amount of data obtained by coding compression of the image;

[0034] The transmission module is used to transmit the data obtained by encoding and compressing the image through the network.

[0035] Furthermore, the macroblock processing module includes:

[0036] A selection unit is used to determine whether there is a sub-macroblock divided from each macroblock divided from the image, and if so, select the sub-macroblock divided from the macroblock, wherein the sub-macroblock is a sub-block further divided from the macroblock;

[0037] A calculation unit, used for averaging the horizontal motion vectors and vertical motion vectors of all selected sub-macroblocks to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY;

[0038] The judgment unit is used to determine that for each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, the macroblock is a target macroblock.

[0039] Further, the macroblock set processing module is configured to: for each determined target macroblock, form a macroblock set with the four adjacent macroblocks above, below, left and right, and average the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set to obtain a horizontal motion vector mean VDX and a vertical motion vector mean VDY; if the horizontal motion vector mean VDX is greater than the horizontal motion vector mean VAX, or the vertical motion vector mean VDY is greater than the vertical motion vector mean VAY, then the macroblock set is determined to be a target macroblock set.

[0040] Furthermore, the target area processing module is configured to:

[0041] For each obtained target macroblock set, the set is taken as the center and expanded upward, downward, leftward and rightward by a number of macroblocks respectively, so as to obtain the corresponding target area in the image.

[0042] Furthermore, the encoding and compression module is configured to:

[0043] If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed using a coding compression coefficient n1, and the background area is coded and compressed using a coding compression coefficient m1, wherein n1 is less than m1;

[0044] If the bandwidth of the network is within the second interval [K2, K1), the foreground area is coded and compressed using a coding compression coefficient n2, and the background area is coded and compressed using a coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1;

[0045] If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2;

[0046] If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is deleted;

[0047] The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

[0048] Furthermore, the encoding compression method includes H.264 encoding method;

[0049] The K1 is any value between 500 kbps and 520 kbps, the K2 is any value between 390 kbps and 410 kbps, and the K3 is any value between 190 kbps and 210 kbps.

[0050] To achieve the above-mentioned purpose, the technical solution of the present invention also provides a video network transmission device, comprising: a processor and a memory coupled to the processor, wherein the memory stores instructions for execution by the processor, and when the processor executes the instructions, any one of the above-mentioned video network transmission methods can be implemented.

[0051] To achieve the above-mentioned purpose, the technical solution of the present invention further provides a video monitoring system, comprising any one of the above-mentioned video network transmission devices.

[0052] To achieve the above objectives, the technical solution of the present invention further provides a computer-readable storage medium storing a computer program, which can implement any of the above-mentioned video network transmission methods when executed by a processor.

[0053] The video network transmission method provided by the present invention obtains the central position of the region of interest to the human eye by using the macroblocks of interest to the human eye and the adjacent macroblocks in the image, and then obtains the region of interest to the human eye in the image by expanding outwards, so as to realize accurate and efficient division of the region of interest and the region of no interest in the image, and when the bandwidth of the network becomes larger, the amount of data obtained by encoding and compressing the image is mainly increased by increasing the bits of the foreground region, and when the bandwidth of the network becomes smaller, the amount of data obtained by encoding and compressing the image is mainly reduced by reducing the bits of the background region, so that the quality of the region of interest to the human eye can be improved as much as possible while making the encoding adapt to the network state, which can not only realize the maximum utilization of the transmission bandwidth resources in network monitoring, but also enable the important video details to be preserved, which is conducive to realizing high-quality network monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0055] Figure 1is a flow chart of a video network transmission method provided by an embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a macroblock set provided by an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of a region of interest of a human eye provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. In order to avoid confusing the essence of the present invention, known methods, processes, procedures, and components are not described in detail.

[0059] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0060] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include", "comprising" and similar words should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

[0061] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0062] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is only for the purpose of convenience and brevity of description, and is by no means intended to limit the order of these method steps. Those skilled in the art will appreciate that the order of the relevant method steps should be determined by the technology itself and should not be inappropriately limited by the existence of step numbers.

[0063] See also Figure 1 , Figure 1 1 is a flow chart of a video network transmission method provided by an embodiment of the present invention, the video network transmission method comprising:

[0064] Step 100: Divide the image to be transmitted in the video into coding macroblocks, and determine whether each divided macroblock is a target macroblock according to whether its motion vector reaches a predetermined level, and the target macroblock is a macroblock of interest to the human eye;

[0065] The above-mentioned coding macroblock division includes the division of macroblocks and whether each divided macroblock is further divided into sub-macroblocks. Specifically, an existing coding method can be used. For example, the existing H.264 coding method can be used to divide the image into coding macroblocks. The size of the divided macroblocks is 16x16 pixels, and the sub-macroblocks may include: 16x8 pixels, 8x16 pixels, 8x8 pixels, 8x4 pixels, 4x8 pixels, and 4x4 pixels.

[0066] Generally speaking, a macroblock with a larger motion vector is a location with richer details. Therefore, it is possible to determine whether a macroblock is a macroblock of interest to the human eye based on the size of the macroblock's motion vector. That is, the larger the motion vector of the macroblock is, the higher the degree of interest to the human eye. Therefore, the motion vector of the macroblock can be compared with a predetermined level, and the macroblock whose motion vector reaches the predetermined level can be regarded as a macroblock of interest to the human eye. For example, if the horizontal motion vector of the macroblock is greater than a first threshold, or the vertical motion vector of the macroblock is greater than a second threshold, it can be determined that the macroblock is a macroblock of interest to the human eye. The predetermined level can be determined based on the division of the coded macroblocks of the image (such as the mean value of the motion vectors of all sub-macroblocks), or can be determined in other ways.

[0067] The above-mentioned motion vector may include a horizontal motion vector and / or a vertical motion vector;

[0068] Among them, for each macroblock, if it has sub-macroblocks divided into it, its motion vector is the average of the motion vectors of its divided sub-macroblocks, that is, the horizontal motion vector of the macroblock is the average of the horizontal motion vectors of its divided sub-macroblocks, and the vertical motion vector of the macroblock is the average of the vertical motion vectors of its divided sub-macroblocks.

[0069] Step 200: for each determined target macroblock, respectively, form a macroblock set with the adjacent macroblocks, and determine whether the macroblock set is a target macroblock set according to whether the mean value of the motion vector of the macroblocks in the macroblock set reaches a predetermined level. The target macroblock set is a macroblock set of interest to human eyes. Specifically, the macroblock set of a macroblock may be composed of the macroblock and four adjacent macroblocks above, below, left, and right of the macroblock (a total of five macroblocks);

[0070] For example, see Figure 2 , for macroblock No. 0, the area where its macroblock set is located is a diamond-shaped area formed by five macroblocks, namely, macroblock No. 0, macroblock No. 1, macroblock No. 2, macroblock No. 3, and macroblock No. 4;

[0071] In this embodiment, the target macroblock and adjacent macroblocks form a macroblock set, and the mean value of the motion vectors of the macroblocks in the macroblock set can reflect the overall human eye interest level of the region, thereby efficiently determining the human eye interest region in the image;

[0072] Step 300: for each target macroblock set obtained, obtaining the target area corresponding to it in the image, wherein the target area corresponding to each target macroblock set is a rectangular area obtained by expanding outward by a preset value with the target macroblock set as the center, and the target area is an area of ​​interest to the human eye;

[0073] After obtaining the macroblock set of the human eye interest area, the corresponding human eye interest area can be determined by rectangular frame selection with the area where the macroblock set is located as the center, so that the human eye interest area in the image can be efficiently obtained; for example, for each obtained target macroblock set, it is taken as the center and a number of macroblocks are respectively extended upward, downward, left and right, so as to obtain its corresponding target area in the image;

[0074] Specifically, for each macroblock set of human eye interest area, the corresponding human eye interest area is a rectangular area consisting of the same number of macroblocks extending upward, downward, left and right, with the area where the macroblock set is located as the center; for example, Figure 2 The macroblock set shown is shown in Figure 3 , the corresponding personnel interest area is a rectangular area (boxed area) consisting of four macroblocks extending upward, downward, left and right respectively, with the diamond-shaped area where the macroblock set is located as the center;

[0075] Step 400: taking each obtained target area as a foreground area, and taking the remaining area in the image except the target area as a background area;

[0076] After obtaining a number of human eye regions of interest in step 300, the remaining area of ​​the image except the human eye regions of interest can be used as a background area (i.e., a non-interest area), thereby dividing the image into two parts, and performing different degrees of encoding processing on each part independently;

[0077] Step 500: Obtain the bandwidth of the network currently transmitting the video, and perform encoding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to achieve encoding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the encoding compression degree of the foreground area is reduced to increase the amount of data obtained by encoding and compressing the image, and if the bandwidth of the network decreases from the first value to a third value, the encoding compression degree of the background area is increased to reduce the amount of data obtained by encoding and compressing the image;

[0078] It can be understood that the greater the coding compression degree, the smaller the amount of data after coding compression, and the worse the image quality obtained; the smaller the coding compression degree, the larger the amount of data after coding compression, and the better the image quality obtained; in specific implementation, the coding compression degree can be quantified by using the coding compression coefficient, and the two are positively correlated;

[0079] That is, in the present invention, the FMO (Flexible Macroblock Ordering) Type 2 scheme can be adopted, and the transmission is divided according to the rectangular area. According to the bandwidth of the current network, the area of ​​interest and the area of ​​non-interest are independently coded to different degrees, and the two parts are independent of each other and then transmitted. Among them, the area of ​​interest to the human eye is coded, and its quality can be higher than the rest of the picture;

[0080] Specifically, if the bandwidth of the network increases from the first value to the second value, the amount of data obtained by encoding and compressing the image is increased by reducing the degree of encoding compression of the foreground area; if the bandwidth of the network decreases from the first value to the third value, the amount of data obtained by encoding and compressing the image is reduced by increasing the degree of encoding compression of the background area; that is, when the bandwidth of the network becomes larger, the amount of data obtained by encoding and compressing the image is mainly increased by increasing the bits of the foreground area; when the bandwidth of the network becomes smaller, the amount of data obtained by encoding and compressing the image is mainly reduced by reducing the bits of the background area. In this way, while adapting the encoding to the network status, the quality of the area of ​​interest to the human eye can be improved as much as possible, which can not only achieve maximum utilization of transmission bandwidth resources, but also preserve important video details;

[0081] Step 600: Transmitting the data obtained by encoding and compressing the image through the network;

[0082] For example, video data obtained by encoding and compressing an image can be transmitted via a network to a storage device for storage so as to be viewed later.

[0083] The video network transmission method provided by the embodiment of the present invention uses the macroblocks of interest to the human eye in the image and its adjacent macroblocks to obtain the central position of the region of interest to the human eye, and then obtains the region of interest to the human eye in the image by expanding outward, so as to realize accurate and efficient division of the region of interest and the region of no interest in the image, and when the bandwidth of the network becomes larger, the amount of data obtained by encoding and compressing the image is mainly increased by increasing the bits of the foreground area, and when the bandwidth of the network becomes smaller, the amount of data obtained by encoding and compressing the image is mainly reduced by reducing the bits of the background area. In this way, while making the encoding adapt to the network status, the quality of the region of interest to the human eye can be improved as much as possible, which can not only realize the maximum utilization of the transmission bandwidth resources in network monitoring, but also enable the important video details to be preserved, which is conducive to realizing high-quality network monitoring.

[0084] Preferably, in one embodiment, in the above step 100, for each divided macroblock, determining whether it is a target macroblock according to whether its motion vector reaches a predetermined level includes:

[0085] Step 101: for each macroblock divided from the image, determine whether there is a sub-macroblock divided therein, and if so, select the sub-macroblock divided therein, wherein the sub-macroblock is a sub-block further divided from the macroblock;

[0086] Step 102: averaging the horizontal motion vectors and vertical motion vectors of all selected sub-macroblocks to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY;

[0087] Step 103: For each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, then the macroblock is determined to be a target macroblock.

[0088] Generally speaking, the segmented sub-macroblocks are usually locations with richer details, that is, locations with larger motion vectors. This embodiment uses the average value of the motion vectors of the sub-macroblocks to obtain a corresponding predetermined level, and can accurately obtain macroblocks that are of interest to the human eye.

[0089] For example, when a macroblock encoding division of a frame of an image in a video is completed, it is determined whether the current macroblock has divided sub-macroblocks. If so, the horizontal components and vertical components of the motion vectors of these sub-macroblocks are accumulated respectively, and the number of sub-macroblocks is calculated. After the image is divided into encoding macroblocks, the mean value VAX of the horizontal motion vector and the mean value VAY of the vertical motion vector are calculated. The mean vectors in these two directions are used as the basis for determining the important area of ​​interest and the unimportant area of ​​interest. It is determined whether the horizontal motion vector of each macroblock is greater than the mean value VAX of the horizontal motion vector, and whether the vertical motion vector is greater than the mean value VAY of the vertical motion vector. If any of the above items is satisfied (the horizontal motion vector of the macroblock is greater than the mean value VAX of the horizontal motion vector, or the vertical motion vector is greater than the mean value VAY of the vertical motion vector), it indicates that the macroblock is a macroblock of interest to the human eye. Then, with this macroblock as the center, the macroblocks surrounding the macroblock (i.e., the adjacent macroblocks) are found, and the mean values ​​of the motion vectors of these macroblocks are counted, including the mean value VDX of the horizontal motion vector and the mean value VDY of the vertical motion vector.

[0090] Preferably, the above step 200 may include:

[0091] For each determined target macroblock, it is respectively formed into a macroblock set with the four adjacent macroblocks above, below, left and right, and the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set are averaged to obtain the horizontal motion vector mean VDX and the vertical motion vector mean VDY. If the horizontal motion vector mean VDX is greater than the horizontal motion vector mean VAX, or the vertical motion vector mean VDY is greater than the vertical motion vector mean VAY, then the macroblock set is determined to be the target macroblock set, that is, the macroblock set of the area of ​​interest to the human eye.

[0092] For example, in one embodiment of the present invention, step 500 may include:

[0093] If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed according to the coding compression coefficient n1, and the background area is coded and compressed using the coding compression coefficient m1, wherein n1 is less than m1;

[0094] If the bandwidth of the network is within the second interval range [K2, K1), the foreground area is coded and compressed according to the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1;

[0095] If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed according to the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2;

[0096] If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed according to the coding compression coefficient n2, and the background area is deleted, that is, the non-interested area can be directly removed, and only the important area (foreground area) in the image is transmitted;

[0097] The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

[0098] For example, in one embodiment of the present invention, the above-mentioned encoding compression method includes H.264 encoding method;

[0099] Among them, the K1 is any value between 500kbps and 520kbps, such as 512kbps, the K2 is any value between 390kbps and 410kbps, such as 400kbps, and the K3 is any value between 190kbps and 210kbps, such as 200kbps.

[0100] In an embodiment of the present invention, the network bandwidth can be divided into multiple levels, and by adopting the FMO (Flexible Macroblock Ordering) Type 2 scheme, the data volume distribution ratio of the interested part and the non-interested part can be independently controlled, so that the data volume bit of the interested area accounts for a high proportion of the entire image, and the data volume bit of the non-interested area accounts for a small proportion of the entire image.

[0101] For example, when the network bandwidth is sufficient, such as when the network bandwidth data is in the preset first interval range [512kbps, +∞), the video stream with normal bit rate can be directly transmitted; when the network bandwidth fluctuates or is restricted, such as when the network bandwidth data is in the preset second interval range [400kbps, 512kbps), the foreground area is encoded and compressed by the encoding compression coefficient n2, and the background area is encoded and compressed by the encoding compression coefficient m2; when the network bandwidth data is in the preset third interval range [200kbps, 400kbps), the foreground area is encoded and compressed by the encoding compression coefficient n2, and the background area is encoded and compressed by the encoding compression coefficient m3, thereby reducing the image quality of the non-interest area; when the network status is very poor, such as when the network bandwidth data is in the preset fourth interval range [0, 200kbps), the non-interest area can be directly removed, and only the important area (foreground area) in the image is transmitted.

[0102] The video network transmission method provided by the embodiment of the present invention introduces the motion vector feature of the current encoded macroblock to divide the region of interest. According to the characteristic that the human eye usually has a higher visual perception of moving objects or regions with rich textures, that is, macroblock segmentation inter-frame prediction modes (16x8, 8x16, 8x8, 8x4, 4x8, 4x4) are mostly used in these regions, and these regions generally have larger motion vectors. The motion vector represents the offset of the moving object between the current frame and the reference frame. From the perspective of motion vector distribution, the region with a larger motion vector happens to be a region of rich detail and of interest to the user, while the region with a smaller or even zero motion vector is generally a background region of no interest.

[0103] The video network transmission method provided by the present invention transmits images of regions of interest based on flexible macroblock sorting, detects the regions of interest of the video in real time and improves the image quality of the regions of interest, compresses the regions of non-interest to reduce the image quality, and transmits images using the flexible macroblock sorting function, so as to ensure that the image details of important regions are retained in the case of insufficient network bandwidth or saving storage costs. For example, in the case of limited bandwidth, more bits can be allocated to the regions of interest during encoding to improve the image quality of this part, perform deep video compression on the background regions of non-interest, and use flexible macroblock sorting (FMO) type 2 to independently encode and transmit the regions of interest and non-interest, so as to effectively utilize the transmission bandwidth resources, preserve the important video details, and adjust the image and video quality in real time to meet the needs of users with different network bandwidths.

[0104] The embodiment of the present invention further provides a video network transmission device, including:

[0105] The macroblock processing module is used to divide the image to be transmitted in the video into coding macroblocks, and determine whether each divided macroblock is a target macroblock according to whether its motion vector reaches a predetermined level;

[0106] A macroblock set processing module, for respectively combining each determined target macroblock with adjacent macroblocks to form a macroblock set, and determining whether the macroblock set is a target macroblock set according to whether the mean value of the motion vectors of the macroblocks in the macroblock set reaches a predetermined level;

[0107] A target region processing module is used to obtain the target region corresponding to each target macroblock set in the image, wherein the target region corresponding to each target macroblock set is a rectangular region obtained by expanding outwards by a preset value with the target macroblock set as the center;

[0108] A partitioning module, used to use each obtained target area as a foreground area, and use the remaining area of ​​the image except the target area as a background area;

[0109] A coding compression module, used for obtaining the bandwidth of the network currently transmitting the video, and performing coding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to realize coding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the coding compression degree of the foreground area is reduced to increase the amount of data obtained by coding compression of the image, and if the bandwidth of the network decreases from the first value to a third value, the coding compression degree of the background area is increased to reduce the amount of data obtained by coding compression of the image;

[0110] The transmission module is used to transmit the data obtained by encoding and compressing the image through the network.

[0111] In one embodiment of the present invention, the macroblock processing module includes:

[0112] A selection unit is used to determine whether there is a sub-macroblock divided from each macroblock divided from the image, and if so, select the sub-macroblock divided from the macroblock, wherein the sub-macroblock is a sub-block further divided from the macroblock;

[0113] A calculation unit, used for averaging the horizontal motion vectors and vertical motion vectors of all selected sub-macroblocks to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY;

[0114] The judgment unit is used to determine that for each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, the macroblock is a target macroblock.

[0115] In one embodiment of the present invention, the macroblock set processing module is configured to: for each determined target macroblock, form a macroblock set with the four adjacent macroblocks above, below, left and right, and average the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set to obtain a horizontal motion vector mean VDX and a vertical motion vector mean VDY. If the horizontal motion vector mean VDX is greater than the horizontal motion vector mean VAX, or the vertical motion vector mean VDY is greater than the vertical motion vector mean VAY, the macroblock set is determined to be a target macroblock set.

[0116] In one embodiment of the present invention, the target area processing module is configured to:

[0117] For each obtained target macroblock set, the set is taken as the center and expanded upward, downward, leftward and rightward by a number of macroblocks respectively, so as to obtain the corresponding target area in the image.

[0118] In one embodiment of the present invention, the encoding compression module is configured to:

[0119] If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed using a coding compression coefficient n1, and the background area is coded and compressed using a coding compression coefficient m1, wherein n1 is less than m1;

[0120] If the bandwidth of the network is within the second interval [K2, K1), the foreground area is coded and compressed using a coding compression coefficient n2, and the background area is coded and compressed using a coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1;

[0121] If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2;

[0122] If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is deleted;

[0123] The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

[0124] In one embodiment of the present invention, the encoding compression method includes H.264 encoding method;

[0125] The K1 is any value between 500 kbps and 520 kbps, the K2 is any value between 390 kbps and 410 kbps, and the K3 is any value between 190 kbps and 210 kbps.

[0126] An embodiment of the present invention also provides a video network transmission device, comprising: a processor and a memory coupled to the processor, wherein the memory stores instructions for execution by the processor, and when the processor executes the instructions, the above-mentioned video network transmission method can be implemented.

[0127] The embodiment of the present invention further provides a video monitoring system, comprising the above-mentioned video network transmission device.

[0128] The embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned video network transmission method when executed by a processor.

[0129] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0130] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A video network transmission method, characterized in that: include: Step 100: dividing the image to be transmitted in the video into coding macroblocks, and determining whether each divided macroblock is a target macroblock according to whether its motion vector reaches a predetermined level, including: Step 101: for each macroblock divided from the image, determine whether there is a sub-macroblock divided therein, and if so, select the sub-macroblock divided therein, wherein the sub-macroblock is a sub-block further divided from the macroblock; Step 102: average the horizontal motion vectors and vertical motion vectors of all the selected sub-macroblocks respectively to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY; Step 103: for each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, determine that the macroblock is a target macroblock; Step 200: for each determined target macroblock, respectively combine it with adjacent macroblocks to form a macroblock set, and determine whether the macroblock set is a target macroblock set according to whether the mean value of the motion vector of the macroblock in the macroblock set reaches a predetermined level, including: for each determined target macroblock, respectively combine it with four adjacent macroblocks above, below, left and right to form a macroblock set, and average the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set to obtain a horizontal motion vector mean value VDX and a vertical motion vector mean value VDY; if the horizontal motion vector mean value VDX is greater than the horizontal motion vector mean value VAX, or the vertical motion vector mean value VDY is greater than the vertical motion vector mean value VAY, then determine that the macroblock set is a target macroblock set; Step 300: for each obtained target macroblock set, obtaining the target area corresponding to the target macroblock set in the image, wherein the target area corresponding to each target macroblock set is a rectangular area obtained by expanding a preset value outward with the target macroblock set as the center, including: for each obtained target macroblock set, taking the target macroblock set as the center and expanding a plurality of macroblocks upward, downward, leftward, and rightward respectively, so as to obtain the target area corresponding to the target macroblock set in the image; Step 400: taking each obtained target area as a foreground area, and taking the remaining area in the image except the target area as a background area; Step 500: Obtain the bandwidth of the network currently transmitting the video, and perform encoding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to achieve encoding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the encoding compression degree of the foreground area is reduced to increase the amount of data obtained by encoding and compressing the image, and if the bandwidth of the network decreases from the first value to a third value, the encoding compression degree of the background area is increased to reduce the amount of data obtained by encoding and compressing the image; Step 600: Transmitting the data obtained by encoding and compressing the image through the network.

2. The method according to claim 1, characterized in that Step 500 includes: If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed using a coding compression coefficient n1, and the background area is coded and compressed using a coding compression coefficient m1, wherein n1 is smaller than m1; If the bandwidth of the network is within the second interval range [K2, K1), the foreground area is coded and compressed using a coding compression coefficient n2, and the background area is coded and compressed using a coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1; If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2; If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is deleted; The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

3. The method according to claim 2, characterized in that The encoding compression method includes H.264 encoding method; Wherein, the K1 is any value between 500 kbps and 520 kbps, the K2 is any value between 390 kbps and 410 kbps, and the K3 is any value between 190 kbps and 210 kbps.

4. A video network transmission device, characterized in that: include: A macroblock processing module is used to divide the image to be transmitted in the video into coding macroblocks, and for each divided macroblock, determine whether it is a target macroblock according to whether its motion vector reaches a predetermined level, the macroblock processing module includes: a selection unit, for each macroblock divided from the image, to determine whether there is a sub-macroblock, if so, to select the sub-macroblock, wherein the sub-macroblock is a sub-block further divided from the macroblock; a calculation unit, for averaging the horizontal motion vectors and vertical motion vectors of all the selected sub-macroblocks, respectively, to obtain a horizontal motion vector mean VAX and a vertical motion vector mean VAY; a judgment unit, for each macroblock divided from the image, if its horizontal motion vector is greater than the horizontal motion vector mean VAX, or its vertical motion vector is greater than the vertical motion vector mean VAY, then determine that the macroblock is a target macroblock; A macroblock set processing module, for each determined target macroblock, respectively, combines it with adjacent macroblocks to form a macroblock set, and determines whether the macroblock set is a target macroblock set according to whether the mean value of the motion vector of the macroblock in the macroblock set reaches a predetermined level, the macroblock set processing module is configured to: for each determined target macroblock, respectively combine it with four adjacent macroblocks above, below, left and right to form a macroblock set, and respectively average the horizontal motion vectors and vertical motion vectors of the five macroblocks in the macroblock set to obtain a horizontal motion vector mean value VDX and a vertical motion vector mean value VDY, if the horizontal motion vector mean value VDX is greater than the horizontal motion vector mean value VAX, or the vertical motion vector mean value VDY is greater than the vertical motion vector mean value VAY, then determine that the macroblock set is a target macroblock set; A target region processing module is used to obtain a target region corresponding to each target macroblock set in the image, wherein the target region corresponding to each target macroblock set is a rectangular region obtained by expanding a preset value outward with the target macroblock set as the center, and the target region processing module is configured to: for each target macroblock set obtained, take the target macroblock set as the center and expand a number of macroblocks upward, downward, left and right respectively, so as to obtain the target region corresponding to the target macroblock set in the image; A partitioning module, used to use each obtained target area as a foreground area, and use the remaining area of ​​the image except the target area as a background area; A coding compression module, used for obtaining the bandwidth of the network currently transmitting the video, and performing coding compression on the foreground area and the background area respectively according to the bandwidth of the network, so as to realize coding compression of the image, wherein if the bandwidth of the network increases from a first value to a second value, the coding compression degree of the foreground area is reduced to increase the amount of data obtained by coding compression of the image, and if the bandwidth of the network decreases from the first value to a third value, the coding compression degree of the background area is increased to reduce the amount of data obtained by coding compression of the image; The transmission module is used to transmit the data obtained by encoding and compressing the image through the network.

5. The device according to claim 4, characterized in that The encoding compression module is configured to: If the bandwidth of the network is within the first interval range [K1, +∞), the foreground area is coded and compressed using a coding compression coefficient n1, and the background area is coded and compressed using a coding compression coefficient m1, wherein n1 is smaller than m1; If the bandwidth of the network is within the second interval range [K2, K1), the foreground area is coded and compressed using a coding compression coefficient n2, and the background area is coded and compressed using a coding compression coefficient m2, wherein n2 is greater than n1 and less than m2, and m2 is greater than m1; If the bandwidth of the network is within the third interval range [K3, K2), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is coded and compressed using the coding compression coefficient m3, wherein m3 is greater than m2; If the bandwidth of the network is within the fourth interval range [0 kbps, K3), the foreground area is coded and compressed using the coding compression coefficient n2, and the background area is deleted; The coding compression coefficient is positively correlated with the coding compression degree, the first value is a value in the second interval, the second value is data in the first interval, and the third value is a value in the third interval.

6. The device according to claim 5, characterized in that The encoding compression method includes H.264 encoding method; Wherein, the K1 is any value between 500 kbps and 520 kbps, the K2 is any value between 390 kbps and 410 kbps, and the K3 is any value between 190 kbps and 210 kbps.

7. A video network transmission device, characterized in that: include: The method comprises a processor and a memory coupled to the processor, wherein the memory stores instructions for execution by the processor, and when the processor executes the instructions, the method according to any one of claims 1 to 3 can be implemented.

8. A video surveillance system, characterized in that: A video network transmission device comprising any one of claims 4 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images