A method for optimizing image transmission

Abstract

The present application relates to the technical field of image processing, and more particularly, to an image transmission optimization method, comprising: acquiring a video image and performing a grayscale processing thereon; and performing a transmission optimization processing on the video image based on a TSS block matching algorithm, comprising: dividing the video image into a plurality of initial image blocks, calculating a texture dynamic contrast and an optical flow motion intensity index of each initial image block, taking a product of a mean value of the texture dynamic contrast and a mean value of the optical flow motion intensity index of all the initial image blocks as a weight, and weighting a number of the initial image blocks to obtain a best image block number of the video image. The present application realizes compression of the video image based on the improved TSS block matching algorithm, improves a video image transmission effect of an intelligent monitoring robot dog, avoids picture distortion, loss and the like in a transmission process, and realizes optimization of image transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing. More particularly, the present application relates to an image transmission optimization method. BACKGROUND

[0002] As a kind of efficient mobile monitoring platform integrating multiple sensors and computing modules, intelligent monitoring robot dog has flexible motion ability and autonomous navigation ability, and can continuously patrol and perceive environment in complex environment. By carrying high-definition camera on intelligent monitoring robot dog, intelligent monitoring robot dog can realize real-time acquisition of video images of monitoring area, and transmit image data to remote control center through wireless network for intelligent analysis and task decision of back-end system, such as identifying abnormal target, warning dangerous state or generating navigation path. However, in actual application process, intelligent monitoring robot dog is often in dynamic moving state, and the network environment in which it moves is easily affected by shielding, interference or area switching, resulting in instability of communication link, and then bringing problems of image transmission delay, frame loss or resolution reduction, which may not meet the real-time and quality of image transmission.

[0003] In the prior art, three-step (TSS) block matching algorithm divides the image into multiple blocks and matches one by one to realize the compression and transmission of the image, which reduces the data amount in the transmission process, but the fixed search template and the number of image blocks result in insufficient estimation accuracy of running vector, especially when the robot dog moves quickly or the scene texture is complex, the TSS block matching algorithm is easy to fall into local optimum, and cannot accurately capture large-scale motion, resulting in block effect or blur phenomenon of the transmitted image, which restricts the real-time and restoration degree of the monitoring picture. SUMMARY

[0004] The present application provides an image transmission optimization method, which aims to solve the problem of insufficient estimation accuracy of running vector caused by the fixed search template and the number of image blocks of three-step (TSS) block matching algorithm in related technology.

[0005] The present invention provides an image transmission optimization method, comprising: obtaining a video image and gray-scaling it; performing transmission optimization processing on the video image based on a TSS block matching algorithm, comprising: dividing the video image into a plurality of initial image blocks, calculating the texture dynamic contrast and optical flow motion intensity index of each initial image block, using the product of the mean texture dynamic contrast and the mean optical flow motion intensity index of all initial image blocks as a weight, weighting the number of initial image blocks, and obtaining an optimal number of image blocks for the video image; the texture dynamic contrast is positively correlated with the sum of the gradients of all pixels in the image block, and the ratio of the image block contrast to the video image contrast is positively correlated with the sum of the ratios of the maximum and minimum entropies of the gray-level co-occurrence matrix of the image block in multiple directions; and the optical flow motion intensity index reflects the motion speed of the pixels in the optical flow field corresponding to the image block. Combining the texture dynamic contrast and the optical flow motion intensity index of the image block achieves weighted dynamic adjustment of the number of image blocks, effectively improving the content adaptation capability during video image compression. While ensuring the preservation of details in key image areas, the generation and transmission of redundant image blocks are reduced, thereby achieving efficient compression and transmission of video images while maintaining high visual restoration quality.

[0006] Furthermore, the method for obtaining the sum of the gradients of all pixels within an image block includes calculating the gradient magnitude of each pixel within the image block using a Sobel operator to obtain the sum of the gradients of all pixels within the image block. The Sobel operator is a classic first-order gradient edge detection operator that can extract pixel grayscale changes in both horizontal and vertical directions. By summing the gradient magnitudes of all pixels within an image block, it can effectively characterize edge information and detail intensity within the image block, reflecting its texture complexity.

[0007] Furthermore, methods for obtaining image block contrast and video image contrast include obtaining the grayscale value of each pixel in the image and calculating the variance of all pixel grayscale values ​​as the image contrast, where the image includes the image blocks and the video image. Using the variance of pixel grayscale values ​​as a contrast metric directly reflects the degree of dispersion of the image grayscale values. A larger value indicates a greater difference between bright and dark areas in the image, i.e., a stronger contrast.

[0008] Furthermore, a method for obtaining the entropy of the gray-level co-occurrence matrix of an image block in multiple directions includes: using the image block as input for the gray-level co-occurrence matrix in the directions of 0°, 45°, 90°, and 135°, and outputting the entropy of the gray-level co-occurrence matrix in the directions of 0°, 45°, 90°, and 135°. The gray-level co-occurrence matrix (GLCM) can describe the spatial co-occurrence relationship of grayscale values ​​between pixels in specific directions and distances. By calculating the entropy in four typical directions of 0°, 45°, 90°, and 135°, the texture distribution of the image block in different directions can be comprehensively reflected.

[0009] Furthermore, the method for calculating the optical flow motion intensity index includes: for any pixel point, calculating the ratio of the motion speed of the pixel point in the optical flow field corresponding to the image block and the sum of the absolute values ​​of the differences between the motion direction of the pixel point and the motion directions of the remaining pixels in the eight neighborhoods; and taking the average of the sum of the ratios of all pixels as the optical flow motion intensity index of the image block.

[0010] Furthermore, the optimal number of image blocks of the video image is obtained, and the calculation formula is: Where, Indicates the optimal number of image blocks when matching image blocks in the TSS block matching algorithm. Indicates the number of initial image blocks corresponding to the video image, Represents the mean value of the optical flow motion intensity index of all image blocks corresponding to the video image, It represents the mean value of the texture dynamic contrast of all image blocks of the video image, tanh() represents the hyperbolic tangent function, The term "ts" represents the nearest square number of the data in the brackets. By constructing a nonlinear product mapping relationship based on the mean optical flow motion intensity index and the mean texture dynamic contrast, combined with the hyperbolic tangent function and the nearest square number function, the optimal number of image blocks in the TSS block matching algorithm is determined. This allows for dynamic adjustment of the block partitioning strategy based on the image's motion and texture characteristics.

[0011] Furthermore, the video image is subjected to transmission optimization processing, including: taking any frame of the video image as a reference frame, obtaining the current video image frame to be subsequently transmitted in real time, and calculating the similarity between the current video image frame and the reference frame, and judging whether the reference frame needs to be replaced according to the similarity; if the reference frame does not need to be replaced, the current video image frame and the reference frame are input into the TSS block matching algorithm, and the best matching displacement of the current video image frame is output, instead of transmitting the video image, to achieve compression of the video image.

[0012] Furthermore, judging whether the reference frame needs to be replaced based on the similarity includes: if the similarity between the current video image frame and the reference frame is greater than or equal to a similarity threshold, then the reference frame does not need to be replaced; otherwise, the current video image frame is updated to a new reference frame.

[0013] Furthermore, the similarity between the current video image frame and the reference frame is calculated, including: obtaining the grayscale histogram of the reference frame and the grayscale histogram of the current video image frame, and also constructing their grayscale histograms; inputting the two grayscale histograms into the histogram intersection method, and outputting the similarity between the two as the similarity between the current video image frame and the reference frame.

[0014] Furthermore, the empirical value of the similarity threshold is 0.75.

[0015] Beneficial effects: The number of image blocks in the TSS block matching algorithm is improved based on the texture dynamic contrast and optical flow motion intensity index, so that image blocks with more dynamic elements and stronger motion intensity have more image blocks, thereby improving the accuracy of matching, and image blocks with fewer dynamic elements and weaker motion intensity have fewer image blocks, thereby improving the efficiency of matching; based on the improved TSS block matching algorithm, video image compression is achieved, the video image transmission effect of the intelligent monitoring robot dog is improved, and picture distortion, missing, etc. during the transmission process are avoided, thereby optimizing image transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 FIG. 1 is a flowchart schematically illustrating transmission optimization of video images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] like Figure 1 As shown, S101: acquiring a video image.

[0019] In one embodiment, a high-definition camera carried by an intelligent monitoring robot dog is used to capture video images in real time, thereby obtaining continuous frames of video images. During the acquisition process, the relevant parameters of the captured video can be set according to the movement speed of the robot dog and the image detail requirements. For example: in this embodiment, the preset video acquisition frame rate is 30FPS and the image resolution is 1080P. In order to facilitate subsequent analysis of the video image, the video image also needs to be grayscaled.

[0020] It should be noted that during video capture, environmental interference (such as lighting changes, airborne particles, dynamic background objects, etc.) as well as jitter or displacement caused by the intelligent robot dog during movement often result in varying degrees of noise in the video image. This noise can manifest as blurred image details, unclear edges, or artifacts, affecting the visual quality of the image and the accuracy of subsequent recognition processing. To improve image clarity and legibility, and thereby ensure image stability and effectiveness during subsequent encoding, transmission, and analysis, a Gaussian filter algorithm can be used to denoise the original video image. The denoising process of the Gaussian filter algorithm is well known and will not be detailed here.

[0021] In one embodiment, when using the TSS block matching algorithm to optimize the transmission of video images, the image needs to be initialized and divided into blocks. Specifically, the video image is evenly divided into several image blocks of equal size. In this embodiment, the size of the image block is 16×16. In other embodiments, the size of the image block can also be 14×14, etc.

[0022] S102: Calculate the texture dynamic contrast of each image block.

[0023] In one embodiment, when an intelligent robot dog captures video images, some dynamic elements may be present, such as the robot dog's movements, pedestrians, or other objects within the monitoring area. The frequency and speed of these dynamic elements can significantly affect image clarity and fidelity. Furthermore, background complexity and texture variations can also affect image transmission quality. Images containing a large amount of detail or complex backgrounds, such as crowds, trees, or buildings, can easily result in blocking artifacts or distortion during compression and transmission.

[0024] Based on the above analysis, a texture dynamic contrast method is constructed to reflect the dynamic features within image blocks in video images. The construction process of texture dynamic contrast is as follows: For each image block in the video image, the gradient of each pixel in the image block is calculated using the Sobel operator. The contrast of each image block and the contrast of the video image are then calculated. The image block contrast and the video image contrast are obtained by obtaining the grayscale value of each pixel in the image and calculating the variance of the grayscale values ​​of all pixels as the image contrast, where the image includes the image block and the video image. The image block is then used as the input of the grayscale co-occurrence matrix in the 0°, 45°, 90°, and 135° directions, and the output is the entropy of the grayscale co-occurrence matrix in the 0°, 45°, 90°, and 135° directions.

[0025] The texture dynamic contrast of each image block is constructed based on the above features. The texture dynamic contrast is calculated as follows: Where, It represents the texture dynamic contrast of the image block in the video image, S represents the sum of the gradients of all pixels in the image block, represents the contrast of the image block, Indicates the contrast of the video image. and They represent the maximum and minimum entropy of the gray-level co-occurrence matrix of the image block in the directions of 0°, 45°, 90°, and 135°, respectively.

[0026] As mentioned above, in the video images collected by the intelligent monitoring robot dog, if there are large dynamic changes in the image blocks, such as when the robot dog moves quickly or when the objects in the scene move quickly, the time difference between two adjacent frames is small, so it can be regarded as a straight line movement. The straight line movement will appear as motion blur in a single direction in the image, resulting in large differences in the entropy of the gray-level co-occurrence matrix in different directions. Larger; in scenes with more complex backgrounds, greater differences in grayscale values ​​between pixels correspond to larger pixel gradients in the corresponding image block, meaning a larger S. The closer the contrast of the image block matches that of the video image, the richer the illumination information and the more sufficient the illumination. Based on the above analysis, the more pronounced the dynamic changes in the image block and the richer the information content, the greater the calculated texture dynamic contrast.

[0027] S103: Calculate the optical flow motion intensity index of each image block.

[0028] In one embodiment, when an intelligent robot dog performs video surveillance, the captured video images not only exhibit dynamic characteristics but also contain complex motion information. This is particularly true during rapid movement, where the robot dog's high-speed motion can cause image blur or distortion. In particular, in rapidly changing scenes, moving objects in the image (such as people, vehicles, or the robot dog itself) often shift. In such cases, motion artifacts, blur, or image distortion may appear during video image transmission. To measure these motion characteristics, an optical flow motion intensity index is constructed to reflect the motion amplitude characteristics of image blocks in the video image. This facilitates the subsequent estimation of the number of parameter image blocks in the TSS block matching algorithm.

[0029] In one embodiment, the process of constructing the optical flow motion intensity index is as follows: for the image blocks in the video image, the optical flow field of each image block is calculated. The optical flow field is a vector field that describes the motion of the pixels in the image block, which can reflect the displacement of each pixel in time, that is, according to the displacement of the pixels between adjacent frames, the motion direction and speed of the object in the image are obtained, and then the motion intensity of the object contained in the image block is analyzed. The process of calculating the optical flow field of the image block is a well-known technology and will not be described here. For any pixel point, the ratio of the motion speed of the pixel point in the optical flow field corresponding to the image block and the sum of the absolute values ​​of the differences between the motion direction of the pixel point and the motion direction of the remaining pixels in the eight neighborhoods is calculated, and the average of the sum of the ratios of all pixels is used as the optical flow motion intensity index of the image block; the specific calculation formula is: Where, Represents the optical flow motion intensity index of the image block, Indicates the number of pixels in the image block, Indicates the motion speed of the ath pixel in the optical flow field corresponding to the image block, It represents the sum of the absolute values ​​of the differences between the motion direction of the a-th pixel in the optical flow field corresponding to the image block and the motion directions of the remaining pixels in the eight-neighborhood. 1 represents a hyperparameter, which is used to avoid the denominator being zero and causing calculation failure. It can be selected according to the situation.

[0030] As mentioned above, in an image block, the pixels belonging to the same object should be in the same area, and the object's movement direction should be consistent in the very short time between two frames. Therefore, when an object moves, the movement direction of the object's pixels and its neighboring pixels is relatively small, that is, Smaller, and if the pixel points in the object move faster, that is The larger it is, the greater the motion intensity of the object in the image block is, and therefore the larger the calculated optical flow motion intensity index is.

[0031] S104: Calculate the final number of image blocks of each video image.

[0032] In one embodiment, in the TSS block matching algorithm, the number of image blocks is an important parameter. A large number of image blocks can easily lead to matching errors, while a small number of image blocks increases the amount of computation and affects computational efficiency. The texture dynamic contrast and optical flow motion intensity index of the image blocks obtained through the above steps respectively reflect the dynamic characteristics and motion intensity characteristics of the image blocks in the video image. For image blocks with more obvious dynamic characteristics and greater motion intensity, the objects within the image blocks are prone to large displacement changes, so a larger number of image blocks is required to improve matching accuracy. For image blocks with less obvious dynamic characteristics and less motion intensity, the objects within the image blocks undergo smaller displacement changes, so only a smaller number of image blocks is required to complete the matching, which facilitates improving matching efficiency.

[0033] Through the above analysis, the number of image blocks in the TSS block matching algorithm for any video image is calculated based on the texture dynamic contrast and optical flow motion intensity index. The calculation method is as follows: Where, Indicates the optimal number of image blocks when matching image blocks in the TSS block matching algorithm. Indicates the number of initial image blocks corresponding to the video image, Represents the mean value of the optical flow motion intensity index of all image blocks corresponding to the video image, It represents the mean value of the texture dynamic contrast of all image blocks of the video image, tanh() represents the hyperbolic tangent function, It means taking the nearest square number of the data in the brackets, for example , , used to ensure uniform segmentation of the video image, 0.5 represents the coordination coefficient, which is used to adjust the value range of the hyperbolic tangent function.

[0034] To sum up, in the video images collected by the intelligent monitoring robot dog, if there are more dynamic elements and objects with higher motion intensity in the image blocks, indicating that the difference between adjacent frames is large, then more image blocks are required in the TSS block matching algorithm to match the same object, so the number of calculated image blocks is large; if there are fewer dynamic elements in the image blocks and the object's motion intensity is small, indicating that the difference between adjacent frames is small, then only a smaller number of image blocks are required to achieve matching of the same object, so the number of calculated image blocks is small.

[0035] S105: Perform transmission optimization based on the final number of image blocks of each video image in the TSS block matching algorithm.

[0036] In one embodiment, the final number of image blocks of each video image can be determined according to the above steps, and then image transmission optimization is performed based on the final number of image blocks and the TSS block matching algorithm.

[0037] Specifically, for the Frame video image , use it as a reference frame, count the grayscale value distribution characteristics of the reference frame and construct the corresponding grayscale histogram. For the current video image frame to be transmitted in real time, its grayscale histogram is also constructed. The grayscale histograms of the reference frame and the current video image frame are input into the histogram intersection method, and the similarity value between the two is output. If the similarity is greater than or equal to the preset similarity threshold, it means that there is no significant change between the current video image frame and the reference frame. In this embodiment, the similarity threshold is set to 0.75, which can be adjusted according to the application scenario.

[0038] When the current video frame and the reference frame have not changed significantly, the TSS block matching algorithm can be used. It takes the video frame and the reference frame as input, divides the video frame into blocks based on their correlation, determines the best matching displacement for each block, and uses this to calculate the final number of blocks in the video frame. Ultimately, compression is achieved by transmitting only the number of blocks and the best matching displacement data for each block, replacing the original video frame. Setting the number of dynamic blocks based on the degree of image change effectively reduces distortion caused by image compression and improves the quality and efficiency of video transmission.

[0039] If the similarity between the current video frame and the reference frame is less than the similarity threshold, it indicates that the video frame has changed significantly and is no longer suitable as a reference frame for matching and compression. In this case, the current video frame is updated as the new reference frame, and the above process is repeated for subsequent video image compression.

[0040] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

Claims

1. An image transmission optimization method, characterized in that: Optimization methods include: Acquire video images and convert them into grayscale; Transmission optimization of video images based on the TSS block matching algorithm, including: The video image is divided into several initial image blocks, and the texture dynamic contrast and optical flow motion intensity index of each initial image block are calculated. The product of the mean texture dynamic contrast and the mean optical flow motion intensity index of all initial image blocks is used as the weight, and the number of initial image blocks is weighted to obtain the optimal number of image blocks for the video image; Texture dynamic contrast is: , It represents the texture dynamic contrast of the image block in the video image, S represents the sum of the gradients of all pixels in the image block, represents the contrast of the image block, Indicates the contrast of the video image. and Respectively represent the maximum and minimum values ​​of the entropy of the gray-level co-occurrence matrix of the image block in the directions of 0°, 45°, 90°, and 135°; The image block optical flow motion intensity index is: , Represents the optical flow motion intensity index of the image block, Indicates the number of pixels in the image block, Indicates the motion speed of the ath pixel in the optical flow field corresponding to the image block, It represents the sum of the absolute values ​​of the differences between the motion direction of the ath pixel in the optical flow field corresponding to the image block and the motion directions of the remaining pixels in the eight-neighborhood. 1 represents a hyperparameter to avoid the denominator being zero, which leads to incomprehensible calculations. The optimal number of image patches is: ； Indicates the optimal number of image blocks when matching image blocks in the TSS block matching algorithm. Indicates the number of initial image blocks corresponding to the video image, Represents the mean value of the optical flow motion intensity index of all image blocks corresponding to the video image, It represents the mean value of the texture dynamic contrast of all image blocks of the video image, tanh() represents the hyperbolic tangent function, It means taking the nearest square number of the data in the brackets.

2. The image transmission optimization method according to claim 1, characterized in that: Optimize the transmission of video images, including: Take any frame of video image as a reference frame, obtain the current video image frame to be transmitted in real time, calculate the similarity between the current video image frame and the reference frame, and determine whether the reference frame needs to be replaced based on the similarity; If the reference frame does not need to be replaced, the current video image frame and the reference frame are input into the TSS block matching algorithm, and the best matching displacement of the current video image frame is output to replace the transmission of the video image and realize the compression of the video image.

3. The image transmission optimization method according to claim 2, characterized in that: Determine whether the reference frame needs to be replaced based on the similarity, including: If the similarity between the current video image frame and the reference frame is greater than or equal to the similarity threshold, the reference frame does not need to be replaced; otherwise, the current video image frame is updated to a new reference frame.

4. The image transmission optimization method according to claim 2, characterized in that: Calculate the similarity between the current video image frame and the reference frame, including: Obtaining a grayscale histogram of the reference frame and a grayscale histogram of the current video image frame; The two grayscale histograms are input into the histogram intersection method, and the similarity between the two is output as the similarity between the current video image frame and the reference frame.

5. The image transmission optimization method according to claim 3, characterized in that: The empirical value of the similarity threshold is 0.75.

Images