Remote video conference data transmission method, system and device

Abstract

The present application relates to the field of video compression technology, and specifically to a remote video conferencing data transmission method, system and equipment, specifically including: based on the difference between the gradient angles of each to-be-searched point in the search template of any pixel point in the reference block and the corresponding pixel point in its target frame, the present application constructs the directional mutation of the reference block in different directions; based on the directional mutation, a new evaluation index is constructed in combination with the directional evaluation standard in the original diamond search method; the optimal matching point of the diamond search method is obtained using the new evaluation index; by correcting the coordinates of the peak point in the phase correlation matrix, the motion vector between the current reference block and its target block is calculated, and video compression transmission is performed based on the motion vector; the existing diamond search method avoids the possibility of incorrect selection of the optimal matching point due to misselection or omission of the optimal matching point, and can identify the sub-pixel offset between the reference block and the target block, thereby improving the video compression quality.

Description

Technical Field

[0001] The present application relates to the field of video compression technology, and in particular to a remote video conference data transmission method, system and device. Background Art

[0002] The evolution of remote video conferencing data transmission technology is closely intertwined with the global digitalization process, driving the continuous innovation of human collaboration. The development of image compression and transmission technology for remote video conferencing systems has always centered around achieving higher-definition real-time interaction with lower bandwidth consumption. Early technologies built a basic compression framework based on discrete cosine transforms and motion compensation, reducing redundant data through inter-frame prediction and macroblock segmentation, and were more common in simple image scenarios. With the emergence of the diamond search method, its advantages in video compression have also been demonstrated. By searching only the center and surrounding points, reducing the number of search points to match the local optimum, it achieves a balance between efficiency and accuracy, making it ideal for real-time encoding.

[0003] However, the current compression algorithms for these video data cannot be used in remote video conferencing scenarios because the commonly used diamond search method cannot obtain the optimal matching point when there is a large perspective change, resulting in errors in video motion estimation and reduced compression quality. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a remote video conferencing data transmission method, system and device, and the technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a remote video conference data transmission method, the method comprising the following steps:

[0006] Real-time capture of uncompressed video of remote conferences;

[0007] Set the reference frame of each video frame and use each video frame as the target frame of its reference frame; divide the reference frame into multiple pixel blocks, which are recorded as reference blocks; use the search method in the motion estimation algorithm to search for its target frame in the current reference block, and obtain the pixel block that matches the current reference block in the target frame, which is recorded as the target block. The specific search process is as follows:

[0008] Based on the difference between the gradient angles of each search point in the search template of any pixel in the current reference block and its corresponding pixel in the target frame, the directional mutation of the current reference block in different directions is constructed;

[0009] Based on the difference between the grayscale values of any pixel point and each of the points to be searched, combined with the directional mutation, constructing the motion deviation of the current reference block in different directions; and using the motion deviation as an evaluation indicator in the search method;

[0010] Obtain the phase correlation matrix between the current reference block and its target block; perform surface fitting on the neighborhood of each peak in the phase correlation matrix; obtain the maximum value point in the fitting surface;

[0011] The peak point coordinates are corrected based on the difference between the maximum point coordinates and the peak point coordinates; a motion vector between the current reference block and its target block is obtained based on the corrected peak point coordinates; and video compression transmission is performed based on the motion vector.

[0012] In one embodiment, the reference frame of each video frame is the previous frame of each video frame.

[0013] In one embodiment, the expression of the directional mutation is:

[0014]

[0015] Where, is the directional mutation of the current reference block in the direction corresponding to the kth search point in the search template; N and M represent the horizontal and vertical lengths of the current reference block respectively; Represents the pixel in the current reference block The gradient angle; Represents the pixel point in the target frame of the current reference block The gradient angle at is the cosine function, where the pixel point Pixel The coordinates of the kth point to be searched in the search template.

[0016] In one embodiment, the direction corresponding to the k-th point to be searched is: Point to pixel direction.

[0017] In one embodiment, the process of obtaining the motion deviation is:

[0018] The first weight ratio and the second weight ratio are preset and recorded as 、 The motion deviation of the current reference block content in the direction corresponding to the kth search point in the search template is recorded as , The expression is:

[0019]

[0020] Where, Represents the pixel in the current reference block Gray value of is the pixel point in the target frame of the current reference block The gray value at ; where .

[0021] In one embodiment, the process of obtaining the first weight ratio and the second weight ratio is as follows:

[0022] Obtain the grayscale co-occurrence matrix of the grayscale image of each video frame; calculate the difference between the contrast of the grayscale co-occurrence matrix of the current reference frame and its target frame, and the difference between the energy of the grayscale co-occurrence matrix, which are recorded as contrast difference and energy difference respectively;

[0023] Using a preset number of image frames before the current frame as input to the sliding window method, obtaining the mean and standard deviation of the contrast difference between the target frame and the reference frame in the preset number of image frames, and using the sum of the mean and three times the standard deviation as the contrast threshold;

[0024] Based on the energy difference between the reference frame and the target frame in the preset number of image frames, an energy threshold is obtained using the same acquisition method as the contrast threshold;

[0025] If the contrast difference between the current reference frame and its target frame is greater than the contrast threshold, and the energy difference is greater than the energy threshold, the first weight ratio is set to be less than the second weight ratio; otherwise, the first weight ratio is set to be greater than the second weight ratio.

[0026] In one embodiment, performing surface fitting on the neighborhood of each peak in the phase correlation matrix is specifically:

[0027] Taking any peak in the phase correlation matrix as the center, construct each peak neighborhood; perform surface fitting on all coordinate points in the neighborhood using a surface fitting algorithm to obtain the surface fitting equation.

[0028] In one embodiment, the process of correcting the peak point coordinates based on the difference between the maximum point coordinates and the peak point coordinates is as follows:

[0029] Mark the maximum point coordinates as , the peak point coordinates are marked as ; If the coordinates of the maximum point meet the , , then the maximum point As the peak point Coordinate correction value; otherwise, the peak point The coordinates remain the original coordinates.

[0030] In a second aspect, an embodiment of the present application further provides a remote video conferencing data transmission system, wherein a computer program is stored in the system, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0031] In a third aspect, an embodiment of the present application further provides a remote video conferencing data transmission device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method described in the first aspect above when executing the computer program.

[0032] The embodiments of the present application have at least the following beneficial effects:

[0033] This application constructs the directional mutation of the reference block in different directions based on the difference between the gradient angles of each search point in the search template of any pixel point in the reference block and the corresponding pixel point in its target frame; according to the directional mutation, combined with the directional evaluation standard in the original diamond search method, a new evaluation index is constructed, and the comprehensive judgment of the dual index has better anti-interference ability, which can improve the matching accuracy when there is a perspective change in the conference video, reduce the mismatch caused by directional dislocation, and avoid the problem of wrong selection or omission of the optimal matching point; the new evaluation index is used to obtain the optimal matching point of the diamond search method, avoiding the problem that the existing diamond search method may cause the optimal matching point to be selected incorrectly due to wrong selection or omission of the optimal matching point, affecting the reduction of video compression quality; by correcting the peak point coordinates in the phase correlation matrix, the sub-pixel offset between the reference block and the target block can be identified, thereby improving the video compression quality; and then the motion vector between the current reference block and its target block is calculated, and video compression transmission is performed based on the motion vector; thereby further solving the problem of poor compression quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 A flowchart of a remote video conference data transmission method provided in one embodiment of the present application;

[0036] Figure 2 Schematic diagram of the process of obtaining the first weight ratio and the second weight ratio. DETAILED DESCRIPTION

[0037] To further illustrate the technical means and effects employed by this application to achieve the intended invention objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a remote video conferencing data transmission method, system, and device proposed in this application, including its specific implementation, structure, features, and effects. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0039] The following describes in detail a method, system and device for remote video conferencing data transmission provided by the present application with reference to the accompanying drawings.

[0040] See also Figure 1 , which shows a flowchart of a remote video conference data transmission method provided by an embodiment of the present application, the method comprising the following steps:

[0041] Step S1: real-time capture of uncompressed video of a remote conference.

[0042] During a remote video conference, a conference terminal is used to obtain a video conferencing dataset. The camera hardware interface captures uncompressed raw video in real time. The video data is then directly encapsulated into raw data packets using the SRT low-latency transmission protocol. The SRT low-latency transmission protocol is well-known, and the specific process will not be described in detail.

[0043] This application converts the video data captured by the conference endpoint into the MP4 (H.264+AAC) format. As a container format, MP4 offers strong compatibility, supports mainstream players and editing tools, and balances compression and image quality.

[0044] Step S2: Set a reference frame for each video frame and use each video frame as a target frame for its reference frame; divide the reference frame into multiple pixel blocks, recorded as reference blocks; and construct the directional mutation of the current reference block in different directions based on the difference between the gradient angles of each search point in the search template of any pixel point in the current reference block and the corresponding pixel point in its target frame.

[0045] In remote video conferencing data transmission, video data compression is extremely important. In the process of video data compression transmission in remote video conferencing, motion estimation technology needs to be used to reduce time redundancy. For the original video, when using the motion estimation algorithm to estimate the motion of the key target, the reference frame must first be divided into pixel blocks, and each pixel block in the reference frame is used as a reference block. The area most similar to the reference block is searched in the target image to be matched as a matching block, and the relative displacement between the matching block and the reference block is the motion vector. Among them, when searching, the present application adopts the diamond search method for searching. As other embodiments of the present application, the implementer may also use other search methods for searching. The acquired video data is divided into blocks and motion estimation is performed separately to capture local motion information.

[0046] Due to the large perspective changes that can occur during video conferencing, the diamond search method can misselect the global optimal matching point due to rapid changes in direction, leading to inaccurate motion vectors. Large perspective changes can cause local deformation, occlusion, or non-rigid deformations during video conferencing, such as those caused by the speaker's movement or gestures. To address this issue, the search process is modified as follows:

[0047] First, in an embodiment of the present application, the previous frame of two adjacent frames of image is used as a reference frame, and the next frame is used as a target frame; the reference frame is evenly divided into 16×16 pixel blocks, each pixel block in the reference frame is used as each reference block, and the pixel block that matches the reference block in the target frame is used as the target block of the reference block.

[0048] It should be noted that with respect to the setting of the reference frame and the method of dividing the pixel blocks, this application only provides a reference frame setting method and a pixel block division method. The implementer may also use other images as reference frames and adopt other division methods to obtain pixel blocks. This application does not impose any specific restrictions.

[0049] When using the diamond search method, taking pixel (0,0) in the reference block as an example, a coarse search and a fine search are performed on pixel (0,0) using a large diamond search template (LDSP) and a small diamond search template (SDSP), respectively. The large diamond search template (LDSP) searches for a 2-pixel diamond formed by nine points including pixel (0,0) (coordinates: (0,0), (±2,0), (0,±2), (-1,±1), (1,±1)). After a coarse search using the large search template, the small diamond search template (SDSP) searches for a 1-pixel diamond formed by five points including the starting point (coordinates: (0,0), (±1,0), (0,±1)). Through both coarse and fine search processes, the optimal matching point is found, and the displacement between the starting point and the optimal matching point is used as the motion vector. Furthermore, for each reference frame and its target frame, the gradient angle of each pixel in the reference and target frames is calculated using the Sobel operator. The Sobel operator is a well-known technology, and the specific process will not be described in detail.

[0050] It should be noted that for the calculation of the gradient angle of pixel points in the reference frame and the target frame, this application only provides a gradient detection method. There are many existing gradient detection methods, and implementers can also use other gradient detection algorithms to calculate the gradient angle of pixel points in the reference frame and the target frame. This application does not make specific restrictions.

[0051] Furthermore, for any reference block in any reference frame, based on the difference between the gradient angles of any search point in the diamond search template of each pixel in the reference block and the corresponding pixel in its target frame, the similarity between the gradient angles of the pixel points in the corresponding area of the reference block in the target frame in different offset directions and the gradient angles of the pixel points in the reference block is analyzed to construct the directional mutation of the reference block in different directions. The expression is:

[0052]

[0053] Where, is the directional mutation of the current reference block in the direction corresponding to the kth search point in the diamond search template, where any pixel point of the current reference block For example, in the pixel In the diamond search template centered on , then the direction corresponding to the kth point to be searched in the diamond search template is the direction from the pixel point Point to pixel direction; N and M represent the horizontal and vertical lengths of the current reference block respectively; Represents the pixel in the current reference block The gradient angle; Represents the pixel point in the target frame of the current reference block The gradient angle at is the cosine function.

[0054] It is understandable that pixels The number of search points in the large diamond search template centered on is 9, and their coordinates are 、 、 、 、 ; Pixel The number of search points in the small diamond search template centered on is 5, and their coordinates are 、 、 .use 180-degree directional symmetry interference can be eliminated.

[0055] The smaller it is, the more likely it is that the content of the reference block and the content in the direction corresponding to the kth search point are the same object; directional mutation can improve the matching accuracy when there is a large perspective change in the conference video, and reduce the mismatch caused by direction misalignment. The closer the directional mutation is to 0, the more consistent the movement direction of the same object in adjacent frame images; the larger the directional mutation is, the more significant the directional change of the same object in adjacent frame images.

[0056] Step S3: constructing the motion deviation of the current reference block in different directions based on the difference between the grayscale values of any pixel point and the points to be searched and in combination with the directional mutation.

[0057] Furthermore, based on the directional mutation and combined with the directional evaluation criteria in the original diamond search method, a new evaluation index is constructed, which is expressed as follows:

[0058]

[0059] Where, is the motion deviation of the current reference block content in the direction corresponding to the kth search point in the diamond search template, 、 are the first weight ratio and the second weight ratio respectively, where, ; N, M represent the horizontal length and vertical length of the current reference block respectively; Represents the pixel in the current reference block Gray value of is the pixel point in the target frame of the current reference block The gray value at Pixels The coordinates of the kth point to be searched in the diamond search template centered at .

[0060] Among them, for The value of is determined as follows:

[0061] First, grayscale processing is performed on each frame of the conference video to obtain a grayscale image for each frame, and the grayscale co-occurrence matrix of each grayscale image is obtained. For the grayscale co-occurrence matrices of the current reference frame and the corresponding target frame, the difference in contrast between the two grayscale co-occurrence matrices and the difference in energy between the two grayscale co-occurrence matrices are calculated, denoted as the contrast difference and energy difference, respectively. The calculation of the grayscale co-occurrence matrix and the contrast and energy of the grayscale co-occurrence matrix are well-known techniques, and the specific process is not repeated here.

[0062] Then, the F frames before the current frame are used as input to the sliding window method, and the contrast difference and energy difference between the target frame and the reference frame are statistically analyzed to obtain the mean and standard deviation of the contrast difference, as well as the mean and standard deviation of the energy difference, between the target frame and the reference frame in the F frames. Preferably, in the embodiment of the present application, the value of F is set to 30. As other embodiments of the present application, the implementer can set the value of F according to actual conditions.

[0063] Finally, since the difference in texture features between frames in a stationary video sequence usually follows an approximate normal distribution, especially when the background is stationary or in motion, using the sum of the mean and 3 times the standard deviation as the threshold can cover more than 99% of the values. Specifically, the sum of the mean and 3 times the standard deviation of the contrast difference between the target frame and the reference frame in the F-frame image is used as the contrast threshold, and the sum of the mean and 3 times the standard deviation of the energy difference between the target frame and the reference frame in the F-frame image is used as the energy threshold; the threshold is set dynamically. If the contrast difference between the current reference frame and its target frame is greater than the contrast threshold, and the energy difference is greater than the energy threshold, then the current reference frame and its target frame are judged to have a strong texture transformation; other cases are judged to be weak texture transformations. In the case of weak texture transformation, let , preferably, in the embodiment of the present application, and The values of are set to 0.6 and 0.4 respectively to reduce the contribution of direction index judgment; when the perspective changes drastically in the conference video, resulting in strong texture changes, , preferably, in the embodiment of the present application, and The values of are set to 0.4 and 0.6 respectively, which tends to enhance the discrimination in the gradient direction. As other embodiments of this application, the implementer can set them according to the actual situation. and value.

[0064] Step S4, when the current reference block uses the search method in the motion estimation algorithm to search its target frame, the motion deviation is used as a new evaluation index to replace the original evaluation index in the search method, and the pixel block that matches the current reference block in the target frame is obtained and recorded as the target block.

[0065] When searching, taking the current reference block as an example, the motion deviation of all search points in the corresponding direction within the corresponding large diamond search template is calculated. The search point with the smallest motion deviation is selected as the optimal matching point. Similarly, when searching within the small diamond search template, the search point with the smallest motion deviation is also selected as the optimal matching point. Thus, the diamond search method is used to evaluate the motion of the current reference block, recording its motion vector (MV), that is, the position offset between the current reference block and the matching pixel block in the target frame, to preliminarily obtain the motion estimation target block position. By combining the original evaluation criteria with directional abruptness, a new evaluation metric is constructed to search for the optimal matching point. The introduction of directional abruptness compensates for the lack of geometric structure sensitivity of SAD (sum of absolute differences), improves matching accuracy, and reduces mismatches caused by directional misalignment.

[0066] Step S5, obtaining the phase correlation matrix between the current reference block and its target block; performing surface fitting on the neighborhood of each peak in the phase correlation matrix; obtaining the maximum point in the fitting surface; and correcting the peak point coordinates based on the difference between the maximum point coordinates and the peak point coordinates.

[0067] In video compression, the key to combining diamond search and phase correlation for motion estimation lies in balancing computational efficiency and motion vector accuracy through phase-by-phase optimization. Fine-tuning position offsets using phase correlation typically only yields integer-pixel offsets, not sub-pixel ones. The general steps of the phase correlation method are to first perform a fast Fourier transform (FFT) on the reference block and its matched target block, respectively, to obtain two complex spectral matrices. The FFT operation encodes image translation information as phase differences. Subsequently, the cross-power spectrum between these two spectral matrices is calculated. The phase information of this cross-power spectrum directly reflects the translation relationship between the two pixel blocks. The cross-power spectrum is then subjected to an inverse fast Fourier transform (IFFT) to obtain a phase correlation matrix. The peak position in this phase correlation matrix corresponds to the integer pixel-level offset between the two pixel blocks. For example, if the actual position offset between the reference block and the target block is (1.7, -0.8), a peak will appear at coordinates (2, -1) in the phase correlation matrix, indicating only that the target block is shifted 2 pixels to the right and 1 pixel up relative to the reference block. The peak coordinates (2, -1) of the phase correlation matrix cannot reveal the sub-pixel offset between the two pixel blocks. The phase correlation method, fast Fourier transform, cross-power spectrum, and inverse fast Fourier transform are all well-known technologies, and the specific processes will not be repeated here.

[0068] Since low sub-pixel accuracy in the compression process of remote data transmission in video conferencing will lead to a decrease in motion smoothness and a sense of video freeze or jump, the peak position in the phase correlation matrix needs to be interpolated and corrected again.

[0069] First, the phase correlation matrix between the current reference block and its target block is obtained. Ideally, if the two pixel blocks are perfectly aligned, the phase correlation matrix will have a peak at (0,0). If there is a shift (Δx, Δy) between the two pixel blocks, the peak will appear at (Δx, Δy), where Δx and Δy are integers.

[0070] Furthermore, a quadratic surface fitting is performed on the neighborhood of any peak in the phase correlation matrix. Specifically, the value at each coordinate point in the phase correlation matrix is the response value corresponding to each coordinate point. With the peak in the phase correlation matrix as the center, a 3*3 window is obtained as the neighborhood of the peak. The surface fitting is performed on all coordinate points in the neighborhood by the least squares method to construct a local surface model of the peak neighborhood. The expression of the obtained surface fitting equation is:

[0071]

[0072] Where, Peak point In the local surface model of the neighborhood, the coordinates The response value at ; m is the abscissa, n is the ordinate; a, b, c, d, e, and f are the coefficients of the surface fitting equation solved by the data in the peak neighborhood.

[0073] Among them, the least squares method for surface fitting is a well-known technology, and the specific process will not be repeated here. It should be noted that for surface fitting of data within the neighborhood, this application only provides one surface fitting method. There are many existing surface fitting methods, and implementers can also use other surface fitting algorithms to perform surface fitting on data within the neighborhood. This application does not make specific restrictions.

[0074] Further, find the partial derivative of the surface fitting equation and find the maximum point of the fitting surface , if the maximum point coordinates also meet , , that is, the extreme point is at the peak point In the neighborhood, the maximum point As the peak point Otherwise, the peak point The coordinates of remain the original coordinates, i.e., the peak point is not corrected. The process of obtaining the maximum point by taking the partial derivative of the equation is a well-known technique, and the specific process is not described here. The implementer may also use other methods to obtain the maximum point, and this application does not impose any specific restrictions.

[0075] Step S6, correcting the peak point coordinates based on the difference between the maximum point coordinates and the peak point coordinates; obtaining the motion vector between the current reference block and its target block based on the corrected peak point coordinates; and performing video compression transmission based on the motion vector.

[0076] The coordinate difference between the corresponding pixels in the reference block and the target block is obtained by correcting the peak values of the phase correlation matrix between the reference block and the target block. Furthermore, the coordinate correction value of each pixel in the target block is obtained by using the coordinate difference between the coordinates of each pixel in the reference block and the corresponding pixel. Furthermore, the motion vector between the reference block and the target block is obtained by using the coordinates of the pixels in the reference block and the corrected coordinate values of the pixels in the target block. The process of obtaining the motion vector is well known in the art and will not be described in detail here.

[0077] After obtaining the motion vectors between each reference block in the reference frame and the corresponding target block in its target frame, the H.264 encoder is used to perform motion compensation and residual compression based on the motion vectors. The video is then encapsulated into a bitstream using FFmpeg and finally transmitted using the WebRTC protocol. The content delivery network (CDN) and adaptive bitrate (ABR) are combined to optimize network transmission, achieving efficient video compression and real-time transmission.

[0078] This method uses frequency-domain phase differences to lock the coarse offset and then refines the accuracy using spatial-domain surface interpolation. This method retains the sensitivity of the phase correlation method to translational motion while overcoming its integer pixel limitations. The corrected motion vector significantly reduces the prediction residual energy, improving compression efficiency. While ensuring real-time performance, it approaches the theoretically optimal motion estimation accuracy, avoiding the stuttering and jumpiness associated with reduced motion smoothness in video conferencing. By combining diamond search and phase correlation, the video encoder can efficiently and accurately estimate the motion vectors of video conferencing data, thereby reducing residual data and improving compression efficiency.

[0079] The schematic diagram of the process of obtaining the first weight ratio and the second weight ratio is as follows Figure 2 shown.

[0080] Based on the same inventive concept as the above method, an embodiment of the present application also provides a remote video conference data transmission system, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above remote video conference data transmission methods are implemented.

[0081] Based on the same inventive concept as the above method, an embodiment of the present application also provides a remote video conference data transmission device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-mentioned remote video conference data transmission methods.

[0082] In summary, the embodiment of the present application provides a remote video conferencing data transmission method, which constructs the directional mutation of the reference block in different directions based on the difference between the gradient angles of each to-be-searched point in the search template of any pixel point in the reference block and the corresponding pixel point in its target frame; according to the directional mutation, combined with the directional evaluation standard in the original diamond search method, a new evaluation index is constructed, and the dual-index comprehensive judgment has better anti-interference ability, which can improve the matching accuracy when there is a perspective change in the conference video, reduce the mismatch caused by directional dislocation, and avoid the problem of misselection or omission of the optimal matching point; the new evaluation index is used to obtain the optimal matching point of the diamond search method, avoiding the problem that the existing diamond search method may cause the optimal matching point to be selected incorrectly due to misselection or omission of the optimal matching point, affecting the reduction of video compression quality; by correcting the peak point coordinates in the phase correlation matrix, the sub-pixel offset between the reference block and the target block can be identified, thereby improving the video compression quality; and then the motion vector between the current reference block and its target block is calculated, and video compression transmission is performed based on the motion vector, thereby further solving the problem of poor compression quality.

[0083] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the above descriptions are of specific embodiments of the present application. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0084] The various embodiments in this application are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0085] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A remote video conference data transmission method, characterized in that: The method comprises the following steps: Real-time capture of uncompressed video of remote conferences; Set the reference frame of each video frame and use each video frame as the target frame of its reference frame; divide the reference frame into multiple pixel blocks, which are recorded as reference blocks; use the search method in the motion estimation algorithm to search for its target frame in the current reference block, and obtain the pixel block that matches the current reference block in the target frame, which is recorded as the target block. The specific search process is as follows: Based on the difference between the gradient angles of each search point in the search template of any pixel in the current reference block and its corresponding pixel in the target frame, the directional mutation of the current reference block in different directions is constructed; Based on the difference between the grayscale values of any pixel point and each of the points to be searched, combined with the directional mutation, constructing the motion deviation of the current reference block in different directions; and using the motion deviation as an evaluation indicator in the search method; Obtain the phase correlation matrix between the current reference block and its target block; perform surface fitting on the neighborhood of each peak in the phase correlation matrix; obtain the maximum value point in the fitting surface; Correcting the peak point coordinates based on the difference between the maximum point coordinates and the peak point coordinates; obtaining a motion vector between the current reference block and its target block based on the corrected peak point coordinates; and performing video compression transmission based on the motion vector. The expression of the directional mutation is: Where, is the directional mutation of the current reference block in the direction corresponding to the kth search point in the search template; N and M represent the horizontal and vertical lengths of the current reference block respectively; Represents the pixel in the current reference block The gradient angle; Represents the pixel point in the target frame of the current reference block The gradient angle at is the cosine function, where the pixel Pixel The coordinates of the kth point to be searched in the search template; The process of obtaining the motion deviation is as follows: The first weight ratio and the second weight ratio are preset and recorded as The motion deviation of the current reference block content in the direction corresponding to the kth search point in the search template is recorded as The expression is: Where, Represents the pixel in the current reference block Gray value of is the pixel point in the target frame of the current reference block The gray value at ; where .

2. A remote video conference data transmission method according to claim 1, characterized in that: The reference frame of each video frame is the previous frame of each video frame.

3. A remote video conference data transmission method according to claim 1, characterized in that: The kth point to be searched corresponds to the direction: Point to pixel direction.

4. A remote video conference data transmission method according to claim 1, characterized in that: The process of obtaining the first weight ratio and the second weight ratio is as follows: Obtain the grayscale co-occurrence matrix of the grayscale image of each video frame; calculate the difference between the contrast of the grayscale co-occurrence matrix of the current reference frame and its target frame, and the difference between the energy of the grayscale co-occurrence matrix, which are recorded as contrast difference and energy difference respectively; Using a preset number of image frames before the current frame as input to the sliding window method, obtaining the mean and standard deviation of the contrast difference between the target frame and the reference frame in the preset number of image frames, and using the sum of the mean and three times the standard deviation as the contrast threshold; Based on the energy difference between the reference frame and the target frame in the preset number of image frames, an energy threshold is obtained using the same acquisition method as the contrast threshold; If the contrast difference between the current reference frame and its target frame is greater than the contrast threshold, and the energy difference is greater than the energy threshold, the first weight ratio is set to be less than the second weight ratio; otherwise, the first weight ratio is set to be greater than the second weight ratio.

5. A remote video conference data transmission method according to claim 1, characterized in that: The surface fitting is performed on the neighborhood of each peak in the phase correlation matrix, specifically: Taking any peak in the phase correlation matrix as the center, construct each peak neighborhood; perform surface fitting on all coordinate points in the neighborhood using a surface fitting algorithm to obtain the surface fitting equation.

6. A remote video conference data transmission method according to claim 1, characterized in that: The process of correcting the peak point coordinates based on the difference between the maximum point coordinates and the peak point coordinates is as follows: Mark the maximum point coordinates as , the peak point coordinates are marked as ; If the maximum point coordinates also meet , , then the maximum point As the peak point Coordinate correction value; otherwise, the peak point The coordinates remain the original coordinates.

7. A remote video conference data transmission system, wherein a computer program is stored in the system, characterized in that: When the computer program is executed by a processor, the steps of a remote video conference data transmission method as described in any one of claims 1 to 6 are implemented.

8. A remote video conference data transmission device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the remote video conference data transmission method as described in any one of claims 1 to 6 are implemented.

Images