One-screen multi-channel video display control system

Abstract

The invention discloses a same-screen multi-channel video display control system, relates to the technical field of video display, and solves the technical problem that video pictures of all channels cannot be accurately and synchronously displayed in real time due to transmission synchronization abnormity of video signals of different channels. A violent matching algorithm is combined to carry out feature matching, then a geometrical relationship of matching points and deflection angle analysis are utilized to determine mismatching points and eliminate the mismatching points, and finally time offset is accurately calculated to synchronously adjust videos of all channels, so that the problem of signal transmission synchronization is effectively solved, and real-time, stable and synchronous display of video pictures of all channels is realized. The accuracy and timeliness of information acquisition are greatly improved, the video channel image is preprocessed, the color, brightness and contrast differences are eliminated, the resolution limitation is overcome, the noise interference is removed, the quality of the video image is improved, and the requirement for the high-quality video image is met.

Description

Technical Field

[0001] The present invention relates to the technical field of video display, in particular to a same-screen multi-channel video display control system. Background Art

[0002] In the current same-screen multi-channel video display control system, with the increasing requirements for video display effects in video surveillance, command and dispatch, conference presentation and other fields, the system faces many challenges.

[0003] The patent application with patent application number CN202420064234.1 discloses a multi-channel special video switching control system, which relates to the field of video signal processing technology, including a video switching box, including multiple video signal input channels and multiple video signal output channels, and the video signal output channels are all connected to a display through an interface; a video control box communicatively connected to the video switching box, the video control box includes a button module and a display screen, and the button module can be used to select the video signal input channel that needs to be switched output, and the current system working status and display channel information are displayed through the display screen; the video switching box receives the video input signal and selectively outputs the video input signal according to the corresponding buttons on the video control box.

[0004] However, some existing video display control systems, when in use, on the one hand, have prominent transmission synchronization problems of video signals from different channels, and signal delay, attenuation and synchronization difficulties are common, resulting in the inability to synchronize the video images of each channel in real time and accurately, which seriously affects the acquisition and judgment of information. On the other hand, the image quality varies, and there may be differences in color, brightness, contrast, resolution limitations and noise interference between different channels, which greatly reduces the clarity and visibility of the video images. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a same-screen multi-channel video display control system, which solves the problem that the transmission synchronization of video signals from different channels is abnormal, resulting in the inability to synchronously display the video images of each channel in real time and accurately.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a same-screen multi-channel video display control system, comprising: The channel signal transmission analysis module extracts the image features of the video channel in the channel information transmitted by the channel information acquisition module according to the SIFT algorithm, generates image feature points, and screens the image feature points through the brute force matching algorithm to generate matching point information, and transmits the matching point information to the adaptive adjustment analysis module; The adaptive adjustment analysis module is used to eliminate the wrong matching points in the matching point information based on the geometric relationship of the matching points, obtain the correct matching points, calculate the time offset of the video frames of different channels according to the number of interval matching points, generate transmission adjustment information, and transmit it to the display analysis information output module; The channel image quality analysis module is used to preprocess the channel image to obtain a preprocessed channel image, and use the calculated grayscale values ​​of all colors of the preprocessed channel image as color balance values ​​to generate color adjustment information, and then generate comprehensive adjustment information by selecting the values ​​with the largest number of brightness and contrast of the same channel image as the standard for adjustment, and at the same time, redistribute the grayscale value of the image based on the image grayscale value to generate comprehensive adjustment information, and combine the color adjustment information and the comprehensive adjustment information to generate image quality adjustment information, and then transmit it to the display analysis information output module.

[0007] As a further solution of the present invention, it also includes a channel information acquisition module and a display analysis information output module; A channel information acquisition module, used to transmit the acquired channel information to the channel signal transmission analysis module and the adaptive adjustment analysis module respectively; The display analysis information output module is used to adjust the channel image according to the acquired transmission adjustment information and image quality adjustment information.

[0008] As a further solution of the present invention, the specific manner in which the channel signal transmission analysis module extracts image feature points of the video channel is: Get the video frame corresponding to video channel i, perform feature extraction on each frame of each channel video, detect extreme points as feature points in different scale spaces through SIFT algorithm, and calculate the 128-dimensional feature vector of each feature point as a descriptor, generate a feature point set and the corresponding feature descriptor, and obtain the image feature points.

[0009] As a further solution of the present invention, the specific manner in which the channel signal transmission analysis module generates matching point information is: Select brute force matching to match the obtained image feature points, obtain all the image feature points of the video channel and mark them as Di, then calculate each image feature point Di of channel i and compare them one by one with all the feature points of other channels, calculate the Euclidean distance between their feature descriptors, and compare them with the threshold at the same time, select the image feature points with Euclidean distance less than the threshold and record them as matching points; All matching points are determined in this way and the matching point information is generated.

[0010] As a further solution of the present invention, the specific method of the adaptive adjustment analysis module to determine the mismatch point is: Get all the matching points n, where n=1, 2, …, m, where m represents the number of matching points. Establish a rectangular coordinate system with the center point of the image as the origin. Use coordinates to represent the matching points and the deflection angles corresponding to the matching points. Then generate distribution features according to the distribution of the deflection angles, and obtain matching points whose deflection angles do not conform to the distribution features. Record them as mismatched points. Remove the mismatched points and record the remaining matching points as correct matching points.

[0011] As a further solution of the present invention, the specific manner in which the adaptive regulation analysis module generates the transmission adjustment information is: Get the video frame corresponding to video channel i and record it as the target object, and get the frame rate f corresponding to the target object, then get the matching video frame corresponding to the target object, and get the interval matching points between the target object and the matching video frame as k, calculate the time interval of each frame as 1 / f, and use the formula The time offset between the target object and the matching video frame is calculated, and the time offset of all video channels i is calculated in the same way; According to the calculated time offset, the video i of each channel is adjusted synchronously. If the target object video frame lags behind other channels as a whole, it can be played in advance by the corresponding time; if it is ahead, it is played back with delay, and transmission adjustment information is generated at the same time.

[0012] As a further solution of the present invention, the specific manner in which the channel image quality analysis module generates the color adjustment information is: The video channel i of the channel image is obtained, and the channel image is preprocessed to obtain a preprocessed channel image, the mean of the grayscale values ​​of all colors of the preprocessed channel image is calculated, and the mean is used as the color balance value, and then the image color is adjusted with the color balance value to generate color adjustment information.

[0013] As a further solution of the present invention, the specific manner in which the channel image quality analysis module generates the comprehensive adjustment information is: Get the brightness and contrast corresponding to all channel images of video channel i, and select the values ​​with the largest number of brightness and contrast of the same channel image as the standard for adjustment to obtain the adjusted image, and compare the image effect of the adjusted image with the channel image. If it does not exist, adjust it with the adjusted image as the standard to generate comprehensive adjustment information. If it exists, perform secondary adjustment.

[0014] As a further solution of the present invention, the specific manner in which the channel image quality analysis module performs secondary adjustment is: Count the frequency of occurrence of each gray value of the image, create an array of length 256, traverse each pixel of the image, use the gray value of each pixel as the index of the array, add 1 to the value of the corresponding array element, and get the gray histogram of the image; Based on the grayscale histogram obtained by statistics, its cumulative distribution function is calculated, and then the grayscale value of the original image is remapped according to the cumulative distribution function. The new grayscale value P can be obtained by the formula , where L is the range of grayscale values. Specifically, for an 8-bit grayscale image, L=256, h is the grayscale value in the original image, and round is the rounding function.

[0015] According to the calculated gray value mapping relationship, each pixel of the original image is traversed, and the gray value of each pixel is replaced with the remapped gray value to generate a new image with a more uniform histogram distribution, and at the same time generate comprehensive adjustment information.

[0016] The present invention provides a same-screen multi-channel video display control system. Compared with the prior art, it has the following beneficial effects: The present invention adopts SIFT algorithm to extract image feature points, combines brute force matching algorithm to perform feature matching, and then uses the geometric relationship and deflection angle analysis of matching points to determine and eliminate mismatching points. Finally, the time offset is accurately calculated to synchronously adjust the video of each channel, which effectively solves the signal transmission synchronization problem, realizes the real-time, stable and synchronous display of the video screen of each channel, and greatly improves the accuracy and timeliness of information acquisition. The video channel images are preprocessed, including median filtering denoising, color balance adjustment, brightness and contrast optimization, and secondary adjustment for geometric distortion, which effectively eliminates the differences in color, brightness, and contrast, overcomes the resolution limitation, removes noise interference, improves the quality of the video image, makes the picture clearer and more realistic, and meets the demand for high-quality video images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram of the system principle of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] For example, see Figure 1The present application provides a same-screen multi-channel video display control system, including a channel information acquisition module, a channel image quality analysis module, a channel information transmission analysis module and a display analysis information output module, and combined with Figure 1 It can be known that the above functional modules are electrically connected in a unidirectional manner.

[0020] Channel information acquisition module, this module is used to acquire all video channels in the same screen, and label the video channels at the same time, and specifically label the video channels as i, and i=1, 2, ..., j, where j represents the number of video channels, generate channel information, and then transmit the channel information to the channel image quality analysis module and the channel signal transmission analysis module respectively.

[0021] Channel signal transmission analysis module, which is used to process the acquired channel information. First, the image features corresponding to the video channel are extracted, and the feature points or feature areas in the video frame are analyzed. By matching the same features in video frames of different channels, the time correspondence between them is determined. The specific extraction method is: The video frame corresponding to the video channel i is obtained randomly. The feature extraction operation is performed on each frame image of each channel video to generate a feature point set and a corresponding feature descriptor. The feature extraction is performed by the SIFT algorithm. Taking the SIFT algorithm as an example, it detects extreme points as feature points in different scale spaces and calculates the 128-dimensional feature vector of each feature point as a descriptor. These descriptors can well represent the local image information of the feature points and obtain the image feature points. Then, a suitable matching algorithm is selected to match the obtained image feature points. Commonly used matching algorithms include brute-force matching and a fast approximate nearest neighbor search algorithm based on a kd tree. In the present application, a brute-force matching algorithm is used for analysis to obtain all image feature points of the video channel and mark them as Di. Then, each image feature point Di of channel i is calculated and compared with all feature points of other channels one by one. The distance between their feature descriptors (such as Euclidean distance) is calculated, and the calculated Euclidean distance is compared with a threshold. The specific value of the threshold is set by the operator. At the same time, the image feature points corresponding to the Euclidean distance less than the threshold are selected and recorded as matching points. All matching points are determined by analogy, and matching point information is generated. Then, the matching point information is transmitted to the adaptive adjustment analysis module. In a specific example, the principle of the brute force matching algorithm is to compare all feature points. Suppose we have two video channels, channel A and channel B, channel A extracts 100 image feature points, and channel B extracts 120 image feature points. All image feature points of channel A are labeled Di (i=1, 2, ..., 100). For each image feature point Di in channel A, it must be compared with the 120 feature points in channel B one by one. When comparing, the distance between their feature descriptors is calculated, and the Euclidean distance is used as the metric here.

[0022] For example, for feature point D1 in channel A, calculate the feature descriptor Euclidean distance between it and feature point E1 in channel B. The formula is , where the feature descriptor is a two-dimensional coordinate vector, the calculated Euclidean distance needs to be compared with the threshold set by the operator. Assume that the threshold is set to 50, if the calculated distance between feature point D1 and feature point E1 is 30, and 30 is less than the threshold 50, then feature point D1 and feature point E1 are matching points, and this operation is performed for all 100 feature points of channel A, and so on to determine all matching points.

[0023] The adaptive adjustment analysis module is used to analyze the acquired matching point information and determine the mismatching points by analyzing the geometric relationship of the matching points. The specific determination method is as follows: Get all the matching points and record them as n, and n=1, 2, ..., m, where m represents the number of matching points. Then, establish a rectangular coordinate system with the center point of the image as the origin. At the same time, use coordinates to represent the matching points, and use the origin as the standard point to obtain the deflection angle corresponding to the matching point. The deflection angle here is obtained by rotating clockwise. Then, obtain the distribution of the obtained deflection angles to generate distribution features. Specifically, in actual situations, the angle relationship between most matching point pairs meets expectations, and obtain matching points whose deflection angles do not meet the distribution features, which are recorded as mismatched points. At the same time, remove the obtained mismatched points, and record the remaining matching points as correct matching points. For example, in a specific example, m=80 matching points are obtained through the previous matching operation. In order to further analyze the characteristics of the matching points, a rectangular coordinate system is established with the center point of the image as the origin. In this coordinate system, each matching point can be represented by the coordinates (a, b). For example, the coordinates of one matching point n1 in the coordinate system are (30, 40). With the origin as the standard point, the deflection angle corresponding to the matching point is calculated by trigonometric functions. At the same time, according to the formula , and a≠0, then for the coordinates of the matching point n1 (30, 40), the deflection angle calculated according to the formula is 53.13°; Through the analysis of a large number of normal matching points, it is found that the deflection angles of normal matching points are mainly concentrated between 30° and 60°. Therefore, for matching points whose deflection angles do not conform to this distribution feature, they are recorded as mismatched points.

[0024] Then, the time offset of all correct matching points is calculated. Combined with the video frame rate, shooting time and other information, the time offset between video frames of different channels is calculated. The specific calculation method is as follows: Get the video frame corresponding to video channel i and record it as the target object, and get the frame rate f corresponding to the target object, then get the matching video frame corresponding to the target object. Specifically, through feature matching, it is found that the video frame of a certain channel (i.e., the target object) corresponds to the video frame of another channel, then the corresponding video frame is recorded as the matching video frame, and the interval matching points between the target object and the matching video frame are obtained and recorded as k. The specific interval matching points are expressed as the number of interval video frames between the two. The time interval of each frame is calculated and recorded as 1 / f, and according to the formula The time offset between the target object and the matching video frame is calculated, and the time offset of all video channels i is calculated in the same way; In a specific example, there are two channels 1 and 2. The video frame corresponding to video channel 1 is first obtained and recorded as the target object. It is known that the frame rate f of the video is 30 frames / second. Through feature matching, it is found that a video frame (target object) in video channel 1 corresponds to a video frame in video channel 2. The corresponding video frame in video channel 2 is recorded as the matching video frame. Further statistics show that the number of interval matching points between the target object and the matching video frame is k=5. The time offset is further calculated according to the formula to be 1 / 6 second.

[0025] According to the calculated time offset, the video i of each channel is adjusted synchronously. If the video frame of a certain channel lags behind other channels as a whole, it can be advanced by the corresponding time during playback; if it is ahead, the playback is delayed. Taking the software player as an example, the video frame can be advanced or delayed by setting the playback timestamp or adjusting the reading position of the playback buffer, so that the videos of each channel can be displayed synchronously in time. For example, it is calculated that the video frame of video channel 3 lags behind by 0.2 seconds as a whole. In the software player, the playback timestamp of the video of this channel can be advanced by 0.2 seconds, and the transmission adjustment information is generated and transmitted to the display analysis information output module.

[0026] The display analysis information output module is used to adjust the video channel i accordingly according to the acquired transmission adjustment information.

[0027] Embodiment 2: This embodiment is implemented on the basis of Embodiment 1, and the difference from Embodiment 1 is as follows: A channel image quality analysis module, which is used to analyze the acquired channel information, acquire the channel image corresponding to the video channel i, and preprocess the channel image, and the preprocessing operation specifically includes denoising, wherein the denoising is mainly processed by a median filter, sorting each pixel in the image and the pixels in its neighborhood according to the grayscale value, taking the middle value as the new grayscale value of the pixel, further obtaining a preprocessed channel image, and then calculating the grayscale values ​​of all colors of the preprocessed channel image, and calculating the mean of all color grayscale values, and taking the obtained mean as the color balance value, and then adjusting the image color with the color balance value to generate color adjustment information; Then, the brightness and contrast of the channel image are analyzed, all channel images corresponding to the video channel i are obtained, and the brightness and contrast corresponding to the channel image are obtained, and the values ​​of brightness and contrast are obtained here, and the values ​​of the brightness and contrast of the same channel image with the largest number are selected as the standard for adjustment to obtain an adjusted image, and the obtained adjusted image is compared with the channel image for image effect, and the channel image here is the original image before adjustment, and the image effect comparison is to determine whether the adjusted image has geometric distortion, such as stretching and distortion. If not, the adjusted image is used as the standard for adjustment to generate comprehensive adjustment information. If it exists, a secondary adjustment is performed; In case secondary adjustment is required, the grayscale value of the image is redistributed, and the specific allocation method is as follows: First, we need to count the frequency of occurrence of each grayscale value in the image. The grayscale value in the image is usually between 0 and 255 (for 8-bit grayscale images). We can create an array of length 256 to record the number of pixels for each grayscale value. We traverse each pixel of the image, use the grayscale value of each pixel as the index of the array, and add 1 to the value of the corresponding array element. This will give us the grayscale histogram of the image, which intuitively shows the distribution of each grayscale value in the image. Based on the grayscale histogram obtained by statistics, its cumulative distribution function is calculated. The cumulative distribution function represents the proportion of pixels with grayscale values ​​less than or equal to a certain value in the image. For each grayscale value h, its cumulative distribution function CDF (h) is equal to the sum of the occurrence frequencies of all pixels with grayscale values ​​from 0 to h. By performing cumulative operations on the grayscale histogram, the cumulative distribution function can be obtained; Then, the grayscale value of the original image is remapped according to the cumulative distribution function, and the new grayscale value P can be obtained by the formula , where L is the range of grayscale values, specifically for an 8-bit grayscale image, L=256, h is the grayscale value in the original image, and round is the rounding function.

[0028] According to the calculated gray value mapping relationship, each pixel of the original image is traversed, and the gray value of each pixel is replaced with the remapped gray value, so as to generate a new image with a more uniform histogram distribution and generate comprehensive adjustment information at the same time.

[0029] The color adjustment information and the comprehensive adjustment information are combined to obtain the image quality adjustment information, and the image quality adjustment information is transmitted to the display analysis information output module.

[0030] The display analysis information output module is used to adjust the video channel according to the acquired image quality adjustment information.

[0031] Embodiment 3, as the embodiment 3 of the present invention, focuses on combining the implementation processes of embodiment 1 and embodiment 2 for implementation.

[0032] Some of the data in the above formulas are calculated by taking their numerical values, and are not substituted into parameter units for calculation. At the same time, the contents not described in detail in this specification belong to the existing technologies known to those skilled in the art.

[0033] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A same-screen multi-channel video display control system, characterized in that: include: The channel signal transmission analysis module extracts the image features of the video channel in the channel information transmitted by the channel information acquisition module according to the SIFT algorithm, generates image feature points, and screens the image feature points through the brute force matching algorithm to generate matching point information, and transmits the matching point information to the adaptive adjustment analysis module; The adaptive adjustment analysis module is used to eliminate the wrong matching points in the matching point information based on the geometric relationship of the matching points, obtain the correct matching points, calculate the time offset of the video frames of different channels according to the number of interval matching points, generate transmission adjustment information, and transmit it to the display analysis information output module; The channel image quality analysis module is used to preprocess the channel image to obtain a preprocessed channel image, and use the calculated grayscale values ​​of all colors of the preprocessed channel image as color balance values ​​to generate color adjustment information, and then generate comprehensive adjustment information by selecting the values ​​with the largest number of brightness and contrast of the same channel image as the standard for adjustment, and at the same time, redistribute the grayscale value of the image based on the image grayscale value to generate comprehensive adjustment information, and combine the color adjustment information and the comprehensive adjustment information to generate image quality adjustment information, and then transmit it to the display analysis information output module.

2. A same-screen multi-channel video display control system according to claim 1, characterized in that: It also includes a channel information acquisition module and a display analysis information output module; A channel information acquisition module, used to transmit the acquired channel information to the channel signal transmission analysis module and the adaptive adjustment analysis module respectively; The display analysis information output module is used to adjust the channel image according to the acquired transmission adjustment information and image quality adjustment information.

3. A same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method in which the channel signal transmission analysis module extracts the image feature points of the video channel is: Get the video frame corresponding to video channel i, perform feature extraction on each frame of each channel video, detect extreme points as feature points in different scale spaces through SIFT algorithm, and calculate the 128-dimensional feature vector of each feature point as a descriptor, generate a feature point set and the corresponding feature descriptor, and obtain the image feature points.

4. A same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method of the channel signal transmission analysis module to generate matching point information is as follows: Select brute force matching to match the obtained image feature points, obtain all the image feature points of the video channel and mark them as Di, then calculate each image feature point Di of channel i and compare them one by one with all the feature points of other channels, calculate the Euclidean distance between their feature descriptors, and compare them with the threshold at the same time, select the image feature points with Euclidean distance less than the threshold and record them as matching points; All matching points are determined in this way and the matching point information is generated.

5. The same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method of determining the mismatch point by the adaptive adjustment analysis module is: Get all the matching points n, where n=1, 2, …, m, where m represents the number of matching points. Establish a rectangular coordinate system with the center point of the image as the origin. Use coordinates to represent the matching points and the deflection angles corresponding to the matching points. Then generate distribution features according to the distribution of the deflection angles, and obtain matching points whose deflection angles do not conform to the distribution features. Record them as mismatched points. Remove the mismatched points and record the remaining matching points as correct matching points.

6. A same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method in which the adaptive adjustment analysis module generates the transmission adjustment information is as follows: Get the video frame corresponding to video channel i and record it as the target object, and get the frame rate f corresponding to the target object, then get the matching video frame corresponding to the target object, and get the interval matching points between the target object and the matching video frame as k, calculate the time interval of each frame as 1 / f, and use the formula The time offset between the target object and the matching video frame is calculated, and the time offset of all video channels i is calculated in the same way; According to the calculated time offset, the video i of each channel is adjusted synchronously. If the target object video frame lags behind other channels as a whole, it can be played in advance by the corresponding time; if it is ahead, it is played back with delay, and transmission adjustment information is generated at the same time.

7. A same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method of generating color adjustment information by the channel image quality analysis module is as follows: The video channel i of the channel image is obtained, and the channel image is preprocessed to obtain a preprocessed channel image, the mean of the grayscale values ​​of all colors of the preprocessed channel image is calculated, and the mean is used as the color balance value, and then the image color is adjusted with the color balance value to generate color adjustment information.

8. The same-screen multi-channel video display control system according to claim 1, characterized in that: The specific method of generating comprehensive adjustment information by the channel image quality analysis module is as follows: Get the brightness and contrast corresponding to all channel images of video channel i, and select the values ​​with the largest number of brightness and contrast of the same channel image as the standard for adjustment to obtain the adjusted image, and compare the image effect of the adjusted image with the channel image. If it does not exist, adjust it with the adjusted image as the standard to generate comprehensive adjustment information. If it exists, perform secondary adjustment.

9. A same-screen multi-channel video display control system according to claim 8, characterized in that: The specific method of the secondary adjustment by the channel image quality analysis module is as follows: Count the frequency of occurrence of each gray value of the image, create an array of length 256, traverse each pixel of the image, use the gray value of each pixel as the index of the array, add 1 to the value of the corresponding array element, and get the gray histogram of the image; Based on the grayscale histogram obtained by statistics, its cumulative distribution function is calculated, and then the grayscale value of the original image is remapped according to the cumulative distribution function. The new grayscale value P can be obtained by the formula , where L is the range of grayscale values. Specifically, for an 8-bit grayscale image, L=256, h is the grayscale value in the original image, and round is the rounding function; According to the calculated gray value mapping relationship, each pixel of the original image is traversed, and the gray value of each pixel is replaced with the remapped gray value to generate a new image with a more uniform histogram distribution, and at the same time generate comprehensive adjustment information.

Images