Automatic switching method and device for multi-channel video signals based on a video switcher

Abstract

This application relates to the field of video signal switching technology, and discloses a multi-channel video signal automatic switching method and device based on a production switcher. The method includes: performing preprocessing and feature calculation on multiple video signal channels to generate a signal feature vector set and a signal quality index, and constructing a dynamic weight matrix; performing adaptive frame rate control and energy optimization to obtain an optimized hardware working state and a warm-up cache state; generating a scene change mapping; performing intelligent switching decision-making and signal fusion to obtain a video signal after transition processing and a switching log; performing data compression and transmission optimization to generate a compressed data stream and transmission control parameters; performing signal demultiplexing and reconstruction to output a target video signal. This application realizes the full-process automation from video signal preprocessing to final output, greatly improves the efficiency of multi-channel video signal processing and switching, and reduces the intervention of manual operations and possible errors.

Description

Technical Field

[0001] This application relates to the technical field of video signal switching, and particularly to a multi-channel video signal automatic switching method and device based on a video switcher. Background Art

[0002] With the rapid development of the radio and television industry, the processing and switching of multi-channel video signals have become an indispensable part of program production. Traditional video switchers mainly rely on manual operation and face many challenges when dealing with complex multi-channel video signals. Manual operation is easily affected by subjective factors and it is difficult to ensure the continuity and stability of switching, especially in high-intensity live broadcast environments where mistakes are more likely to occur. In addition, with the continuous improvement of video signal quality and the increasingly strict requirements for program production, the efficiency and accuracy of manual operation can no longer meet the needs of modern radio and television production.

[0003] At the same time, the diversity and complexity of video signals are also increasing continuously, including different resolutions, frame rates, coding formats, etc., which brings greater difficulties to signal processing and switching. In the process of processing multi-channel video signals, how to effectively perform feature extraction, quality assessment, adaptive control, and energy optimization has become a technical problem to be solved urgently. Traditional switching methods often have difficulty coping with these complex scenarios and are prone to problems such as decreased picture quality and discontinuous switching. In addition, in the signal transmission and reconstruction links, how to achieve efficient data compression and transmission optimization while ensuring video quality is also an important challenge in current research. Traditional methods often face bandwidth pressure and real-time problems when dealing with large-scale and high-quality video data and are difficult to meet the high requirements of modern radio and television production. Summary of the Invention

[0004] This application provides a multi-channel video signal automatic switching method and device based on a video switcher, which is used to realize the full-process automation from video signal preprocessing to final output, improve the efficiency of multi-channel video signal processing and switching, and reduce the intervention and possible errors of manual operation.

[0005] In the first aspect, this application provides a multi-channel video signal automatic switching method based on a video switcher. The multi-channel video signal automatic switching method based on a video switcher includes:

[0006] Preprocess and calculate features for multiple video signal channels, generate a signal feature vector set and signal quality indicators, and construct a corresponding dynamic weight matrix;

[0007] Based on the signal feature vector set and the signal quality indicators, perform adaptive frame rate control and energy optimization to obtain an optimized hardware working state and warm-up cache state;

[0008] Perform multi-scale feature extraction and matching on video frames according to the optimized hardware working state and the preheating cache state to generate a scene change map;

[0009] Based on the scene change map and the dynamic weight matrix, perform intelligent switching decision-making and signal fusion to obtain a video signal after transition processing and a switching log;

[0010] Perform data compression and transmission optimization on the video signal after transition processing, and adjust the compression strategy according to the switching log to generate a compressed data stream and transmission control parameters;

[0011] Based on the compressed data stream and the transmission control parameters, perform signal demultiplexing and reconstruction to output a target video signal.

[0012] In a second aspect, the present application provides a multi-channel video signal automatic switching device based on a video switcher. The multi-channel video signal automatic switching device based on a video switcher includes:

[0013] A preprocessing module for preprocessing and feature calculation on multiple video signal channels to generate a signal feature vector set and a signal quality index, and construct a corresponding dynamic weight matrix;

[0014] An optimization module for performing adaptive frame rate control and energy optimization based on the signal feature vector set and the signal quality index to obtain an optimized hardware working state and a preheating cache state;

[0015] A matching module for performing multi-scale feature extraction and matching on video frames according to the optimized hardware working state and the preheating cache state to generate a scene change map;

[0016] A fusion module for performing intelligent switching decision-making and signal fusion based on the scene change map and the dynamic weight matrix to obtain a video signal after transition processing and a switching log;

[0017] A generation module for performing data compression and transmission optimization on the video signal after transition processing, and adjusting the compression strategy according to the switching log to generate a compressed data stream and transmission control parameters;

[0018] An output module for performing signal demultiplexing and reconstruction based on the compressed data stream and the transmission control parameters to output a target video signal.

[0019] The third aspect of the present application provides a multi-channel video signal automatic switching device based on a video switcher, including: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory to enable the multi-channel video signal automatic switching device based on the video switcher to execute the above-mentioned multi-channel video signal automatic switching method based on the video switcher.

[0020] The fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions run on a computer, the computer is enabled to execute the above-mentioned multi-channel video signal automatic switching method based on the video switcher.

[0021] In the technical solution provided by the present application, by performing preprocessing and feature calculation on multiple video signal channels, a signal feature vector set and a signal quality index are generated. Based on the signal features and quality index, adaptive frame rate control and energy optimization are performed, which can effectively improve the operation efficiency, reduce energy consumption, and at the same time ensure the quality of video processing. By performing multi-scale feature extraction and matching on video frames to generate a scene change map, the changes in video content can be captured more accurately. Based on the scene change map and the dynamic weight matrix, intelligent switching decisions and signal fusion are performed, which can achieve smoother and more natural picture transitions and enhance the viewing experience. By performing data compression and transmission optimization on the video signal after transition processing and adjusting the compression strategy according to the switching log, the transmission efficiency can be improved while ensuring the video quality, and the bandwidth pressure can be reduced. Through the optimized signal demultiplexing and reconstruction process, the original video signal can be restored quickly and accurately, ensuring the quality of the finally output video. The present application realizes the full-process automation from video signal preprocessing to final output, greatly improving the efficiency of multi-channel video signal processing and switching, and reducing the intervention of human operations and possible errors. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of an embodiment of the multi-channel video signal automatic switching method based on the video switcher in the embodiments of the present application;

[0024] Figure 2 It is a schematic diagram of an embodiment of the multi-channel video signal automatic switching device based on the video switcher in the embodiments of the present application. Detailed Embodiments

[0025] The embodiments of the present application provide a multi-channel video signal automatic switching method and device based on a video switcher. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order different from that illustrated or described here. In addition, the term "including" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] For ease of understanding, the specific process of the embodiments of the present application will be described below. Please refer to Figure 1 In an embodiment of the multi-channel video signal automatic switching method based on a video switcher in the embodiments of the present application, it includes:

[0027] Step S101: Perform preprocessing and feature calculation on multiple video signal channels to generate a signal feature vector set and a signal quality index, and construct a corresponding dynamic weight matrix;

[0028] It can be understood that the execution subject of the present application can be a multi-channel video signal automatic switching device based on a video switcher, or a terminal or a server. Specifically, it is not limited here. The embodiments of the present application will be described by taking the server as the execution subject as an example.

[0029] Specifically, noise reduction processing is performed on the original video signal through spatial domain filtering technology to reduce the noise components in the video signal, and the noise-reduced video signal is obtained. Color correction is performed on the noise-reduced video signal to correct the color distortion in the video signal and ensure the accuracy of color restoration, and the color-corrected video signal is obtained. Resolution normalization processing is performed on the color-corrected video signal to eliminate the resolution differences between different video signal channels by unifying the resolution of the video signal, and the normalized video signal is obtained. Texture feature extraction is performed on the normalized video signal, and a texture feature map is generated by analyzing the texture information of the video signal. Motion vector field calculation is performed based on the texture feature map and the normalized video signal, and the motion intensity feature is obtained by analyzing the motion vectors between video frames. At the same time, color histogram analysis and clustering are performed on the normalized video signal, and the color distribution feature is obtained by statistically analyzing the distribution of different colors in the video image. These features help to describe the color information in the video signal. Scene change frequency calculation is performed on the motion intensity feature and the color distribution feature, and the scene change feature is obtained by analyzing the scene change frequency in the video signal. At the same time, discrete cosine transform and quantization are performed on the normalized video signal, and the signal-to-noise ratio and structural similarity index are calculated to evaluate the quality of the video signal, and the signal quality index is obtained. Multidimensional feature vector construction is performed on the texture feature map, the motion intensity feature, the color distribution feature, and the scene change feature, and the signal feature vector set is obtained by integrating these features into a multidimensional vector. Principal component analysis is performed on the signal feature vector set and the signal quality index, and the main feature components are extracted through dimensionality reduction and feature extraction to simplify the complexity of the feature vector. At the same time, the fuzzy comprehensive evaluation method is used to comprehensively evaluate the feature vector, and the dynamic weight matrix is obtained. The dynamic weight matrix generates a matrix reflecting the importance of video signal features by comprehensively considering the weights of each feature.

[0030] Step S102, based on the signal feature vector set and the signal quality index, perform adaptive frame rate control and energy optimization to obtain the optimized hardware working state and warm-up cache state;

[0031] Specifically, the signal feature vector set is compressed to obtain compressed feature vectors. By reducing the dimension of the feature vectors, the computational complexity is reduced. Based on the compressed feature vectors and signal quality metrics, a frame rate prediction model is constructed, and the frame rate prediction values for each video signal channel are calculated. The frame rate prediction values are smoothed to obtain frame rate control parameters. These parameters, in combination with the current hardware load information, are used to calculate the optimal operating frequency of the processor cores, and then frequency adjustment instructions are generated. This process optimizes the utilization of energy by adjusting the operating frequency of the processor. The usage information of the hardware resources is analyzed, and the future resource requirements are predicted to obtain resource requirement prediction values. The resource requirement prediction values, combined with the frequency adjustment instructions, are used to optimize the switching strategy of the functional units to form a functional unit configuration plan. By intelligently managing the usage of hardware resources, the energy efficiency and system performance are improved. At the same time, through the analysis of historical scene change data and typical scene pattern recognition, the category labels of each scene are obtained. Based on the scene category labels and the characteristics of the current scene, a preheating data list is generated. The elements in these data lists are sorted according to priority, and the optimal loading order is calculated to form a cache preheating sequence. According to the cache preheating sequence and the functional unit configuration plan, the hardware resources are reasonably allocated to ensure that each functional unit can operate in the most suitable state, obtaining an optimized hardware operating state and a preheated cache state.

[0032] Step S103: According to the optimized hardware operating state and the preheated cache state, perform multi-scale feature extraction and matching on the video frames to generate a scene change map;

[0033] Specifically, according to the optimized hardware working state, the video frames are decomposed at multiple scales to obtain multiple image sequences of different scales. Each scale image sequence contains multiple levels from the original image to the low-resolution image. Feature extraction is performed on each scale image in the multi-scale image sequence to obtain a target point set and a descriptor set. The feature points and descriptors describe the local features of the image at different scales. While performing feature extraction, according to the warm-up cache state, predefined features are loaded from the cache. Using the predefined features, the previously extracted target point set is filtered to obtain a filtered target point set. Distribution optimization is performed on the filtered target point set to ensure that the feature points are more evenly distributed in the image, resulting in a uniformly distributed feature point set, reducing the situation where feature points are too dense in some areas and sparse in other areas. Based on the uniformly distributed feature point set and the corresponding descriptor set, a feature database is constructed and a feature index structure is generated. The feature database stores all feature points and their descriptors, while the feature index structure is used to quickly search for and match feature points. Based on the feature database and the feature index structure, partition matching is performed on the feature points between consecutive frames to obtain an initial matching result, and the corresponding relationship of feature points between consecutive frames is initially determined. According to the initial matching result, a geometric consistency matching set is generated, and through geometric consistency verification, the spatial consistency and rationality of the matched feature points are ensured, and false matches are excluded. Motion information calculation is performed on the geometric consistency matching set to obtain a motion vector map. The motion vector map describes the motion of feature points between consecutive frames. According to the motion vector map, regional segmentation and analysis of the image are performed to determine the scene change situation in different regions of the image, and a scene change region is obtained. Encoding and feature fusion are performed on the scene change region. The encoding process converts the spatial information and motion information of the scene change region into a digital representation, while feature fusion combines multiple features to form a unified description of the scene change, and finally a scene change map is obtained.

[0034] Step S104: Based on the scene change map and the dynamic weight matrix, perform intelligent switching decision-making and signal fusion to obtain a video signal after transition processing and a switching log;

[0035] Specifically, perform regional importance analysis on the scene change mapping to obtain the scene importance score, and evaluate the degree of attraction of each region to the audience's attention. Based on the scene importance score and the dynamic weight matrix, calculate the comprehensive score of each video signal channel and generate a channel priority list. The channel priority list reflects the importance ranking of each video signal channel and determines the priority order of signal switching. Perform temporal smoothing processing on the channel priority list to avoid instability and incoherence caused by frequent switching. Through the smoothing processing, obtain a more stable and reasonable switching candidate sequence. According to the switching candidate sequence and the preset program structure rules, generate an initial switching plan. These program structure rules include specific requirements for switching frequency, duration, and scene type to ensure that the switching plan conforms to the norms and styles of program production. After generating the initial switching plan, perform conflict detection and adjustment on it to ensure the feasibility and rationality of the plan. The purpose of conflict detection is to discover potential resource conflicts or logical errors, and the adjustment process is to resolve these conflicts to obtain an optimized switching plan. According to the optimized switching plan, select the corresponding video signal channels and generate the source signals to be fused. In the signal fusion stage, perform spatio-temporal alignment processing on the source signals to be fused to ensure the consistency of different video signals in time and space and avoid visual discomfort caused by out-of-sync or inconsistent positions. According to the aligned signals, calculate the best transition effect parameters. These parameters include the type, duration, and effect intensity of the transition. Through these parameters, obtain specific transition effect instructions. The transition effect instructions are an important basis for actually performing the transition processing and guide the system on how to achieve smooth switching between different video signals. Execute the transition effect instructions through the aligned signals to obtain the video signals after the transition processing. At the same time, record the execution process of the optimized switching plan and the transition effect instructions to generate a switching log. The switching log records information such as the time point of each signal switch, the signal source, and the transition effect.

[0036] Step S105: Perform data compression and transmission optimization on the video signals after the transition processing, and adjust the compression strategy according to the switching log to generate a compressed data stream and transmission control parameters;

[0037] Specifically, perform spatio-temporal analysis on the video signal after transitional processing to obtain a signal complexity graph, which reflects the complexity of the video signal at different times and spaces. According to the signal complexity graph and the switching log, divide the video frames into different compression priority regions to generate a regional compression strategy. The regional compression strategy allocates different compression resources according to the importance and complexity of the video content to achieve an optimized compression effect. Perform dynamic quantization parameter matrix analysis on the video signal to obtain an initial quantization result. The dynamic quantization parameter matrix analysis balances the compression ratio and the image quality by adjusting the quantization parameters. According to the regional compression strategy, refine and adjust the initial quantization result to ensure that important regions are retained with higher quality, while secondary regions can be compressed more highly to obtain an optimized quantization result. Perform entropy coding on the optimized quantization result to obtain the original data stream. Entropy coding is a key step in data compression, which reduces the amount of data through efficient coding. According to the scene switching information in the switching log, perform segmentation processing on the original data stream to obtain a segmented data stream. The segmentation processing divides the data stream so that the transmission strategy can be adjusted more flexibly when a scene switch occurs. Perform error recovery coding on the segmented data stream to obtain a compressed data stream. The error recovery coding improves the reliability of data transmission. By adding redundant information, it can be effectively recovered when an error occurs during transmission. According to the network condition and the switching log, calculate the transmission priority of the data packets to obtain a transmission scheduling strategy. The transmission scheduling strategy ensures that important data can be transmitted first through priority control, improving the overall transmission efficiency and stability. Perform packetization and encryption processing on the compressed data stream to obtain the data packets to be transmitted. The packetization process divides the large data stream into small data packets for easy transmission and management. The encryption processing ensures that the data is not stolen or tampered with during transmission, protecting the security of the data. According to the transmission scheduling strategy, generate transmission control parameters including transmission timing and retransmission mechanism. The transmission timing controls the sending order and time of the data packets, and the retransmission mechanism retransmits the data packets when they are lost to ensure data integrity and transmission reliability.

[0038] Step S106: Based on the compressed data stream and the transmission control parameters, perform signal demultiplexing and reconstruction, and output the target video signal.

[0039] Specifically, the data packets to be transmitted are sorted and integrity-checked according to the transmission control parameters to ensure that the data packets are arranged in the correct order and without loss, obtaining an ordered data packet sequence. The ordered data packet sequence is decrypted and unpacked to extract the compressed data stream. According to the error recovery information in the transmission control parameters, error detection and correction are performed on the compressed data stream to obtain the corrected data stream. By using redundant information to detect and repair possible errors during transmission, the integrity and accuracy of the data are ensured. Entropy decoding is performed on the corrected data stream to restore the original quantized data. Entropy decoding is a key step in data decompression and can convert the compressed bitstream back to quantized data. According to the quantized data and the preset inverse quantization matrix, inverse quantization operation is performed to obtain the frequency-domain coefficients. The frequency-domain coefficients are the representation of the image data in the frequency domain, and through the inverse quantization operation, their original values can be restored. Inverse transformation is performed on the frequency-domain coefficients to obtain the initially reconstructed video frame. Inverse transformation is the process of converting frequency-domain data back to spatial-domain data, thus restoring the original image structure of the video frame. According to the scene switching information in the transmission control parameters, the initially reconstructed video frame is segmented to obtain segmented video frames. Deblocking and deringing processing are performed on the segmented video frames to obtain optimized video frames. Deblocking processing can eliminate the blocky artifacts that appear during compression, while deringing processing can reduce the ringing effect at the image edges and improve the visual quality of the image. According to the optimized video frames and adjacent frame information, motion compensation and interpolation processing are performed to obtain a smoothly transitioning video sequence. Motion compensation and interpolation techniques can generate transitional frames based on the motion information of adjacent frames, making the video sequence smoother and more natural during scene switching and motion. The smoothly transitioning video sequence is transformed and corrected to output the target video signal. The transformation and correction process includes color space conversion, brightness and contrast adjustment, etc., to ensure that the finally output video signal meets the requirements of the display device and the visual habits of the audience.

[0040] In the embodiments of the present application, by performing preprocessing and feature calculation on multiple video signal channels, a signal feature vector set and a signal quality index are generated. Based on the signal features and quality index, adaptive frame rate control and energy optimization are performed, which can effectively improve the operation efficiency, reduce energy consumption, and at the same time ensure the quality of video processing. By performing multi-scale feature extraction and matching on video frames to generate a scene change mapping, the changes in video content can be captured more accurately. Based on the scene change mapping and the dynamic weight matrix, intelligent switching decisions and signal fusion are performed, which can achieve smoother and more natural picture transitions and enhance the viewing experience. Data compression and transmission optimization are performed on the video signal after transition processing, and the compression strategy is adjusted according to the switching log, which can improve the transmission efficiency and reduce the bandwidth pressure while ensuring the video quality. Through the optimized signal demultiplexing and reconstruction process, the original video signal can be restored quickly and accurately, ensuring the quality of the finally output video. The present application realizes the full-process automation from video signal preprocessing to final output, greatly improving the efficiency of multi-channel video signal processing and switching, and reducing the intervention of human operations and possible errors.

[0041] In a specific embodiment, the process of performing step S101 may specifically include the following steps:

[0042] (1) Perform spatial domain filtering processing on each video signal channel to obtain the denoised video signal, and perform color correction on the denoised video signal to obtain the color-corrected video signal;

[0043] (2) Perform resolution normalization processing on the color-corrected video signal to obtain the normalized video signal, and perform texture feature extraction on the normalized video signal to obtain the texture feature map;

[0044] (3) Calculate the motion intensity feature according to the texture feature map and the normalized video signal, and perform color histogram analysis and clustering on the normalized video signal to obtain the color distribution feature;

[0045] (4) Calculate the scene change frequency for the motion intensity feature and the color distribution feature to obtain the scene change feature;

[0046] (5) Perform discrete cosine transform and quantization on the normalized video signal and calculate the signal-to-noise ratio and the structural similarity index to obtain the signal quality index;

[0047] (6) Construct a multi-dimensional feature vector for the texture feature map, the motion intensity feature, the color distribution feature, and the scene change feature to obtain the signal feature vector set;

[0048] (7) Perform principal component analysis and fuzzy comprehensive evaluation on the signal feature vector set and the signal quality index to obtain the dynamic weight matrix.

[0049] Specifically, spatial domain filtering technology is applied to each video signal channel. Spatial domain filtering is a method of removing noise by operating on the pixel values of an image. Common filtering methods include mean filtering, median filtering, and Gaussian filtering. Taking Gaussian filtering as an example, its principle is to perform a convolution operation on the image through a Gaussian kernel, so that the noise in the image is smoothed. The formula for Gaussian filtering is as follows:

[0050]

[0051] where G(x,y) is the Gaussian kernel function, σ is the standard deviation, and x and y are the offsets relative to the center point. By performing Gaussian filtering on the video signal, the random noise in the image is effectively reduced, and the denoised video signal is obtained. Color correction is performed on the denoised video signal to adjust the color balance of the image, making the colors of the image more realistic and natural. Common color correction methods include white balance correction and gamma correction. White balance correction is to adjust the RGB channels of the image so that white objects in the image appear white. Gamma correction is to adjust the brightness distribution of the image to make the contrast of the image more appropriate. The formula is as follows:

[0052]

[0053] where I out is the pixel value of the corrected image, I i n is the pixel value of the original image, and γ is the gamma value. Through gamma correction, the brightness and contrast of the image can be improved, and the color-corrected video signal is obtained. Resolution normalization is performed on the color-corrected video signal to unify video signals with different resolutions to the same resolution for subsequent processing. Resolution normalization can be achieved through image interpolation methods. Common interpolation methods include bilinear interpolation and bicubic interpolation. Through resolution normalization processing, a standardized video signal is obtained. Texture feature extraction is performed on the standardized video signal. Texture features are important indicators for describing texture information in an image. Common texture feature extraction methods include gray-level co-occurrence matrix, local binary pattern (LBP), etc. Taking the gray-level co-occurrence matrix as an example, it extracts the texture features of the image by calculating the co-occurrence relationship of gray values in the image. The obtained texture feature map can be used for subsequent feature analysis. Motion intensity features are obtained by calculating the motion vector field based on the texture feature map and the standardized video signal. The motion vector field describes the motion information between video frames. Common methods include optical flow method and block matching method. The optical flow method calculates the motion vectors of image pixels to obtain motion intensity features. The formula is as follows:

[0054]

[0055] Among them, v is the motion vector, ΔI is the grayscale change of the image, and Δt is the time interval. Through the calculation of the motion vector field, the motion information of each pixel is obtained, and the motion intensity feature is obtained. At the same time, color histogram analysis and clustering are performed on the normalized video signal to obtain the color distribution feature. The color histogram is to statistically analyze the distribution of different colors in the image. Through color histogram analysis, the color feature of the image is obtained. Clustering algorithms, such as K-means clustering, are used to perform clustering analysis on the color histogram to obtain the main color distribution feature of the image. The scene change frequency is calculated for the motion intensity feature and the color distribution feature to obtain the scene change feature. The scene change frequency reflects the frequency of scene switching in the video signal, and can be calculated by analyzing the changes in the motion intensity feature and the color distribution feature. By calculating the scene change frequency, the scene change feature is obtained. The normalized video signal is subjected to discrete cosine transform and quantization, and the signal-to-noise ratio and structural similarity index are calculated to obtain the signal quality index. The discrete cosine transform converts the image from the spatial domain to the frequency domain for easy compression and feature extraction. The formula is as follows:

[0056]

[0057] Among them, F(u, v) is the frequency domain coefficient, I(x, y) is the spatial domain pixel value, and N is the image size. Through discrete cosine transform and quantization, the frequency domain coefficients of the image are obtained. The signal-to-noise ratio and structural similarity index are important indicators for evaluating image quality. The signal-to-noise ratio measures the ratio of the signal intensity to the noise intensity of the image, and the structural similarity index measures the structural similarity of the image. A multi-dimensional feature vector is constructed for the texture feature map, motion intensity feature, color distribution feature, and scene change feature to obtain a set of signal feature vectors. Principal component analysis and fuzzy comprehensive evaluation are performed on the set of signal feature vectors and the signal quality index to obtain the dynamic weight matrix. Principal component analysis simplifies the complexity of the feature vector through dimensionality reduction and feature extraction, and fuzzy comprehensive evaluation synthesizes various features to generate a dynamic weight matrix for intelligent switching decision-making and signal fusion.

[0058] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0059] (1) Compress the set of signal feature vectors to obtain compressed feature vectors, and construct a frame rate prediction model and calculate the frame rate prediction values of each video signal channel according to the compressed feature vectors and the signal quality index;

[0060] (2) Smooth the frame rate prediction values to obtain frame rate control parameters, and calculate the optimal operating frequency of the processor core according to the frame rate control parameters and the current hardware load to obtain a frequency adjustment instruction;

[0061] (3) Analyze the hardware resource usage information and predict future resource requirements to obtain the predicted resource requirement values. Then, optimize the switching strategy of the functional units based on the predicted resource requirement values and the frequency adjustment instructions to obtain the functional unit configuration plan.

[0062] (4) Analyze the historical scenario change data and identify typical scenario patterns to obtain the scenario category labels. Then, generate a preheating data list based on the scenario category labels and the current scenario characteristics.

[0063] (5) Sort the elements in the preheating data list by priority and calculate the optimal loading order to obtain the cache preheating sequence.

[0064] (6) Allocate hardware resources according to the cache preheating sequence and the functional unit configuration plan to obtain the optimized hardware working state and the preheated cache state.

[0065] Specifically, compress the signal feature vector set to obtain the compressed feature vector. The signal feature vector set is a multi-dimensional description of the video signal. Through compression processing, its dimension can be reduced, and the computational complexity can be decreased. Based on the compressed feature vector and the signal quality index, construct a frame rate prediction model. The frame rate prediction model predicts the frame rate of each video signal channel by analyzing historical data and current features. Common methods include linear regression and time series analysis. Assume that a linear regression model is used, and its formula is as follows:

[0066]

[0067] Among them, Y is the compressed feature vector, is the predicted frame rate, a is the regression coefficient, and b is the bias term. Through regression analysis, obtain the frame rate prediction values for each video signal channel. Smooth the frame rate prediction values to reduce prediction errors and fluctuations. A common smoothing method is the exponentially weighted moving average, and its formula is as follows:

[0068]

[0069] Among them, S t is the current frame rate control parameter, α is the smoothing coefficient, is the current predicted frame rate, and S t-1 is the smoothed value at the previous moment. Through smoothing processing, obtain a more stable frame rate control parameter. According to the frame rate control parameter and the current hardware load, calculate the optimal operating frequency of the processor core. Assume that the energy consumption of the processor is quadratic with the operating frequency, and its power consumption model can be expressed as:

[0070] P = kf 2 ;

[0071] Where P is the power consumption, k is a constant related to the processor characteristics, and f is the operating frequency. According to the frame rate control parameter and the hardware load, the frequency of the processor is optimized to obtain a frequency adjustment instruction. Analyze the hardware resource usage information and predict the future resource requirements. Resource requirement prediction can be achieved through time series analysis methods, such as the autoregressive integrated moving average model. After obtaining the resource requirement value through prediction, combined with the frequency adjustment instruction, optimize the switching strategy of the functional units to ensure the maximization of resource usage efficiency and obtain a functional unit configuration plan. Analyze the historical scenario change data and identify typical scenario patterns. Through clustering algorithms, such as K-means, the scenario change data can be divided into different categories to obtain scenario category labels. According to the current scenario characteristics and scenario category labels, generate a preheating data list. The preheating data list includes the data that needs to be preferentially loaded under different scenarios. Sort the elements in the preheating data list by priority and calculate the optimal loading order. Use the priority queue algorithm to sort the preheating data. According to the sorting result, form a cache preheating sequence. According to the cache preheating sequence and the functional unit configuration plan, allocate hardware resources. When allocating resources, it is necessary to consider the availability and priority of the resources to ensure that critical tasks are executed first, and obtain an optimized hardware working state and a preheated cache state.

[0072] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0073] (1) According to the optimized hardware working state, perform multi-scale decomposition on the video frame to obtain a multi-scale image sequence, and extract features from each scale image in the multi-scale image sequence to obtain a target point set and a descriptor set;

[0074] (2) According to the preheated cache state, load predefined features from the cache, filter the target point set, and obtain a filtered target point set;

[0075] (3) Optimize the distribution of the filtered target point set to obtain a uniformly distributed feature point set, and construct a feature database and generate a feature index structure according to the uniformly distributed feature point set and the descriptor set;

[0076] (4) Perform partition matching on the feature points between consecutive frames according to the feature database and the feature index structure to obtain an initial matching result, and generate a geometric consistency matching set according to the initial matching result;

[0077] (5) Calculate the motion information of the geometric consistency matching set to obtain a motion vector map, and perform region segmentation and analysis according to the motion vector map to obtain a scene change region;

[0078] (6) Encode and feature fuse the scene change region to obtain a scene change map.

[0079] Specifically, multi-scale decomposition is performed on video frames. By filtering and downsampling the image at different scales, a series of images with decreasing scales are generated. These images can represent different levels of detail of the original image. Common methods include the Gaussian pyramid and the Laplacian pyramid. Taking the Gaussian pyramid as an example, its formula is as follows:

[0080]

[0081] where I8(x, y) is the image at the l-th layer, w(i, j) is the Gaussian kernel function, k is the kernel size, and I 8-1(x, y) is the image of the (l - 1)-th layer. Through the Gaussian pyramid, a multi-scale image sequence is obtained, and each scale image contains different detail information of the original image. Feature extraction is performed on each scale image in the multi-scale image sequence to find the key points and their descriptors in the image, and these features are used for matching and tracking. Commonly used feature extraction methods include SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features). Taking SIFT as an example, its feature extraction process includes detecting the extreme points in the scale space, accurately positioning the key points, assigning directions, and generating descriptors, obtaining the target point set and descriptor set, and these feature points and descriptors describe the local features of the image at different scales. According to the warm-up cache status, predefined features are loaded from the cache, and the target point set is filtered to obtain the filtered target point set. Predefined features are features pre-stored in the cache based on previous experience and knowledge, which are used to accelerate the feature matching process. The purpose of filtering is to remove redundant or unimportant feature points and only retain those feature points that match the predefined features, improving the accuracy and efficiency of feature matching. Distribution optimization is performed on the filtered target point set to obtain a uniformly distributed feature point set. The purpose of distribution optimization is to ensure that the feature points are more evenly distributed in the image, avoiding the situation where feature points are too dense in some areas and sparse in other areas. A grid-based distribution optimization method can be used to divide the image into several grids, and a certain number of feature points are retained in each grid to achieve uniform distribution. Through optimization, a uniformly distributed feature point set is obtained. According to the uniformly distributed feature point set and descriptor set, a feature database is constructed and a feature index structure is generated. The feature database is used to store all the feature points and their descriptors, while the feature index structure is used to quickly search and match feature points. Commonly used index structures include k-d trees and FLANN (Fast Library for Approximate Nearest Neighbors), and these structures can efficiently perform feature matching and searching. According to the feature database and feature index structure, partition matching is performed on the feature points between consecutive frames to obtain an initial matching result. The purpose of partition matching is to improve the efficiency and accuracy of matching by dividing the image into several regions and performing feature matching within each region. Through the feature index structure, the nearest neighbor feature points of each feature point are quickly found to obtain the initial matching result. According to the initial matching result, a geometrically consistent matching set is generated. Geometrically consistent matching verifies the geometric relationship between feature points, excludes mis-matched feature points, and improves the accuracy of matching. Commonly used methods include RANSAC (Random Sample Consensus), which finds the largest consistent set through random sampling and iterative optimization to obtain the geometrically consistent matching set. Motion information calculation is performed on the geometrically consistent matching set to obtain a motion vector map. The motion vector map describes the motion information between video frames and can be obtained by calculating the displacement between the matching feature points. The formula is as follows:

[0082]

[0083] Among them, v i is the motion vector of the i-th feature point, and are the coordinates of the feature point at time t, and are the coordinates of the feature point at time t + 1. Through motion information calculation, the motion vector of each feature point is obtained. According to the motion vector map, region segmentation and analysis are carried out to obtain the scene change region. The purpose of region segmentation is to divide the image into several regions, and each region contains similar motion information. Commonly used methods include graph-based segmentation and clustering-based segmentation. Through these methods, the scene change region is obtained. The scene change region describes the parts in the image where significant changes occur and is the key to subsequent processing. Encoding and feature fusion are performed on the scene change region to obtain the scene change map. The purpose of encoding is to convert the information of the scene change region into a digital representation, while feature fusion combines multiple features to form a unified description of the scene change, and finally the scene change map is obtained.

[0084] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0085] (1) Perform region importance analysis on the scene change map to obtain the scene importance score, and calculate the comprehensive score of each video signal channel and generate a channel priority list according to the scene importance score and the dynamic weight matrix;

[0086] (2) Perform temporal smoothing processing on the channel priority list to obtain a switching candidate sequence, and generate an initial switching scheme according to the switching candidate sequence and the preset program structure rules;

[0087] (3) Perform conflict detection and adjustment on the initial switching scheme to obtain an optimized switching scheme, and select the corresponding video signal channel according to the optimized switching scheme and generate the source signal to be fused;

[0088] (4) Perform spatio-temporal alignment processing on the source signal to be fused to obtain the aligned signal, and calculate the best transition effect parameters according to the aligned signal to obtain the transition effect instruction;

[0089] (5) Execute the transition effect instruction through the aligned signal to obtain the video signal after transition processing, and record the execution process of the optimized switching scheme and the transition effect instruction to generate a switching log.

[0090] Specifically, perform region importance analysis on the scene change map to determine the importance of each region in the video frame. For example, methods based on visual attention models and semantic information are used. The method based on the visual attention model determines which regions are more likely to attract the audience's attention by calculating the saliency map of the image. The formula is as follows:

[0091]

[0092] where S(x, y) is the significance value, w i is the feature weight, f i (x, y) is the value of the i-th feature at the position (x, y), and N is the number of features. Through the significance analysis of the scene change mapping, the importance score of each region is obtained. According to the scene importance score and the dynamic weight matrix, the comprehensive score of each video signal channel is calculated. The dynamic weight matrix reflects the importance of different features and can be obtained by principal component analysis or fuzzy comprehensive evaluation method. The calculation formula of the comprehensive score is as follows:

[0093]

[0094] where C ； is the comprehensive score of the j-th video signal channel, W ；: is the weight value in the dynamic weight matrix, S ；: is the scene importance score of the k-th feature of the j-th channel, and M is the number of features. By calculating the comprehensive score, the importance of each video signal channel is obtained. According to the comprehensive score, a channel priority list is generated. The channel priority list is sorted in descending order of the comprehensive score, reflecting the priorities of each video signal channel. The channel priority list is smoothed in time series to reduce the fluctuation of the priority change. For example, the exponential weighted moving average method is used, and its formula is as follows:

[0095] P t = αC t + (1 - α)P t-1 ;

[0096] where P t is the smoothed priority at the current moment, α is the smoothing coefficient, C t is the comprehensive score at the current moment, P t-1is the smoothed priority at the previous moment. Through temporal smoothing processing, a more stable handover candidate sequence is obtained. According to the handover candidate sequence and the preset program structure rules, an initial handover scheme is generated. The program structure rules include specific requirements for handover frequency, duration, and scene type to ensure that the handover scheme conforms to the norms and styles of program production. The initial handover scheme needs to meet these rules and at the same time try to select video signal channels with high priority. Conflict detection and adjustment are performed on the initial handover scheme to ensure the executability and rationality of the scheme. The purpose of conflict detection is to discover potential resource conflicts or logical errors, such as two channels occupying the same resource simultaneously. The adjustment process is to resolve these conflicts, and an optimized handover scheme is obtained by reallocating resources or adjusting the handover order. According to the optimized handover scheme, the corresponding video signal channels are selected and the source signals to be fused are generated. The source signals to be fused are multiple video signals selected according to the handover scheme, and these signals need to be processed for spatio-temporal alignment to ensure the synchronization and consistency between the signals. Spatio-temporal alignment processing includes time alignment and spatial alignment. Time alignment is achieved by adjusting the frame rate and time stamps, and spatial alignment is achieved by adjusting the image position and size. According to the aligned signals, the optimal transition effect parameters are calculated. The transition effect parameters include the type, duration, and effect intensity of the transition. Through these parameters, specific transition effect instructions are obtained. The calculation of the transition effect parameters can be achieved by minimizing the transition error, and its formula is as follows:

[0097]

[0098] where E is the transition error, and are the pixel values of two frames before and after the transition respectively, and T is the number of transition frames. By minimizing the transition error, the optimal transition effect parameters are obtained. The transition effect instructions are executed through the aligned signals to obtain the video signals after transition processing. The video signals after transition processing should be smooth, natural, and of high quality, meeting the visual needs of the audience and the standards of program production. At the same time, the execution process of the optimized handover scheme and the transition effect instructions is recorded to generate a handover log. The handover log records information such as the time point of each signal handover, the signal source, and the transition effect.

[0099] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0100] (1) Perform spatio-temporal analysis on the video signals after transition processing to obtain a signal complexity map, and according to the signal complexity map and the handover log, divide the compression priority regions of the video frames to obtain a regional compression strategy;

[0101] (2) Analyze the dynamic quantization parameter matrix of the video signal to obtain the initial quantization result, and refine and adjust the initial quantization result according to the region compression strategy to obtain the optimized quantization result;

[0102] (3) Perform entropy coding on the optimized quantization result to obtain the original data stream, and segment the original data stream according to the scene switching information in the switching log to obtain the segmented data stream;

[0103] (4) Perform error recovery coding on the segmented data stream to obtain the compressed data stream, and calculate the transmission priority of the data packets according to the network condition and the switching log to obtain the transmission scheduling strategy;

[0104] (5) Perform packet splitting and encryption processing on the compressed data stream to obtain the data packets to be transmitted, and generate the transmission control parameters including the transmission timing and the retransmission mechanism according to the transmission scheduling strategy.

[0105] Specifically, perform spatio-temporal analysis on the video signal after transitional processing. Spatio-temporal analysis determines the complexity of the signal by evaluating the changes in the video signal in terms of time and space. The complexity map is an image used to represent the complexity level of the video signal at different positions and time points. By calculating the change amount and difference of each pixel, the complexity map can be obtained. The formula is as follows:

[0106] C(x,y,t) = |I(x,y,t) - I(x,y,t - 1)|;

[0107] Where, C(x,y,t) is the complexity value at the position (x,y) and time t, and I(x,y,t) is the pixel value at the position (x,y) and time t. Through the complexity map, the regions with larger changes in the video signal can be visually seen. According to the signal complexity map and the switching log, divide the compression priority regions of the video frames. The switching log records the detailed information of the video signal switching, including the switching time and the switching region. This information can help determine which regions require a higher compression priority. Based on the complexity map and the switching log, divide the video frames into different compression priority regions. The high-complexity regions and the frequently switched regions are given a higher compression priority, while the low-complexity regions and the less switched regions are given a lower compression priority. The obtained region compression strategy can effectively allocate compression resources and improve the compression efficiency of the video signal. Analyze the dynamic quantization parameter matrix of the video signal to obtain the initial quantization result. The dynamic quantization parameter matrix analysis balances the compression ratio and the image quality by adjusting the quantization parameters. The selection of the quantization parameters can significantly affect the compression result. The formula is as follows:

[0108]

[0109] Where, Q(i,j) is the quantization parameter at the position (i,j), QSTUV (i, j) is the basic quantization parameter, Q UWT8V (i, j) is the quantization scale factor. Through the analysis of the dynamic quantization parameter matrix, the initial quantization result is obtained. According to the region compression strategy, the initial quantization result is refined and adjusted. According to the compression priorities of different regions, the quantization parameters are optimized. Smaller quantization parameters are used in regions with high compression priority to retain more details; larger quantization parameters are used in regions with low compression priority to increase the compression ratio. After refinement and adjustment, the optimized quantization result is obtained. Entropy coding is performed on the optimized quantization result to obtain the original data stream. Entropy coding is a key step in data compression, which reduces the amount of data through efficient coding. Common entropy coding methods include Huffman coding and arithmetic coding. Through entropy coding, the quantized data is converted into a compact bit stream to obtain the original data stream. According to the signal complexity graph and the scene switching information in the switching log, the original data stream is segmented. Segmentation is to divide the original data stream into several small segments, each corresponding to a video frame or a scene switching point. This helps to flexibly adjust the data stream during transmission and improve the transmission efficiency. Error recovery coding is performed on the segmented data stream to obtain the compressed data stream. Error recovery coding adds redundant information to enable effective recovery when errors occur during transmission. Common methods include forward error correction (FEC) and cyclic redundancy check (CRC). Through error recovery coding, the reliability of data transmission is improved. According to the network condition and the switching log, the transmission priority of each data packet is calculated to obtain the transmission scheduling strategy. The transmission priority reflects the importance and urgency of each data packet. High-priority data packets will be transmitted first to ensure the timely delivery of important data. The transmission scheduling strategy includes the transmission timing and the retransmission mechanism to ensure the transmission order and retransmission strategy of data packets. The compressed data stream is packetized and encrypted to obtain the data packets to be transmitted. Packetization divides the large data stream into small data packets for easy transmission and management. Encryption ensures that the data is not stolen or tampered with during transmission, protecting the security of the data. Through packetization and encryption, the data packets to be transmitted are obtained. According to the transmission scheduling strategy, transmission control parameters including the transmission timing and the retransmission mechanism are generated. The transmission timing controls the sending order and time of data packets, and the retransmission mechanism retransmits the data packets when they are lost to ensure data integrity and transmission reliability. The transmission control parameters provide guidance and control for the transmission process.

[0110] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0111] (1) Sort and perform integrity check on the data packets to be transmitted according to the transmission control parameters to obtain an ordered data packet sequence;

[0112] (2) Decrypt and depacketize the ordered data packet sequence to obtain the compressed data stream;

[0113] (3) Perform error detection and correction on the compressed data stream according to the error recovery information in the transmission control parameters to obtain a corrected data stream;

[0114] (4) Perform entropy decoding on the corrected data stream to obtain quantized data, and perform an inverse quantization operation according to the quantized data and a preset inverse quantization matrix to obtain frequency domain coefficients;

[0115] (5) Perform an inverse transform on the frequency domain coefficients to obtain an initially reconstructed video frame, and perform segmentation processing on the initially reconstructed video frame according to the scene switching information in the transmission control parameters to obtain segmented video frames;

[0116] (6) Perform deblocking and deringing processing on the segmented video frames to obtain an optimized video frame;

[0117] (7) Perform motion compensation and interpolation processing according to the optimized video frame and adjacent frame information to obtain a smoothly transitioning video sequence, and perform conversion and correction on the smoothly transitioning video sequence to output a target video signal.

[0118] Specifically, the received data packets are sorted according to the transmission control parameters. These data packets may have their arrival order changed during transmission due to network latency, packet loss, etc. Therefore, they need to be re-sorted according to the timestamps or sequence numbers in the transmission control parameters to restore the original transmission order. Integrity checking is to verify the integrity of each data packet through a checksum or a hash function to ensure that the data packet has not been tampered with or damaged. Through these steps, an ordered and complete sequence of data packets is obtained. The ordered data packet sequence is decrypted and unpacked to obtain the compressed data stream. Data packets are usually encrypted during transmission to protect the privacy and security of the data. The received data packets need to be decrypted. The decryption algorithm can be a symmetric encryption algorithm such as AES, or an asymmetric encryption algorithm such as RSA. The decrypted data packets need to be further unpacked to extract the actual compressed data stream. According to the error recovery information in the transmission control parameters, error detection and correction are performed on the compressed data stream to obtain the corrected data stream. Error detection and correction are achieved through forward error correction (FEC) or cyclic redundancy check (CRC). FEC adds redundant information to the data stream so that the receiving end can detect and correct errors that occur during transmission. CRC detects errors in the data stream by calculating the checksum. Through these steps, the errors in the data stream are corrected to obtain the corrected data stream. Entropy decoding is performed on the corrected data stream to obtain the quantized data. Entropy decoding is a key step in data decompression, which converts the compressed bit stream back to the original quantized data. Common entropy decoding methods include Huffman decoding and arithmetic decoding. Through entropy decoding, the quantized data before compression is restored. According to the quantized data and the preset inverse quantization matrix, an inverse quantization operation is performed to obtain the frequency domain coefficients. Quantization is part of data compression. By quantizing the data, the representation precision of the data is reduced, thereby reducing the amount of data. Inverse quantization is the inverse process of quantization. By using the inverse quantization matrix, the original frequency domain coefficients are restored. The formula is as follows:

[0119] D(i,j) = Q(i,j) × Z(i,j);

[0120] Among them, D(i, j) is the frequency-domain coefficient after inverse quantization, Q(i, j) is the quantization data, and Z(i, j) is the corresponding element in the inverse quantization matrix. An inverse transform is performed on the frequency-domain coefficients to obtain the initially reconstructed video frame. The inverse transform is a process of converting frequency-domain data back to spatial-domain data. Common methods include the Inverse Discrete Cosine Transform (IDCT) and the Inverse Discrete Wavelet Transform (IDWT). Through the inverse transform, the original video frame is reconstructed. According to the scene switching information in the transmission control parameters, the initially reconstructed video frame is segmented to obtain segmented video frames. The scene switching information records the time points and regions of scene changes in the video signal. Using this information, the initially reconstructed video frame is divided into several segments, each corresponding to a scene change region. Deblocking and deringing processing are performed on the segmented video frames to obtain optimized video frames. Deblocking processing improves the image quality by eliminating the blocky artifacts generated during video compression. Common methods include low-pass filtering and adaptive filtering. Deringing processing improves the visual quality of the image by reducing the ringing effect at the image edges. Through these processes, optimized video frames are obtained. According to the optimized video frames and adjacent frame information, motion compensation and interpolation processing are performed to obtain a smoothly transitioning video sequence. Motion compensation reduces motion blur by calculating the motion vectors between adjacent frames, and interpolation processing smooths the video sequence by generating transitional frames. The formula is as follows:

[0121] P(x, y, t) = 21 - α5P2x - v " , y - v y , t - 15 + αP(x, y, t);

[0122] Among them, P(x, y, t) is the pixel value at position (x, y) and time t, v " and v y are the motion vectors in the x and y directions, and α is the interpolation coefficient. Through motion compensation and interpolation processing, a smoothly transitioning video sequence is obtained. The smoothly transitioning video sequence is converted and corrected to output the target video signal. The conversion and correction process includes color space conversion, brightness and contrast adjustment, etc., to ensure that the finally output video signal meets the requirements of the display device and the visual habits of the audience.

[0123] The above described the multi-channel video signal automatic switching method based on a production switcher in the embodiments of the present application. Next, the multi-channel video signal automatic switching device based on a production switcher in the embodiments of the present application will be described. Please refer to Figure 2 , an embodiment of the multi-channel video signal automatic switching device based on a production switcher in the embodiments of the present application includes:

[0124] The preprocessing module 201 is used to preprocess multiple video signal channels and calculate features, generate a set of signal feature vectors and signal quality indicators, and construct a corresponding dynamic weight matrix;

[0125] The optimization module 202 is used to perform adaptive frame rate control and energy optimization based on the set of signal feature vectors and signal quality indicators, and obtain an optimized hardware working state and warm-up cache state;

[0126] The matching module 203 is used to perform multi-scale feature extraction and matching on video frames according to the optimized hardware working state and warm-up cache state, and generate a scene change map;

[0127] The fusion module 204 is used to perform intelligent switching decision-making and signal fusion based on the scene change map and the dynamic weight matrix, and obtain a video signal after transition processing and a switching log;

[0128] The generation module 205 is used to perform data compression and transmission optimization on the video signal after transition processing, and adjust the compression strategy according to the switching log, and generate a compressed data stream and transmission control parameters;

[0129] The output module 206 is used to perform signal demultiplexing and reconstruction based on the compressed data stream and transmission control parameters, and output the target video signal.

[0130] Through the collaborative cooperation of the above-mentioned components, by preprocessing multiple video signal channels and calculating features, generating a set of signal feature vectors and signal quality indicators, and performing adaptive frame rate control and energy optimization based on the signal features and quality indicators, the operating efficiency can be effectively improved, the energy consumption can be reduced, and at the same time, the quality of video processing can be ensured. By performing multi-scale feature extraction and matching on video frames to generate a scene change map, the changes in video content can be captured more accurately. Based on the scene change map and the dynamic weight matrix, performing intelligent switching decision-making and signal fusion can achieve a smoother and more natural picture transition and enhance the viewing experience. Performing data compression and transmission optimization on the video signal after transition processing and adjusting the compression strategy according to the switching log can improve the transmission efficiency and reduce the bandwidth pressure while ensuring the video quality. Through the optimized signal demultiplexing and reconstruction process, the original video signal can be restored quickly and accurately, ensuring the quality of the finally output video. This application realizes the full-process automation from video signal preprocessing to final output, greatly improves the efficiency of multi-channel video signal processing and switching, and reduces the intervention of manual operations and possible errors.

[0131] The present application also provides a multi-channel video signal automatic switching device based on a vision mixer. The multi-channel video signal automatic switching device based on a vision mixer includes a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the steps of the multi-channel video signal automatic switching method in the above-mentioned respective embodiments.

[0132] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on a computer, the computer is caused to execute the steps of the multi-channel video signal automatic switching method.

[0133] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, systems, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0135] As described above, the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing respective embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A multi-channel video signal automatic switching method based on a director switcher, characterized in that: The method comprises: Preprocess and calculate features of multiple video signal channels, generate signal feature vector sets and signal quality indicators, and construct corresponding dynamic weight matrices; Based on the signal feature vector set and the signal quality indicator, performing adaptive frame rate control and energy optimization to obtain an optimized hardware working state and a preheating cache state; According to the optimized hardware working state and the preheated cache state, multi-scale feature extraction and matching are performed on the video frame to generate a scene change map; Based on the scene change mapping and the dynamic weight matrix, intelligent switching decision and signal fusion are performed to obtain a video signal and a switching log after transition processing; specifically including: performing regional importance analysis on the scene change mapping to obtain a scene importance score, and calculating the comprehensive score of each video signal channel and generating a channel priority list based on the scene importance score and the dynamic weight matrix; performing time-series smoothing processing on the channel priority list to obtain a switching candidate sequence, and generating an initial switching plan based on the switching candidate sequence and a preset program structure rule; performing conflict detection and adjustment on the initial switching plan to obtain an optimized switching plan, and selecting a corresponding video signal channel and generating a source signal to be fused based on the optimized switching plan; performing time-space alignment processing on the source signal to be fused to obtain an aligned signal, and calculating the optimal transition effect parameter based on the aligned signal to obtain a transition effect instruction; executing the transition effect instruction through the aligned signal to obtain a video signal after transition processing, and recording the execution process of the optimized switching plan and the transition effect instruction to generate a switching log; Performing data compression and transmission optimization on the video signal after the transition processing, and adjusting the compression strategy according to the switching log to generate a compressed data stream and transmission control parameters; Signal demultiplexing and reconstruction are performed based on the compressed data stream and the transmission control parameters, and a target video signal is output.

2. The method for automatically switching multi-channel video signals based on a director switcher according to claim 1, characterized in that: The preprocessing and feature calculation of multiple video signal channels, generating a signal feature vector set and a signal quality index, and constructing a corresponding dynamic weight matrix include: Performing spatial domain filtering processing on each video signal channel to obtain a denoised video signal, and performing color correction on the denoised video signal to obtain a color-corrected video signal; Performing resolution standardization processing on the color-corrected video signal to obtain a standardized video signal, and performing texture feature extraction on the standardized video signal to obtain a texture feature map; Performing motion vector field calculation based on the texture feature map and the standardized video signal to obtain motion intensity features, and performing color histogram analysis and clustering on the standardized video signal to obtain color distribution features; Calculating the scene change frequency of the motion intensity feature and the color distribution feature to obtain a scene change feature; Performing discrete cosine transformation and quantization on the standardized video signal and calculating the signal-to-noise ratio and the structural similarity index to obtain a signal quality index; Constructing a multi-dimensional feature vector for the texture feature map, the motion intensity feature, the color distribution feature, and the scene change feature to obtain a signal feature vector set; The signal feature vector set and the signal quality index are subjected to principal component analysis and fuzzy comprehensive evaluation to obtain a dynamic weight matrix.

3. The method for automatically switching multi-channel video signals based on a director switcher according to claim 1, characterized in that: The method of performing adaptive frame rate control and energy optimization based on the signal feature vector set and the signal quality indicator to obtain an optimized hardware working state and a preheating cache state includes: Compressing the signal feature vector set to obtain a compressed feature vector, and constructing a frame rate prediction model and calculating a frame rate prediction value of each video signal channel according to the compressed feature vector and the signal quality index; Smoothing the frame rate prediction value to obtain a frame rate control parameter, and calculating the optimal operating frequency of the processor core according to the frame rate control parameter and the current hardware load to obtain a frequency adjustment instruction; Analyze the hardware resource usage information and predict future resource demand to obtain a resource demand prediction value, and optimize the switch strategy of the functional unit according to the resource demand prediction value and the frequency adjustment instruction to obtain a functional unit configuration plan; Analyze historical scene change data and identify typical scene patterns to obtain scene category labels, and generate a preheating data list based on the scene category labels and current scene features; Prioritize and calculate the optimal loading order of the elements in the preheating data list to obtain a cache preheating sequence; According to the cache preheating sequence and the functional unit configuration scheme, hardware resources are allocated to obtain an optimized hardware working state and a preheating cache state.

4. The method for automatically switching multi-channel video signals based on a director switcher according to claim 1, characterized in that: The step of extracting and matching multi-scale features of video frames according to the optimized hardware working state and the preheating cache state to generate a scene change map includes: According to the optimized hardware working state, the video frame is decomposed into multiple scales to obtain a multi-scale image sequence, and features are extracted for each scale image in the multi-scale image sequence to obtain a target point set and a descriptor set; According to the preheating cache state, predefined features are loaded from the cache, and the target point set is screened to obtain a screened target point set; Performing distribution optimization on the screened target point set to obtain a uniformly distributed feature point set, and constructing a feature database and generating a feature index structure based on the uniformly distributed feature point set and the descriptor set; Performing partition matching on feature points between consecutive frames according to the feature database and the feature index structure to obtain an initial matching result, and generating a geometric consistency matching set according to the initial matching result; Calculating motion information of the geometric consistency matching set to obtain a motion vector map, and performing region segmentation and analysis based on the motion vector map to obtain a scene change region; The scene change region is encoded and feature-fused to obtain a scene change map.

5. The method for automatically switching multi-channel video signals based on a director switcher according to claim 1, characterized in that: The step of compressing and optimizing the transmission of the video signal after the transition processing, adjusting the compression strategy according to the switching log, and generating a compressed data stream and transmission control parameters includes: Performing spatiotemporal analysis on the video signal after the transition processing to obtain a signal complexity map, and dividing the compression priority areas of the video frame according to the signal complexity map and the switching log to obtain a regional compression strategy; Performing dynamic quantization parameter matrix analysis on the video signal to obtain an initial quantization result, and refining and adjusting the initial quantization result according to the regional compression strategy to obtain an optimized quantization result; Performing entropy coding on the optimized quantization result to obtain an original data stream, and performing segmentation processing on the original data stream according to the scene switching information in the switching log to obtain a segmented data stream; Performing error recovery coding on the segmented data stream to obtain a compressed data stream, and calculating the transmission priority of the data packet according to the network status and the switching log to obtain a transmission scheduling strategy; The compressed data stream is packetized and encrypted to obtain data packets to be transmitted, and transmission control parameters including transmission timing and retransmission mechanism are generated according to the transmission scheduling strategy.

6. The method for automatically switching multi-channel video signals based on a director switcher according to claim 5, characterized in that: The signal demultiplexing and reconstruction based on the compressed data stream and the transmission control parameter to output the target video signal includes: Sorting and integrity checking the data packets to be transmitted according to the transmission control parameters to obtain an ordered sequence of data packets; Decrypting and unpacking the ordered data packet sequence to obtain the compressed data stream; performing error detection and correction on the compressed data stream according to the error recovery information in the transmission control parameter to obtain a corrected data stream; Performing entropy decoding on the modified data stream to obtain quantized data, and performing an inverse quantization operation according to the quantized data and a preset inverse quantization matrix to obtain frequency domain coefficients; Performing an inverse transformation on the frequency domain coefficients to obtain an initial reconstructed video frame, and performing segmentation processing on the initial reconstructed video frame according to the scene switching information in the transmission control parameter to obtain segmented video frames; Performing deblocking and de-ringing processing on the segmented video frames to obtain optimized video frames; According to the optimized video frame and adjacent frame information, motion compensation and interpolation processing are performed to obtain a video sequence with a smooth transition, and the video sequence with a smooth transition is converted and corrected to output a target video signal.

7. A multi-channel video signal automatic switching device based on a director switcher, characterized in that: The device is used to execute the multi-channel video signal automatic switching method based on the director switcher according to any one of claims 1 to 6, the device comprising: A preprocessing module is used to preprocess and calculate features of multiple video signal channels, generate a signal feature vector set and a signal quality index, and construct a corresponding dynamic weight matrix; An optimization module, configured to perform adaptive frame rate control and energy optimization based on the signal feature vector set and the signal quality index, to obtain an optimized hardware working state and a preheating cache state; A matching module, configured to extract and match multi-scale features of video frames according to the optimized hardware working state and the preheating cache state, and generate a scene change map; A fusion module, used to perform intelligent switching decision and signal fusion based on the scene change mapping and the dynamic weight matrix to obtain a transition-processed video signal and a switching log; A generating module, used for performing data compression and transmission optimization on the video signal after the transition processing, adjusting the compression strategy according to the switching log, and generating a compressed data stream and transmission control parameters; The output module is used to perform signal demultiplexing and reconstruction based on the compressed data stream and the transmission control parameters, and output a target video signal.

8. A multi-channel video signal automatic switching device based on a director switcher, characterized in that: The multi-channel video signal automatic switching device based on the director switcher includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory so that the multi-channel video signal automatic switching device based on the director switcher executes the multi-channel video signal automatic switching method based on the director switcher as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the multi-channel video signal automatic switching method based on the director switcher as described in any one of claims 1-6 is implemented.

Images