Signal processing method for remote video communication

Abstract

The invention proposes a signal processing method for remote video communication, and relates to the technical field of signal processing, and the method specifically comprises the steps: obtaining a communication signal and link information, and carrying out the preprocessing of the information into a data set; constructing a coupling topology based on the time-delay transmission coupling fractional flexibility and the optimal transmission constraint, and generating fractional flexibility features; obtaining a geometric irregularity state vector by using the non-local fractional graph curvature, the optimal transmission Richi curvature and the time lag torsion amount; a preprocessing gating factor is introduced to realize anisotropic gating scaling, and a gating state sequence is formed; constructing factorization measurement, a potential function and a transmission divergence weight, calculating an action quantity and a local index, and adaptively adjusting entropy risk aggregation to obtain a time sequence anomaly index; multi-interval decoding is carried out through robust threshold and time period aggregation, abnormal intervals and intensity are output, and a communication signal processing result is obtained; and finally, establishing an end-to-end model, and training to convergence to realize robust optimization and exception handling of the remote video communication signal.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of signal processing, and in particular to a signal processing method for remote video communication. BACKGROUND

[0002] At present, remote video communication plays an important role in online meetings, remote education, emergency command and remote medical scenarios, and its end-to-end experience is directly affected by multiple factors such as network link fluctuations, codec adaptation and terminal computing power constraints. In actual operation, video streams under heterogeneous networks and complex business loads present strong time-varying, non-stationary and multi-scale spatio-temporal statistical characteristics: there are packet loss, error code, jitter and time delay mutation on the link side, there are code control switching, layering and reference frame dependence on the application side, and the signal side presents structural response driven by motion vectors, transform residuals and quantization parameters. These factors are intertwined to cause picture quality degradation to present complex forms such as cross-segment propagation, regional mutation and multi-source coupling, which puts higher requirements on traditional processing methods that only rely on a single quality indicator and local time domain characteristics.

[0003] Publication No. CN105847788B discloses a video quality estimation technology based on code stream information, which objectively predicts video quality using bit stream layer features without reference to the source, and is suitable for quality monitoring at the decoding end and network side. Publication No. CN109302603A proposes a video call quality evaluation method and device, which acquires video parameters and audio parameters when detecting a video call, evaluates video quality and audio quality, and then fuses to obtain overall call quality, which is used for online evaluation needs in VoLTE scenarios.

[0004] However, the existing technology in the remote video communication scene relies on single-frame local statistics and simple weighting, which is difficult to reflect the associated changes across time segments, resulting in insufficient perception of long-term degradation. Quality evaluation often focuses on a single objective indicator and shallow code stream features, and fails to simultaneously depict the coupling relationship between link disturbance, coding structure and signal response, and is sensitive to sudden bandwidth fluctuations, code control switching and retransmission complexity. SUMMARY

[0005] The application provides a signal processing method for remote video communication, aiming at picture quality mutation, unstable structural response and multi-region concurrent degradation identification difficulty under complex network and service fluctuation, and constructs a robust thresholding and time period aggregation mechanism based on time series anomaly index and flexible irregular index and non-local fractional graph curvature irregularity detection and time delay transmission coupling fractional flexibility modeling: firstly, a coupling topological structure is constructed based on time delay transmission coupling fractional flexibility and optimal transmission constraint term, and a fractional flexibility feature sequence is generated; then, taking fractional graph Laplace curvature and optimal transmission Ricci curvature as cores, a geometric irregularity state vector is formed by combining time delay direction torsion; a pretreatment gating factor is introduced to anisotropically gate and scale the vector to obtain a gated state sequence; then, a factorized measure, a potential function and a transmission divergence weight are constructed, an action density is calculated, a local index is calculated, an entropy risk aggregation parameter is adaptively adjusted, and a time series anomaly index is obtained; finally, the robust thresholding and time period aggregation based on the time series anomaly index are adopted, a multi-region signal anomaly set and intensity are output, and a signal processing result of the remote video communication is obtained.

[0006] A signal processing method for remote video communication, and the specific method is: S1, acquiring signal, link and coding and decoding structure information of remote video communication and preprocessing, and constructing remote video communication signal dataset; S2, according to the dataset, setting a local window length, generating an overlapping window set, calculating a time delay correlation coefficient of adjacent time indexes, and obtaining a window pair transmission divergence under the entropy regular optimal transmission constraint, constructing a coupling topological structure according to the correlation coefficient and the transmission divergence, and performing fractional order flexibility mapping to obtain a fractional flexibility feature sequence; S3, according to the fractional flexibility feature sequence and the coupling topological structure, calculating a fractional graph Laplace curvature, based on optimal transmission aggregation Ricci curvature, and through forward and backward non-local gradient, a time delay direction torsion is obtained, the above results are combined, and a geometric irregularity state vector is obtained; S4, according to the state vector and the link information, introducing a pretreatment gating factor, extracting a candidate factor, mapping to obtain an anisotropic gating weight, scaling the state vector at the component level according to the gating weight, and obtaining a gated state sequence; S5, according to the gated state sequence and the pretreatment gating factor, constructing a factorized measure, a potential function and a transmission divergence weight, calculating an action density and generating a local flexible irregular index, adaptively setting an entropy risk aggregation parameter according to the working condition, windowing and time aggregating the index, and obtaining a time series anomaly index; S6, performing index smoothing, robust thresholding and hysteresis gate decision on the time series anomaly index, performing time period aggregation, obtaining a multi-interval anomaly set intensity, and outputting a remote video communication signal processing result through cross-channel fusion and mapping modulation; S7, constructing a signal processing model for remote video communication, inputting a remote video communication signal data set, sequentially passing through the above steps, and completing optimization processing of signals for remote video communication through iterative training until convergence.

[0007] Preferably, in the S1 step, for the construction of the remote video communication signal data set, first, the frame-level image sequence, the encoded bit stream and the decoded reconstructed frame of the sending end and the receiving end are synchronously collected under different working conditions of steady state, code-controlled switching, bandwidth fluctuation and link disturbance, and the link side running information and the encoding and decoding structure information are recorded at the same time, and the working condition label, channel and sub-channel identification and spatial position identification are attached to each time point; then the multi-source data is standardized in format and aligned in uniform time reference, denoising, normalization, missing compensation and abnormal value suppression are completed, and a one-to-one alignment mapping between the reference state and the actual state is established when the reference transmission condition is available; on this basis, the sequence is organized by frame and time period, the motion vector, the transform coefficient, the compressed residual error, the inter-frame prediction error signal and the time-frequency envelope derived quantity corresponding to the bit stream are generated, and are bound with the link side and the encoding and decoding side metadata; finally, the frame-level and segment-level features, time index, reference, actual mapping, working condition and quality labeling information are compiled and checked to form a remote video communication signal data set for subsequent time lag transmission coupling fractional flexibility modeling, non-local fractional graph curvature irregularity detection, flexible irregularity index calculation and multi-region decoding processing.

[0008] Preferably, in the S2 step, the local window length, the sliding step and the maximum time lag threshold are set, the remote video communication signal is divided by overlapping according to the time index, and a corresponding local window sample set is established for each time index; According to the local window sample set, the normalized correlation of any two adjacent time indexes in a preset time lag range is calculated, the maximum value of the correlation under each time lag is taken as the time lag correlation coefficient of the two, and a time lag correlation coefficient set is obtained; According to the sample distance between any two local windows, an optimal transport constraint term with entropy regularization is constructed, and the local transmission divergence of window pairing is obtained by optimal transport solution, and a transmission constraint quantity set is obtained; Based on the time lag correlation coefficient and the optimal transport constraint quantity set, the coupling weight between the time index pairs is jointly determined, the threshold is processed to be sparse, and a coupling topological structure is constructed. A regular operator with time smoothing and shape constraint is established on the coupling topological structure, fractional flexibility mapping and feature extraction are performed, and a fractional flexibility feature sequence of the actual state is obtained.

[0009] Further, in order to solve the problems of complex cross-frame association, arrival timing misalignment and structure response drift of remote video communication signals under multiple working conditions, the application proposes a modeling mechanism based on time delay transmission coupling fractional flexibility: first, the communication signals are overlapped and divided according to time index to form a local window sample set; the transient state and cross-frame association can be covered at the same time, and the perception of sudden disturbance and slow drift is enhanced; then, the normalized correlation of the first-order adjacent time index is calculated within the preset time delay range, and the maximum correlation value under each time delay is taken as the time delay correlation coefficient; the influence of arrival timing misalignment and buffer jitter on correlation is explicitly described, and the sensitivity to time delay disturbance is improved; then, the optimal transmission constraint term with entropy regularization is constructed according to the sample distance between any two windows, and the local transmission divergence is solved; the matching robustness is maintained in the scene of packet loss, retransmission and structure mutation, and the outlier interference is suppressed; on this basis, the coupling weight of time index pair is determined by combining the time delay correlation coefficient and the optimal transmission constraint term, and the coupling topological structure is constructed by threshold sparsification; the real time delay-transmission coupling neighborhood is highlighted, the long-distance weak association and pseudo-connection are weakened, and the interpretability and computability of the graph structure are improved; finally, the time smoothing and shape constraint regularization term is set on the coupling topological structure, the time delay-transmission coupling fractional flexibility mapping and feature extraction are performed, and the fractional flexibility feature sequence is generated; while preserving the boundary response, the noise immunity and continuity are improved, providing a stable input for subsequent non-local fractional graph curvature irregularity detection.

[0010] Preferably, in S3 step, a graph Laplacian is constructed based on the coupling topological structure, and a fractional graph Laplacian operation is performed on the fractional flexibility feature sequence according to a preset fractional order, to obtain a fractional graph Laplacian curvature response. According to the coupling topological structure, a probability distribution is constructed in the local neighborhood, the distance contraction degree of any adjacent time index pair is calculated using the optimal transport cost, the optimal transport Ricci curvature is obtained, and the node-level Ricci curvature is aggregated. Under the constraint of the coupling topological structure, non-local gradients are calculated in the time forward and backward directions respectively, and the time delay direction twist is obtained according to the difference between the forward and backward directions. The fractional graph Laplacian curvature, the Ricci curvature based on optimal transport and the time delay direction twist are subjected to robust scaling and scale normalization processing. The fractional graph Laplacian curvature, the aggregated Ricci curvature and the time delay direction twist subjected to robust scaling are vectorized and combined to obtain a geometric irregularity state vector.

[0011] Further, in the S3 step, for the curvature comparability and robustness of the fractional flexibility feature sequence under noise and scale difference, the application proposes a processing mechanism based on non-local fractional graph curvature irregularity detection: first, according to the coupling topological structure and the fractional flexibility feature sequence, time reference alignment and amplitude normalization are performed to obtain a flexibility sequence for curvature analysis; clock drift and amplitude scale difference can be eliminated to ensure cross-section comparability and stability; then, a graph Laplacian is constructed based on the coupling topological structure, and a fractional graph Laplacian operation is performed on the flexibility sequence according to a preset fractional order to obtain a fractional graph Laplacian curvature response; the sensitivity to weak boundaries and multi-scale deformation can be improved, and high-frequency noise interference can be suppressed; subsequently, a probability distribution is constructed in a local neighborhood according to the coupling topological structure, the distance contraction degree of adjacent time index pairs is calculated using optimal transport cost, the optimal transport-based Ricci curvature is obtained, and the node-level Ricci curvature is aggregated; the geometric contraction and expansion can be robustly measured under packet loss and jitter conditions, and the topological consistency is enhanced; on this basis, the non-local gradients are calculated in the time forward and backward directions under the constraint of the coupling topological structure, and the time delay direction twist is obtained according to the difference between the forward and backward directions; the propagation direction and time delay bias can be explicitly represented, and the time resolution of abnormal positioning is improved; then, the fractional graph Laplacian curvature, the optimal transport-based Ricci curvature, and the time delay direction twist are robustly scaled and scaled; the dimensions can be unified, extreme values can be suppressed, and the stability of the composite index can be improved; finally, the robustly scaled fractional graph Laplacian curvature, the aggregated Ricci curvature, and the time delay direction twist are vectorized and combined to obtain a geometric irregularity state vector; shape, transport, and time delay information can be fused in the same representation space to provide discriminative input for subsequent preprocessing gating factors and flexible irregularity indexes.

[0012] Preferably, in the S4 step, according to the geometric irregularity state vector and link information, a preprocessing gating factor is introduced, and a candidate factor and a geometric side index are extracted; The candidate factor is time-aligned and interval-normalized to construct an embedding factor vector; The embedding factor vector is mapped to an anisotropic gating weight, and a time-continuous gating weight sequence is obtained by sliding window smoothing; The gating weight is used to scale the geometric irregularity state vector at the component level to obtain a gating state sequence.

[0013] Further, in the S4 step, in order to solve the problems of link indicators and signal abnormalities out of synchronization, short and sharp peaks easy to amplify, and high false positives caused by link jitter, packet retransmission, and code control switching in remote video communication, the application proposes an anisotropic gating mechanism based on pre-processing gating factors: first, according to the geometric irregularity state vector and link information, a pre-processing gating factor is introduced, and candidate factors and geometric side indicators are extracted; it can reflect link side disturbance and geometric side abnormal evidence at the same time, avoid single source distortion and improve context consistency; then, the candidate factors are time-aligned, denoised and interval-normalized, and the input factor vector is constructed by combining and constructing in a preset order; it can eliminate sampling asynchronization and dimension difference, stabilize factor fluctuation and improve cross-section comparability; subsequently, the input factor vector is mapped to anisotropic gating weights, and the time-continuous gating weight sequence is obtained by sliding window smoothing; it can suppress short-time peaks caused by retransmission and transient congestion, and retain the discriminability of persistent abnormalities; finally, the geometric irregularity state vector is scaled at the component level by the gating weight to obtain a gated state sequence; without changing the feature symbol and relative order, it highlights high-confidence abnormalities and weakens low-confidence noise, providing higher signal-to-noise ratio and stronger explainability for subsequent flexible irregularity index input.

[0014] Preferably, in the S5 step, according to the gated state sequence and the pre-processing gating factor, a factorized measure, a potential function, and a transmission divergence weight are constructed; Under the constraints of the factorized measure and the potential function, the action density of the gated state sequence is calculated, and the local flexible irregularity index is generated in combination with the directional vector; According to the factor and the working condition, an entropy risk aggregation parameter is set adaptively, the local index is windowed and time-aggregated to obtain a time series anomaly index.

[0015] Further, in the S5 step, in order to solve the problems of abnormal energy cross-section diffusion, unclear direction and unstable exponential scale in remote video communication, the application proposes a measurement constraint mechanism: first, according to the gating state sequence and the preprocessed gating factor, a factorized measurement, a potential function and a transmission divergence weight are constructed; the sensitivity can be adaptively adjusted in the geometric and transmission domains, noise amplification is suppressed and the matching to the working condition change is improved; then, the action density of the gating state sequence is calculated under the constraint of the factorized measurement and the potential function, and a locally flexible irregular exponent is generated in combination with the direction vector; the direction mutation induced by link disturbance can be highlighted, and the interference of isotropic fluctuations on the exponent can be weakened; then, the entropy risk aggregation parameter is adaptively set according to the factor and the working condition, the locally flexible irregular exponent is windowed and time-aggregated to obtain a time series anomaly exponent; while ensuring the timing positioning accuracy, the robustness to outliers and short sharp peaks is improved, and the false alarm rate and the missed detection rate are considered; finally, the time series anomaly exponent is uniformly scaled and the confidence threshold is calibrated; it is convenient for subsequent multi-region decoding based on Wasserstein robust segmentation and topological persistence to directly call, and an interpretable and comparable anomaly evidence sequence is formed.

[0016] Preferably, in the S6 step, according to the time series anomaly index, the signal is exponentially smoothed to obtain a robust potential intensity sequence for interval detection; Based on robust thresholding and hysteresis gate decision, the robust potential intensity sequence is interval aggregated and cleaned to obtain a multi-region signal anomaly set strength; The multi-region anomaly set is fused and mapped across channels to output a remote video communication signal processing result.

[0017] Preferably, in the S6 step, in order to solve the problems of abnormal region boundary fragmentation, segmentation result sensitive to distribution drift and insufficient cross-channel consistency in remote video communication, the application proposes a robust thresholding and time period aggregation mechanism based on the time series anomaly index: first, the signal is exponentially smoothed according to the time series anomaly index to obtain a robust potential intensity sequence for segmentation; it can suppress noise and outliers while retaining mutation edges and region connectivity, providing a stable segmentation prior; then, robust thresholding and time period aggregation of the time series anomaly index are adopted; it can adapt to link conditions and content distribution changes, and improve the robustness to scale differences and class imbalance; then, the candidate regions are screened and merged according to the topological persistence criterion, short-lived false regions are removed and structures that stably exist in multiple scales are retained, reducing over-segmentation and under-segmentation; finally, the cross-channel fusion and mapping modulation are carried out for the obtained multi-region signal anomaly set and its strength score to generate a segment-level communication quality estimate value, and a remote video communication signal processing result is output; while ensuring the timing continuity, the cross-channel consistency is enhanced, and an interpretable quantitative basis is provided for alarm, code control and link rollback.

[0018] Preferably, in the S7 step, for the multi-source coupling degradation and cross-section diffusion characteristics presented by packet loss, jitter and code-controlled switching in remote video communication, an end-to-end signal processing model for remote video communication is constructed, and the coupling topology coding, curvature irregularity forward, anisotropic gating state sequence, transmission action quantity index, and time series anomaly index robust thresholding and time section aggregation stages are sequentially executed: first, the remote video communication signal dataset is taken as input, the coupling topology structure is learned under the time delay-transmission coupling fractional flexibility and optimal transmission constraint term, and the fractional flexibility feature sequence is generated; then, the geometric irregularity state vector is obtained by the fractional graph Laplacian curvature and the optimal transmission-based Ricci curvature combined with the time delay direction torsion quantity, and is forward propagated; then, the pre-processing gating factor is introduced, the link side and geometric side indicators are mapped to the anisotropic gating weight, and the gating state sequence of the state vector is obtained; under the factorization measure, potential function and transmission divergence weight constraint, the action density and local flexible irregularity index are calculated, and the time series anomaly index is obtained by windowed time aggregation of the entropy risk parameter; finally, the robust thresholding and time section aggregation of the time series anomaly index are performed, the multi-region signal anomaly set and its intensity score are obtained, and the segment-level communication quality estimation value is generated; the training target adopts a joint loss including reconstruction error, distribution robustness loss and topology persistence regularization, and can be supervised and constrained in combination with quality evaluation indicators and alarm records; through multi-task joint optimization and iterative training until convergence, the model realizes the cooperative adaptation of time delay-transmission coupling, non-local geometry and gating modulation mechanism, and completes the optimization processing of the signal for remote video communication.

[0019] 1、The application constructs a signal processing method for remote video communication, proposes a cooperative mechanism of time delay-transmission coupling fractional flexibility modeling and non-local fractional graph curvature irregularity detection, and solves the problem that picture quality cross-section propagation, arrival misplacement and structural mutation are difficult to accurately depict under heterogeneous networks and complex service loads. First, the coupling topology structure is constructed based on time delay-transmission coupling fractional flexibility and optimal transmission constraint term, and the fractional flexibility feature sequence is generated, then the geometric irregularity state vector is obtained by taking fractional graph Laplacian curvature and optimal transmission-based Ricci curvature as the core and combining time delay direction torsion quantity; this design can stably capture contraction / expansion and directional bias under packet loss, jitter and code-controlled switching scenes, and improve the sensitivity and time sequence resolution of cross-section association modeling and anomaly positioning.

[0020] 2、The application introduces a pre-processing gating factor, proposes an adaptive representation mechanism of factorized gating state sequence, and solves the problems of different synchronization of link indicators and signal abnormalities, short and sharp peak amplification, and unstable exponential scale. The candidate factor and geometric side indicators are constructed into a model factor vector and mapped into anisotropic gating weight, the component-level scaling of geometric irregularity state vector is implemented to obtain a gating state sequence; according to the gating state sequence, a factorized metric, a potential function and a transmission divergence weight are constructed, the action density and local flexible irregularity index are calculated, and the time series abnormality index is formed by an entropy risk aggregation parameter; the process significantly improves the signal-to-noise ratio and interpretability without changing the feature symbol and relative order, suppresses the false peak caused by instantaneous congestion and retransmission, and realizes risk adaptive aggregation for different communication conditions.

[0021] 3、The application proposes a multi-region decoding mechanism of robust thresholding and time period aggregation of time series abnormality index in the abnormal evidence utilization and result output link, solves the problems of fragile abnormal boundary, sensitive distribution drift and insufficient cross-channel consistency, performs distribution smoothing processing according to the time series abnormality index to obtain a robust potential intensity sequence for segmentation; the candidate multi-region segmentation result is obtained by robust thresholding and time period aggregation based on the time series abnormality index, and the multi-region signal abnormality set and its intensity score are formed by screening and merging with the topological persistence criterion, and finally the remote video communication signal processing result is output; cooperating with the signal processing model for remote video communication used for end-to-end iterative training, the spatiotemporal continuity and generalization robustness of the result can be maintained in multiple network and multiple service scenarios, and reliable and comparable quantitative basis is provided for online alarm, coding and decoding regulation and link strategy linkage. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 It is a flowchart of a signal processing method for remote video communication provided by the application.

[0023] Figure 2 It is a structure diagram for constructing a fractional flexibility feature sequence provided by the application.

[0024] Figure 3 It is a structure diagram for obtaining a geometric irregularity state vector provided by the application.

[0025] Figure 4 It is a structure diagram for generating a gating state sequence provided by the application.

[0026] Figure 5 It is a structure diagram for constructing a time series abnormality index provided by the application.

[0027] Figure 6 It is a diagram for outputting a remote video communication signal processing result provided by the application.

[0028] Figure 7 is a comparison chart of abnormal intensity, threshold decision and link factor provided by the application.

[0029] Figure 8 is a comparison chart of remote video communication signal processing before and after processing in time domain provided by the application. DETAILED DESCRIPTION

[0030] The application provides a signal processing method for remote video communication, aiming at picture quality mutation, unstable structural response and multi-region concurrent degradation identification difficulty under complex network and service fluctuation, and constructs a robust thresholding and time period aggregation mechanism based on time series anomaly index, which fuses time delay transmission coupling fractional flexibility modeling, non-local fractional graph curvature irregularity detection, flexible irregularity index and time series anomaly index: firstly, a coupling topological structure is constructed based on time delay transmission coupling fractional flexibility and optimal transmission constraint term, and a fractional flexibility feature sequence is generated; then, taking fractional graph Laplace curvature and optimal transmission Ricci curvature as cores, a geometric irregularity state vector is formed in combination with time delay direction torsion; a pretreatment gating factor is introduced to anisotropically gate and scale the vector to obtain a gated state sequence; a factorized measure, potential function and transmission divergence weight are constructed, an action and a local index are calculated, an entropy risk aggregation parameter is adaptively adjusted, and a time series anomaly index is obtained; finally, a robust thresholding and time period aggregation based on the time series anomaly index are adopted, a multi-region signal anomaly set and intensity are output, and a signal processing result of the remote video communication is obtained.

[0031] Please refer to Figure 1 , a signal processing method for remote video communication in the embodiment of the application, and the specific steps are as follows.

[0032] S1, acquiring signal, link and coding and decoding structure information of remote video communication and preprocessing, and constructing a remote video communication signal data set.

[0033] The construction process of the remote video communication signal dataset of the embodiment includes five stages: data acquisition, standardization alignment, denoising and compensation, normalization and segmentation, derived features and structured organization. In the data acquisition stage: under four working conditions of steady state, code control switching, bandwidth fluctuation and link disturbance, the data of the sending end and the receiving end is synchronously collected; the video side resolution is 1280x720, the frame rate is 30fps, the encoding standard is H.264 High Profile Level 4.0, the GOP is 60, the number of B frames is 2, the target code rate is 1.5 Mb / s, and the encoding preset is medium; the sending end reference YUV420 8-bit sequence, the receiving end decoded reconstructed frame and the complete Annex-B bitstream are saved to disk; on the link side, the packet loss rate, the bit error rate, the round-trip delay, the delay jitter, the available bandwidth, the congestion indication, the fragmentation sequence number, the retransmission and redundancy identification, the sending and arrival time stamps, the player buffer occupancy and the stall events are recorded with a sampling rate of 50Hz; the whole link is synchronized to the UTC time reference using NTP, and the frame-level and segment-level VMAF, PSNR and SSIM are calculated as quality labels and supplemented with artificial alarm labels; in the standardization alignment stage: the bitstream is converted to the Annex-B original code stream, and the decoder version is fixed to FFmpeg 6.0; the sending end PTS / DTS is used as the master clock to construct three time axes, the link sequence is interpolated to 10ms resolution and aligned according to the frame timestamp, the frame-link matching tolerance is 16.7ms, and the tolerance is marked as missing and the mask is recorded; in the denoising and compensation stage: the round-trip delay, the delay jitter and the available bandwidth use the Hamming filter window of 9 and the threshold of 3σ; the quantization parameter and the reconstructed residual energy use 3-frame median filtering and perform percentile clipping of 0.5% to 99.5%; missing values are filled forward within 500ms, and linear interpolation is performed to the upper limit of 2s when exceeding 500ms, and the missing mask channel is retained; in the normalization and segmentation stage: the link and code stream scalars within the session are standardized with zero mean and unit variance, and the energy class frame-level features are scaled to [0, 1]; the samples are organized in a sliding window of 2s segments with 1s overlap, each segment contains 60 frames, and the corresponding link subsequence and structure metadata are synchronously cropped; in the derived feature and structured organization stage: 12 dimensions of motion vector statistics are parsed from the bitstream, 8x8 DCT coefficients are extracted and aggregated by subband to obtain 6 dimensions of energy, and 4 dimensions of Y, U, V channel reconstructed residual energy and inter-frame prediction error are calculated; time-frequency analysis is performed on the reconstructed residual, the window size is 32 frames and the step size is 16 frames, and 4 dimensions of 3 main frequency band energy ratios and spectral centroid are output; thus, a 26-dimensional feature vector is formed for each frame, and is stacked into a tensor-shaped segment number x 60 x 26 at the segment level; sparse events and metadata are stored in Parquet, and dense tensors are stored in HDF5; through the above process, a remote video communication signal dataset with strict alignment of signal, link, structure and label and reproducible parameters is obtained as the unified input for subsequent model training.

[0034] S2, according to the data set, set the local window length, generate the overlapping window set, calculate the time lag correlation coefficient of adjacent time indexes, and obtain the window pair transmission divergence under the entropy regular optimal transmission constraint, construct the coupling topological structure according to the correlation coefficient and the transmission divergence, and execute the fractional order flexibility mapping to obtain the fractional flexibility feature sequence.

[0035] Further, in the S2 step, the fractional flexibility feature sequence is constructed, and the flowchart is as shown in the figure. Figure 2 The specific steps of constructing the fractional flexibility feature sequence are as follows.

[0036] Set the local window length, sliding step and maximum time lag threshold, divide the remote video communication signal in time index, and establish the corresponding local window sample set for each time index; In the embodiment, the remote video communication signal is divided in the segment frame level feature sequence, and the frame level feature matrix is obtained. The mathematical model is: ; Wherein, is the feature vector of the t frame, D is the frame level feature dimension, which is 26 in the embodiment, and T is the total number of frames in the segment, which is 60 in the embodiment; set the local window radius as h, the sliding step as s, and the maximum time lag threshold as tau; For each on the time index set, construct the local window centered at t, and use symmetric expansion padding for the out-of-bound index to ensure that each window contains 2h+1 frame vectors; The mathematical model of the time index set is: ; The mathematical model of the local window is: ; Stack the frame vectors in the window in column order to form a long vector, and obtain the local window sample vector The mathematical model of the local window sample vector is: ; The mathematical model of the local window sample vector is: ; Collect all to obtain the local window sample set The mathematical model is: ; Where M is the local window sample vector The dimension; In this embodiment, the local window radius h is set to 4, so that each window covers 2h+1=9 frames; the sliding step size s is set to 1, so that the temporal resolution is not lost under frame-level advancement; the maximum time delay threshold τ is set to 3, which corresponds to an arrival misalignment search range of about 100 ms under 30fps conditions, which can cover the timing deviation caused by common buffer jitter and slight retransmission and avoid false associations caused by excessive time delay; the out-of-bounds index adopts symmetrical expansion to ensure that the length and statistical characteristics of the boundary window and the middle window are consistent.

[0037] Based on the local window sample set, normalized correlation calculation is performed on any two first-order adjacent time indices within a preset time delay range. The maximum correlation value under each time delay is taken as the time delay correlation coefficient between the two, thus obtaining a set of time delay correlation coefficients. In this embodiment, based on the local window sample set For time index sets Constructing a time-delay search set using any pair of first-order nearest neighbors (t, t+s) is mathematically modeled as follows: ; Element-wise standardization of the long vector of the window eliminates amplitude and scale differences, resulting in a standardized vector. The mathematical model is: ; in, For local window sample vectors The mean, For local window sample vectors standard deviation As a numerically stable term, it is taken in this embodiment. ; In this embodiment, the local window sample vector mean The mathematical model is as follows: ; In this embodiment, the local window sample vector Standard deviation The mathematical model is as follows: ; Based on the standardized vector In each time delay Next, calculate the long vector of the window. Vector corresponding to time delay Zero-mean unit variance normalized correlation The mathematical model is: ; According to the normalized correlation , the maximum value is taken as the time lag correlation coefficient of the pair of adjacent indexes and the time lag amount of the maximum correlation, and the mathematical model is: ; ; wherein, is the time lag correlation coefficient, indicating the strongest similarity within the allowed time lag; is the time lag amount to achieve the maximum correlation; All (t, t+s) are traversed to obtain a time lag correlation coefficient set and a time lag amount set of the maximum correlation ; ; .

[0038] According to the sample distance between any two local windows, an optimal transport constraint term with entropy regularization is constructed, and the local transport divergence of window pairing is obtained by optimal transport solving to obtain a transport constraint amount set; In this embodiment, based on the local window sample set and the corresponding optimal time lag amount set , for each pair of first-order adjacent indexes (t, t+s), the following is used as the window pairing, the window length vector is converted into a discrete probability distribution, and for any , a non-negative weight is defined and normalized to obtain a probability vector , and the mathematical model is: ; wherein, is the non-negative weight of the mth component of the local window t, ; δ is a small constant to avoid zero quality, and in this embodiment δ = 0.0001 ; Further, the pairing is defined as a position-content joint cost, and a joint cost matrix is constructed, and the mathematical model is: ; ; wherein, α ∈ [0, 1] is a position and content weighting coefficient, and in this embodiment α = 0.3, is the pooled variance of the paired window, , are the standard deviations of , respectively, and ε is a numerical stability term, and in this embodiment, ; The joint cost matrix with probability vector Entropy regularization is solved on feasible set for input; The mathematical model of the feasible set is: ; The mathematical model of entropy regularization is: ; Wherein, is the entropy regularization strength, which is set to ; Define the kernel matrix , and solve iteratively using entropy regularization optimal transport: ; Until the marginal residual , is the convergence threshold, in this embodiment, ; Use the Frobenius inner product of the optimal coupling and the cost matrix as the local transport divergence of the pair of adjacent indexes : ; Traverse to obtain the set of transport constraint quantities, and the mathematical model is: .

[0039] Based on the time delay correlation coefficient and the set of optimal transport constraint quantities, the coupling weight between the pair of time indexes is determined, the threshold is processed for sparsification, and the coupling topological structure is constructed; Map the time delay correlation coefficient and the local transport divergence to the similarity interval [0, 1] uniformly to obtain the relevant similarity and the transport similarity , and the mathematical model is: , ; Wherein, is the mapping result of mapping [−1, 1] to [0, 1], is the truncated relevant similarity, which is clamped to 0 when <−1, indicating that strong negative correlation is not evidence for positive coupling; ; Wherein, is the transport similarity, which increases with monotonically decreasing, range (0, 1]; β>0 is a scale parameter, in this embodiment, β=1 , ε is a numerical stability term, in this embodiment, ε=1e-6 ; Based on the correlation similarity and the transmission similarity , determine the joint weight, the mathematical model is: ; wherein, is a geometric weighting exponent, in this embodiment, α=0.5 , set to =0.5 Edge construction by optimal adjacent index pairing and symmetrization, construct the weight matrix, the mathematical model is: ; wherein, is a weight matrix, is the number of nodes; Threshold sparsification is done on the weight matrix , and the weighted adjacency matrix is obtained, the mathematical model is: ; wherein, is an edge threshold, in this embodiment, ε=0.3 =0.3, is a threshold operator, used to maintain the weight and sparsification; diagonal zero is used to eliminate the self-loop; symmetrization ensures the undirected graph; Through threshold sparsification and graph Laplacian processing, the coupled topological structure is obtained, the mathematical model is: ; wherein, is a node set, is an edge set, is a weighted adjacency matrix, and L is a symmetric normalized graph Laplacian; In this embodiment, the mathematical model of the symmetric normalized graph Laplacian L is: ; wherein, I is an identity matrix, and D is a degree matrix; The mathematical model of the degree matrix D is: ; wherein, is a weighted adjacency matrix multiplied by a full 1 column vector, specifically .

[0040] A regular operator with time smoothness and shape constraint is established on the coupling topology, fractional order flexibility mapping and feature extraction are performed, and a fractional flexibility feature sequence of the actual state is obtained; In the embodiment, the mathematical model of the fractional flexibility feature sequence is as follows: , ; wherein Y is the feature after fractional graph smoothing; is the feature after fractional graph smoothing of the t-th row, and alpha is an exponential control of the flexibility order, alpha=0.3 in the embodiment; >0 is a regularization scale; norm(·) is an ℓ2 normalization; the t-th row fractional flexibility feature, is the fractional flexibility feature sequence; In the embodiment, the t-th row fractional flexibility feature The mathematical model of the ℓ2 normalization is as follows: ; wherein, is the two-norm of the feature vector of the t-th time point, , is a numerical stability term, in the embodiment, .

[0041] S3, according to the fractional flexibility feature sequence and the coupling topology, a fractional graph Laplacian curvature is calculated, an optimal transport aggregated Ricci curvature is based, and a time delay direction torsion quantity is obtained through forward and backward non-local gradients, the above results are combined, and a geometric irregularity state vector is obtained.

[0042] Further, in the S3 step, the geometric irregularity state vector is obtained, the flow is as shown in Figure 3 , and the specific steps of obtaining the geometric irregularity state vector are as follows.

[0043] Based on the coupling topology, a graph Laplacian operator is constructed, the fractional graph Laplacian operation is performed on the fractional flexibility feature sequence according to a preset fractional order, and a fractional graph Laplacian curvature response is obtained; In the embodiment, the mathematical model of the fractional graph Laplacian curvature response is as follows: ; wherein, is the fractional power of L, is 0.7, in the embodiment, ∈(0, 1], is set to 0.5, is the response vector of the time point t, The score graph Laplacian curvature size of the t-th behavior node t.

[0044] According to the coupling topology, a probability distribution is constructed in a local neighborhood, an optimal transport cost is used to calculate the distance contraction degree of any adjacent time index pair, an optimal transport Ricci curvature is obtained, and node-level Ricci curvature is aggregated; In this embodiment, the optimal transport Ricci curvature The mathematical model is: , ; Wherein, is the optimal transport cost with entropy regularization, , is the neighborhood probability distribution constructed on the graph with as the center; The mathematical model of the neighborhood probability distribution , is: , ; Wherein, is the edge weight between node i and node m in the weighted adjacency matrix, is the edge weight between node j and node n in the weighted adjacency matrix; The mathematical model of aggregating the node-level Ricci curvature is: .

[0045] Under the constraint of the coupling topology, the non-local gradient is calculated respectively in the time forward and backward directions, and the time delay direction twist is obtained according to the difference between the forward and backward directions; In this embodiment, the mathematical model of the time delay direction twist is: ; Wherein, is the forward time, and the most similar forward neighbor under the weighted adjacency matrix A; is the backward time, and the most similar backward neighbor under the weighted adjacency matrix A; is the fractional flexibility feature vector at row index , is the fractional flexibility feature vector at row index .

[0046] The fractional graph Laplacian curvature, the optimal transport-based Ricci curvature, and the time delay direction twist are robustly scaled and scale normalized; Robustly scaled fractional plot Laplace curvature Ricci curvature after aggregation and the amount of torsion in the direction of time delay The mathematical model is as follows: ; in, The median, This represents the absolute deviation of the median. As a numerically stable term, in this embodiment, it is set as follows: ; median The mathematical model is as follows: ; Median absolute skew The mathematical model is: .

[0047] The geometric irregularity state vector is obtained by vectorizing and combining the robustly scaled fractional graph Laplace curvature, the aggregated Ricci curvature, and the time-delay directional torsion. In this embodiment, the mathematical model for the geometrically irregular state vector is: ; in, .

[0048] S4. Based on the state vector and link information, a preprocessing gating factor is introduced, candidate factors are extracted, and anisotropic gating weights are obtained by mapping. The state vector is then scaled by component level according to the gating weights to obtain the gating state sequence.

[0049] Furthermore, in step S4, a gated state sequence is generated, the process of which is as follows: Figure 4 As shown, the specific steps for generating the gated state sequence are as follows.

[0050] Based on the geometric irregularity state vector and link information, a preprocessing gating factor is introduced to extract candidate factors and geometric side indicators. In this embodiment, the preprocessing gate factor The mathematical model is as follows: ; in, This represents the k-th link feature after robust standardization. This is the sensitivity coefficient. ∈[2,8], in this embodiment, =4; The threshold value is set to 2 in this embodiment. For activation functions; robustly normalized kth link feature The mathematical model of the activation function is: ; wherein, is the original multi-dimensional sequence on the link side, and ε is a numerical stability term, in the embodiment, ; The mathematical model of the activation function is: ; The link factor and the geometric side indicator are unified to the same dimension and combined into a candidate factor for modulation, and the mathematical model is: , , ; wherein, is the link candidate factor, is a component-wise multiplication, indicating that the link dimension is first soft-filtered by using a pre-processing gating factor, is the geometric side indicator, is the candidate factor for modulation extracted by combining the link candidate factor and the geometric side indicator.

[0051] The candidate factors are time-aligned and interval-normalized to construct a modulation factor vector; In the embodiment, the optimal minimum delay of the maximum cross-correlation of each candidate factor for modulation is calculated based on the geometric side reference sequence , and an aligned candidate factor is obtained, and the mathematical model is: , ; wherein, is a scalar sequence of the kth candidate factor at time t, is a geometric side reference sequence, taking the mean value of the geometric irregularity state vector , d is a time delay bias, is an upper limit of search, in the embodiment, =3 ; is the aligned candidate factor, which is used to time-shift the original sequence by the optimal displacement δk; The mathematical model of the geometric side reference sequence is: ; wherein, is the transpose of the all-1 vector; The candidate factor is normalized to obtain a modulation factor vector, and the mathematical model is: ; wherein, is a numerical stability term, in the present embodiment, .

[0052] The input factor vector is mapped to anisotropic gating weights, and a time-continuous gating weight sequence is obtained by sliding window smoothing; In the present embodiment, the input factor candidate vector is mapped to a three-dimensional pre-activation quantity through a linear layer to obtain anisotropic gating weights , and the mathematical model is: ; wherein, is a pre-activation quantity, is an activation function; The mathematical model of the pre-activation quantity is: ; wherein, is a gating mapping weight, obtained by a back propagation algorithm in training, is a bias of the gating mapping which can be learned, and is updated adaptively through back propagation along with the model in training, in the present embodiment, the initial value is set to 0.405; A time-continuous gating weight sequence is obtained by sliding window H smoothing, and the mathematical model is: , ; wherein, is a time-smoothed gating weight sequence, is a smoothing coefficient; The mathematical model of the smoothing coefficient is: ; wherein, H is an equivalent window length, in the present embodiment, H=5.

[0053] The geometric irregularity state vector is scaled at a component level by the gating weights to obtain a gating state sequence; In the present embodiment, the mathematical model of the gating state sequence is: .

[0054] S5, according to the gating state sequence and the pre-processed gating factor, factorized metrics, potential functions and transport divergence weights are constructed, action density is calculated and a local flexible irregularity index is generated, an entropy risk aggregation parameter is adaptively set according to the working condition, the index is windowed and time-aggregated to obtain a time series anomaly index.

[0055] Further, in step S5, a time series anomaly index is constructed, and a flowchart of constructing the time series anomaly index is as shown in FIG. 6. Figure 5 Specific steps of constructing the time series anomaly index are as follows.

[0056] According to the gating state sequence and the preprocessed gating factor, a factorized metric, a potential function and a transport divergence weight are constructed. In the embodiment, a mathematical model of the factorized metric is as follows. ; wherein, is a factorized metric tensor, which is modulated by the gating weight and the preprocessed gating factor over time, is a factorized metric weight vector at time t, and diag(·) is diagonalization. A mathematical model of the factorized metric weight vector is as follows. ; wherein, , , is a sensitivity coefficient matrix, in the embodiment, , , ; In the embodiment, a mathematical model of the potential function is as follows. ; wherein, is a transport divergence weight, in the embodiment, .

[0057] Under the constraint of the factorized metric and the potential function, an action density is calculated for the gating state sequence, and a locally flexible irregular index is generated in combination with a directionality vector. In the embodiment, a mathematical model of the action density is as follows. ; wherein, is a first-order time difference operator, is a transport divergence weight, is a reinforced sparse transition. A mathematical model of the locally flexible irregular index is as follows. ; A mathematical model of the transport divergence weight is as follows. ; wherein, , transmit the divergence weight coefficient, in this embodiment, , ; the mathematical model of the reinforced sparse transition is: ; wherein, is a smoothing constant, in this embodiment, ; In this embodiment, the mathematical model of the local flexibility irregular index is: ; wherein, is a coefficient obtained by amplifying the directional mutation in the time delay direction twisting amount, in this embodiment, ;

[0058] According to the entropy risk aggregation parameter adaptively set according to the factor and the working condition, the local index is windowed and time-aggregated to obtain a time series anomaly index; In this embodiment, the mathematical model of the time series anomaly index is: ; wherein, is an entropy risk aggregation parameter; The mathematical model of the entropy risk aggregation parameter is: ; wherein, is an entropy risk aggregation parameter, in this embodiment, , is the value after robust standardization of ; In this embodiment, the mathematical model of is: .

[0059] S6, the time series anomaly index is subjected to exponential smoothing, robust thresholding and hysteresis gate decision, time period aggregation is performed to obtain a multi-interval anomaly set strength, and through cross-channel fusion and mapping modulation, a remote video communication signal processing result is output.

[0060] Further, in the S6 step, the remote video communication signal processing result is output, and the process is as shown in Figure 6 The specific steps of outputting the remote video communication signal processing result are as follows.

[0061] According to the time series anomaly index, the signal is exponentially smoothed to obtain a robust potential intensity sequence for interval detection; In this embodiment, the mathematical model of the robust potential intensity sequence is: ; Wherein, is the time series anomaly index, initialized as ; is the exponential smoothing coefficient, In this embodiment, =0.2; Based on robust thresholding and hysteresis gate decision, the robust potential intensity sequence is aggregated and cleaned to obtain a multi-region signal anomaly set intensity; In this embodiment, the entering and exiting thresholds are adaptively determined based on the median-median absolute deviation, and the hysteresis decision is made to obtain a binary indication sequence , the mathematical model is: ; Wherein, is the entering threshold, is the exiting threshold; In this embodiment, the entering threshold is a mathematical model: ; Wherein, is the entering coefficient, used to adjust the amplification multiple of the relative fluctuation of the entering threshold, in this embodiment =3; In this embodiment, the mathematical model of the exiting threshold is: ; Wherein, is the exiting coefficient, used to adjust the amplification multiple of the relative fluctuation of the exiting threshold, in this embodiment =2; In this embodiment, consecutive =1 segments are aggregated into an anomaly interval set, and the mathematical model is: ; Wherein, is the starting time index of the jth anomaly interval, j is the index value of the anomaly interval, , J is the number of anomaly intervals; In this embodiment, is determined by the left boundary of consecutive =1 segments in the binary sequence In this embodiment, the binary sequence The middle part is scanned from front to back, and when a =1, the time t is recorded as the starting index of the jth continuous "1" segment ; In this embodiment, the intensity score of each interval is calculated , and is robustly normalized to obtain the multi-interval signal anomaly set intensity , and the mathematical model is: ; ; The multi-region anomaly set is fused and mapped to output the remote video communication signal processing result; In this embodiment, the mathematical model of the output remote video communication signal processing result is: ; wherein, is the anomaly set intensity after mapping and modulation, is a gain coefficient, in this embodiment, =1, is a threshold shift term, in this embodiment, =0; The mathematical model of the output remote video communication signal processing result ; wherein, is the number of channels, which is adaptively given by the training process according to the generated score flexibility, link factor and gate-derived quantity; is the signal anomaly set intensity of the jth interval at time t, is a uniform weight, in this embodiment, .

[0062] S7, construct a signal processing model for remote video communication, input a remote video communication signal data set, sequentially pass through the above steps, and through iterative training until convergence, complete the optimization processing of the signal for remote video communication.

[0063] Aiming at the multi-source coupling degradation and cross-segment diffusion characteristics of remote video communication under packet loss, jitter and code control switching, an end-to-end signal processing model for remote video communication is constructed, and the robust thresholding and time segment aggregation stages of coupling topology coding, curvature irregularity forward, anisotropic gating state sequence, transmission action quantity index, and time series anomaly index are executed in turn: first, the remote video communication signal dataset is taken as input, the coupling topology structure is learned under the time delay transmission coupling fractional flexibility and optimal transmission constraint term, and the fractional flexibility feature sequence is generated; then, the geometric irregularity state vector is obtained by combining the time delay direction torsion quantity based on the fractional graph Laplacian curvature and the optimal transmission-based Ricci curvature, and is forward propagated; then, the pre-processing gating factor is introduced, the link side and geometric side indicators are mapped to the anisotropic gating weight, and the gating state sequence of the state vector is obtained; under the factorization measure, potential function and transmission divergence weight constraint, the action density and local flexible irregularity index are calculated, and the time series anomaly index is obtained by windowed time aggregation of the entropy risk parameter; finally, the robust thresholding and time segment aggregation of the time series anomaly index are performed, the multi-region signal anomaly set and its intensity score are obtained, and the segment-level communication quality estimation value is generated; the training target adopts a joint loss, including reconstruction error, distribution robustness loss and topology persistence regularization, and can be supervised and constrained in combination with quality evaluation indicators and alarm records; through multi-task joint optimization and iterative training until convergence, the model realizes the cooperative adaptation of time delay transmission coupling, non-local geometry and gating modulation mechanism, and completes the optimization processing of the signal for remote video communication.

[0064] Further, in the S7 step, the remote video communication signal processing model proposed by the application is implemented by using the Python programming language, and is trained and inferred on a single card NVIDIA RTX 3090 (24 GB) based on the PyTorch 2.0 framework; the model input is a segment-level tensor organized by a 2s sliding window, with a shape of BxCx60x26, and zero mean unit variance standardization and window alignment are uniformly used in the training stage; the optimizer selects Adam, the basic learning rate is 1x10⁻³, and is dynamically reduced in combination with the cosine annealing strategy, the batch size is 32, the training number is 150, and the early stopping is combined with the validation set; the loss function adopts a joint target: reconstruction error, distribution robustness regularization, topology persistence constraint, and a light smoothing term is introduced for time splicing continuity; in the inference stage, the segment-level estimation is reconstructed to output the whole segment result by sliding window splicing.

[0065] Further, in step S7, the time series anomaly index, the gated threshold decision and the candidate link factor obtained in the remote video communication signal processing model are input into the interval decoding and fusion module for experimental verification. The verification indexes are the packet loss rate (PLR), the jitter (RTT) and the video subjective quality prediction score (1-VMAF). The packet loss rate (PLR) is the proportion of the lost data packets in the total number of sent packets in a unit of time, which is expressed by %, and the greater the value, the more unstable the link. The jitter (RTT) is the time from the sending end to the receiving end and back. In practice, the short-term fluctuation amplitude of RTT is often observed. Large jitter means unstable delay, which is easy to cause buffer jitter and playback lag. The video subjective quality prediction score (1-VMAF) indicates that the higher the abnormal intensity, the higher the curve. The results are shown in FIGS. 1-VMAF, PLR and RTT. Figure 7 and Figure 8 . Figure 7 is an abnormal intensity, threshold decision, and link factor comparison chart. The horizontal coordinate is time t (s), the left vertical coordinate is the abnormal intensity, and the right vertical coordinate is the normalized candidate factor. The light gray shadow is the abnormal segment obtained by interval decoding, and the text in the shadow is the intensity score (0-1) of the segment. It can be seen from Figure 7 that in the two obvious burst intervals, the abnormal intensity exceeds the entering threshold τon and remains below the exiting threshold τoff to end. The corresponding PLR and RTT curves are simultaneously lifted, and the 1-VMAF curve is increased, indicating that the link packet loss and subjective picture quality degradation are consistent with the abnormal index. In the non-abnormal period, the abnormal intensity remains below the threshold, and the three candidate factor fluctuations are small. Figure 7 It is shown that the robust thresholding and hysteresis gate decision can accurately capture the continuous interval of communication anomalies, and the interval intensity score and the link quality degradation degree have good correlation, providing a reliable basis for subsequent segment-level quality estimation. Figure 8 is a time domain comparison chart before and after processing. The left chart is the actual communication side time domain signal before processing, which contains obvious pulse peaks and high frequency disturbances. The right chart is the signal after distribution robust de-abnormal, median smoothing and mild low pass filtering. The instantaneous peak is effectively suppressed, the overall envelope is smoother, the baseline drift is reduced, and the slow trend and useful information are still retained. As can be seen from the comparison, the proposed robust preprocessing can suppress burst anomalies and noise, and avoid over-smoothing leading to over-smoothing, which can provide stable input for interval detection and quality evaluation.

[0066] The above is only a preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the inventive concept, several modifications and improvements can be made, which are within the scope of the present application.

Claims

1. A signal processing method for remote video communication, characterized by, Signal, link and coding-decoding structure information of remote video communication are acquired and preprocessed to construct a remote video communication signal dataset; According to the dataset, a local window length is set, an overlapping window set is generated, a time lag correlation coefficient of adjacent time indexes is calculated, and a window pair transmission divergence is obtained under the constraint of entropy regular optimal transport, a coupling topological structure is constructed according to the correlation coefficient and the transmission divergence, and a fractional order flexibility mapping is performed to obtain a fractional flexibility feature sequence; According to the fractional flexibility feature sequence and the coupling topological structure, a fractional graph Laplacian curvature is calculated, an optimal transport aggregated Ricci curvature is based, and a time lag direction torsion is obtained through forward and backward non-local gradients, and the above results are combined to obtain a geometric irregularity state vector; According to the state vector and the link information, a preprocessing gating factor is introduced, a candidate factor is extracted, and an anisotropic gating weight is mapped to obtain a gating state sequence by scaling the state vector at the component level according to the gating weight; According to the gating state sequence and the preprocessing gating factor, a factorized measure, a potential function and a transmission divergence weight are constructed, an action density is calculated and a local flexible irregularity index is generated, an entropy risk aggregation parameter is adaptively set according to the working condition, and the index is windowed and time-aggregated to obtain a time series anomaly index; The time series anomaly index is subjected to exponential smoothing, robust thresholding and hysteresis gate decision, time period aggregation is performed to obtain a multi-interval anomaly set strength, and a remote video communication signal processing result is output through cross-channel fusion and mapping modulation; A signal processing model for remote video communication is constructed, a remote video communication signal dataset is input, and the above steps are sequentially performed, and iterative training is performed until convergence to complete the optimization processing of the signal for remote video communication.

2. The signal processing method for remote video communication according to claim 1, wherein, Raw communication signal data of remote video under different conditions of steady state, code control switching, bandwidth fluctuation and link disturbance is acquired, and the signal data includes frame-level image sequences of the sending end and the receiving end, encoded bit streams, reconstructed frames and residual signals thereof, transform coefficients, motion vectors and statistics thereof, inter-frame prediction error signals and time-frequency envelopes; Link side operation information is acquired, including packet loss rate, bit error rate, round trip delay, delay jitter, available bandwidth estimation, congestion indication, fragmentation and sequence number, retransmission and redundancy identification, sending and arrival time stamps, and play buffering and stall events; Coding and decoding structure information is acquired, including frame type identification, quantization parameter, code rate and resolution, frame rate, image group structure, spatial blocking and slicing, region division and reference frame index; Labels corresponding to time indexes, channel and subchannel identifications, spatial position identifications and quality annotation information are acquired, and the quality annotation includes quality evaluation indicators and alarm records; The raw communication signal data, the link side operation information, the coding and decoding structure information are preprocessed to construct a remote video communication signal dataset.

3. The signal processing method for remote video communication according to claim 2, wherein, A local window length, a sliding step and a maximum time lag threshold are set, the remote video communication signal is divided in overlapping according to time indexes, and a corresponding local window sample set is established for each time index; According to the local window sample set, a normalized correlation calculation is performed on any two first-order adjacent time indexes within a preset time lag range, a maximum value of the correlation under each time lag is taken as a time lag correlation coefficient of the two, and a time lag correlation coefficient set is obtained; According to the sample distance between any two local windows, an optimal transport constraint term with entropy regularization is constructed, and a local transport divergence of window pairing is obtained by optimal transport solving, to obtain a transport constraint quantity set.

4. The signal processing method for remote video communication according to claim 3, wherein, Based on the time lag correlation coefficient and the optimal transport constraint quantity set, the coupling weight between the time index pairs is jointly determined, a threshold is used for sparsification processing, and a coupling topological structure is constructed; A regular operator with time smoothing and shape constraint is established on the coupling topological structure, fractional order flexibility mapping and feature extraction are performed, and a fractional flexibility feature sequence of the actual state is obtained.

5. The signal processing method for remote video communication according to claim 4, wherein, Based on the coupling topological structure, a fractional graph Laplacian operator is constructed, and a fractional graph Laplacian operation is performed on the fractional flexibility feature sequence according to a preset fractional order, to obtain a fractional graph Laplacian curvature response; According to the coupling topological structure, a probability distribution is constructed in a local neighborhood, and the distance contraction degree of any adjacent time index pair is calculated using an optimal transport cost, to obtain a Ricci curvature of optimal transport, and the node-level Ricci curvature is aggregated; Under the constraint of the coupling topological structure, non-local gradients are calculated in the time forward and backward directions respectively, and a time lag direction torsion is obtained according to the difference between the forward and backward directions.

6. The signal processing method for remote video communication according to claim 5, wherein, The fractional graph Laplacian curvature, the Ricci curvature based on optimal transport, and the time lag direction torsion are subjected to robust scaling and scale normalization processing; The fractional graph Laplacian curvature subjected to robust scaling, the aggregated Ricci curvature, and the time lag direction torsion are vectorized and combined to obtain a geometric irregularity state vector.

7. A signal processing method for remote video communication according to claim 6, characterized in that, According to the geometric irregularity state vector and link information, a preprocessing gating factor is introduced, a candidate factor and a geometric side index are extracted; The candidate factor is subjected to time sequence alignment and interval normalization, and an embedding factor vector is constructed; The embedding factor vector is mapped to an anisotropic gating weight, and a time-continuous gating weight sequence is obtained through sliding window smoothing; The geometric irregularity state vector is scaled at the component level by the gating weight to obtain a gated state sequence.

8. The signal processing method for remote video communication according to claim 7, wherein, According to the gated state sequence and the preprocessing gating factor, a factorized measure, a potential function, and a transport divergence weight are constructed; Under the constraint of the factorized measure and the potential function, the action density of the gated state sequence is calculated, and a local flexible irregularity index is generated in combination with a directional vector; According to the factor and the working condition, an entropy risk aggregation parameter is adaptively set, the local index is windowed and time-aggregated to obtain a time series anomaly index.

9. The signal processing method for remote video communication according to claim 8, wherein, According to the time series anomaly index, the signal is subjected to exponential smoothing to obtain a robust potential intensity sequence for interval detection; Based on robust thresholding and delay gate decision, the robust potential intensity sequence is subjected to interval aggregation and cleaning to obtain a multi-region signal anomaly set strength; The multi-region anomaly set is subjected to cross-channel fusion and mapping modulation, and a remote video communication signal processing result is output.

Images