A video public call intelligent communication system supporting multi-terminal interconnection

Abstract

The application discloses a kind of video public call intelligent communication systems supporting multi-terminal interconnection, comprising: time slice hierarchical module, for cutting video public call data stream, form hierarchical data block set;Expansion window construction module, for combining hierarchical data block into initial expansion window, generate hierarchical expansion window;Error correction strategy arrangement module, for assembling three types of strategy of improved RS inequality error correction;Terminal laminated construction module, for performing constraint type fuzzy hierarchical clustering, generate terminal laminated expansion window;Key area identification module, for executing key point identification, form key area expansion window;Multi-phase time sequence coding module, for pairing key area expansion window with terminal laminated expansion window, generate multi-phase expansion window sequence;Joint redundancy and recovery module, for allocating redundancy budget and generating joint redundancy symbol, complete layered data recovery.The application realizes video public call intelligent communication.

Description

Technical Field

[0001] This invention relates to the field of intelligent communication technology, and in particular to a video public phone intelligent communication system that supports multi-terminal interconnection. Background Technology

[0002] Most existing video public phone systems employ a combination of terminal-side adaptive bitrate and traditional forward error correction to ensure real-time performance and basic availability. A common technical approach involves generating base and enhancement layers on a time-slice basis at the encoding end, and outputting redundant symbols using Reed-Solomon erasure codes with equal protection or simple unequal protection. On the network side, congestion control, bandwidth estimation, and jitter buffering mechanisms are relied upon, with the terminal dynamically adjusting encoding parameters or triggering limited retransmission strategies based on packet loss, latency, and bitrate feedback. In scenarios with stable networks and single-terminal point-to-point communication, this approach can meet typical video call needs. However, in environments such as public phone halls, home-to-campus cross-domain communication, multi-terminal concurrent access, mixed weak network access, and environments with significant differences in link conditions, the above approach exposes a series of technical bottlenecks: symbols within a time slice are often concentratedly damaged, the utilization rate of redundant symbols is low, and the recovery threshold is difficult to meet, leading to frequent issues such as video stuttering and voice interruption.

[0003] On the other hand, existing content-aware technologies are mostly used for offline compression, object detection, or subjective quality assessment, and rarely for error correction in real-time public voice communication. For key semantic regions in video public voice communication, such as faces, mouths, and eyes, existing systems typically only adjust quantization parameters at the encoding end based on complexity, without performing differentiated symbol protection based on semantic importance during the forward error correction stage. This makes it difficult to reliably guarantee language coherence, lip-sync, and emotion recognition under weak network conditions. Furthermore, traditional multi-terminal management strategies often adopt a terminal-by-terminal adaptation approach, lacking a mechanism for clustering and hierarchical classification based on terminal business characteristic parameters. This makes it impossible to build a layered window covering high-priority terminals to low-priority terminals, and also fails to achieve differentiated and measurable protection strength under a unified redundancy budget. Consequently, psychological counseling terminals, emergency management terminals, or home remote terminals struggle to obtain stronger protection in concurrent scenarios.

[0004] Therefore, how to provide a video public phone intelligent communication system that supports multi-terminal interconnection is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a video public phone intelligent communication system that supports multi-terminal interconnection. This invention improves the recovery probability of basic voice and facial / lip-sync under weak network conditions through terminal stacking, multi-phase interleaving, and priority protection of key areas; joint redundancy is available across layers, reducing the threshold and lag rate; under the same redundancy budget, it achieves differentiated protection for multiple terminals, is compatible with existing SR decoding, and has low deployment costs.

[0006] According to an embodiment of the present invention, a video public phone intelligent communication system supporting multi-terminal interconnection includes the following steps: The time-slice hierarchical module is used to divide the video public call data stream according to preset time slices, forming a hierarchical data block set; The extended window construction module is used to combine hierarchical data blocks into initial extended windows of the base layer and enhancement layer covering different consecutive time slices based on the window length parameter, and generate hierarchical extended windows. The error correction strategy orchestration module is used to assemble three types of strategies: multi-terminal stacking of improved RS uneven error correction, priority protection of key areas, and window timing multiphase coding. The terminal overlay construction module is used to perform constrained fuzzy hierarchical clustering based on terminal-side service feature parameters and generate terminal overlay extended windows. The key region identification module is used to perform temporal saliency pre-screening, face detection and key point recognition on the video frames corresponding to the window to form an expanded window of the key region. The multiphase timing coding module is used to pair the key region expansion window with the terminal stacked expansion window to generate a multiphase expansion window sequence; The joint redundancy and recovery module is used to allocate redundancy budgets based on the multiphase extended window sequence and perform perturbation-type RS coding to generate joint redundancy symbols, thereby completing hierarchical data recovery.

[0007] Optionally, modules can be integrated using the following methods: The input data stream of the video public call service is obtained and divided into time slices to obtain hierarchical data blocks; An initial extended window covering different consecutive time slices is constructed based on hierarchical data blocks, and a hierarchical extended window is generated. The process of receiving a hierarchical expansion window and completing the preparation process for improved RS inequality error correction includes a multi-terminal stacked expansion window strategy, a content-aware key area priority protection strategy, and a window timing multiphase coding strategy. A multi-terminal overlay expansion window strategy is adopted for the hierarchical expansion window. Based on the terminal-side service characteristic parameters of the access terminals, the terminal dimensions are divided and overlaid to obtain a terminal overlay expansion window with differentiated coverage. Based on the key area priority protection strategy, content-aware analysis is performed on video frames to identify key feature areas including face areas and mouth areas, and data symbols are added to the key area sub-window of the hierarchical expansion window to form a content-aware key area expansion window. By configuring different window start times and timing phase offsets for the key region extended window and the terminal stacked extended window input window timing multiphase coding strategy, a multiphase extended window sequence with phase interleaving on the time axis is generated. Based on the multiphase extended window sequence, and using the improved RS unequal error correction, erasure coding is performed to generate joint redundant symbols. The joint redundancy symbols are encapsulated with hierarchical data blocks, and data recovery is performed at the receiving end. When there are insufficient symbols, the joint redundancy symbols are used for compensation, and the recovered video public voice data is output.

[0008] Optionally, obtaining the hierarchical data blocks specifically includes: The input data stream of the video public call service is obtained, the continuously arriving bit sequence in the input data stream is cached sequentially, and the data is divided in order according to the preset time slice length. During the division process, when the time length corresponding to the cached data reaches the preset time slice length parameter, the currently cached data is divided into a time slice, and a unique time slice number is assigned to each time slice. For each time slice, the cached data corresponding to the current time slice is classified and processed according to the preset hierarchical encoding rules. The part carrying voice information, character outline information and key control information is integrated into the basic layer data block, and the part carrying image detail texture information, resolution enhancement information and visual effect enhancement information is integrated into the enhancement layer data block. The base layer data blocks and enhancement layer data blocks corresponding to each time slice are organized in the order of time slice number to form hierarchical data blocks.

[0009] Optionally, the generation process of the hierarchical expansion window specifically includes: Read the hierarchical data blocks, organize the time slices in numerical order, and maintain the one-to-one correspondence between the base layer data blocks and the enhancement layer data blocks in the same time slice; Set the extended window length parameter and determine the number of consecutive time slices covered by each extended window. Select a time slice number from the hierarchical data block set as the starting number of the extended window. Starting from the starting number, sequentially select adjacent time slices to form the coverage range of the extended window. Repeat the starting number selection and coverage range determination operation for all time slices in turn to generate the time slice number range corresponding to multiple extended windows. For each time slice number range, the base layer data blocks belonging to the current time slice number range are sequentially retrieved in the hierarchical data blocks, and combined in the order of time slice numbers to form the base layer initial expansion window. At the same time, the enhancement layer data blocks are retrieved to form the enhancement layer initial expansion window, and each pair of base layer initial expansion windows and enhancement layer initial expansion windows is stored as a hierarchical expansion window.

[0010] Optionally, the preparation process for the improved RS inequality error correction specifically includes: Read the hierarchical expansion windows, organize the basic layer expansion windows and enhancement layer expansion windows corresponding to each expansion window number, establish an expansion window index table arranged in the order of expansion window numbers, and treat the basic layer expansion windows and enhancement layer expansion windows under the same number as a window unit to be processed. An improved RS unequal error correction execution strategy is set for each window unit to be processed. The improved RS unequal error correction refers to the differential window division, priority setting of important regions, and time phase coding scheduling while keeping the basic coding rules of Reed-Solomon erasure coding unchanged. This enables the basic layer data to obtain higher recovery priority and the enhancement layer data to obtain additional error correction supplementation capabilities. The error correction execution strategy is limited to three types of strategies in sequence: multi-terminal stacked extended window strategy, content-aware key region priority protection strategy, and window time-series multi-phase coding strategy. According to the preset strategy execution order, the three types of strategies in the improved RS inequality error correction are executed sequentially for the basic layer extended window and the enhancement layer extended window corresponding to each number in the extended window index table.

[0011] Optionally, obtaining the terminal's overlay extended window specifically includes: The system acquires terminal-side service characteristic parameters for each access terminal, normalizes network bandwidth, latency, packet loss rate, service priority, and processing capability, and uses constrained fuzzy hierarchical clustering to cluster the terminals. During the clustering iteration process, service priority constraints and minimum distinguishability constraints are applied simultaneously. The service priority constraints are based on the system's preset priority configuration for different service categories, and the minimum distinguishability constraints are based on the system's minimum difference requirement between different terminal levels. This results in a set of terminal levels with different level weights and target window coverage ratios, and each terminal level is assigned a corresponding level number and level weight. For each window unit to be processed in the hierarchical expansion window, the hierarchical number and weight of the serviceable terminal are read. An envelope optimization and overlay are used to construct a terminal stacked expansion window structure. The basic layer expansion window and the enhancement layer expansion window are used as the initial envelope. Under the condition of satisfying the target window coverage ratio of each terminal level, the window envelope expansion and overlay operations are performed sequentially from low to high according to the hierarchical weight. This ensures that the window corresponding to the higher-level terminal includes the window corresponding to the lower-level terminal in terms of time slice coverage and data protection range, thus generating a terminal stacked expansion window with differentiated coverage range.

[0012] Optionally, the formation of the key area expansion window specifically includes: Each terminal's overlay expansion window is associated with the covered time slice number range. Based on the time slice number range, the corresponding original video frame sequence is extracted from the input data of the video public call service. Each terminal's overlay expansion window is then associated with a set of consecutive video frame sequences as the input object for the critical area priority protection strategy. For each set of video frame sequences corresponding to the terminal's overlay extended window, content-aware analysis processing is performed on each frame. A preset face detection and facial key point recognition model is used. The facial key point recognition model is a pre-trained regression model used to locate multiple facial key points in the detected face area, including eyes, nose tip, corner of mouth, lip contour points, and chin contour points. The position of each key point in the image coordinate system is output to define the precise spatial range of the face area and mouth area, determine the spatial position of the face area and mouth area in the current frame image, and record the pixel set belonging to the face area and mouth area as the feature area set of the current frame. According to the video coding mapping relationship, the coding unit corresponding to each feature area is marked as the feature area data symbol set, so that the feature area data symbol set corresponding to each terminal overlay extended window can be determined. For each terminal's layered extended window, based on the set of feature region data symbols, basic layer data symbols and enhanced layer data symbols are extracted from the basic layer extended window and the enhanced layer extended window. The data symbols are grouped according to the time slice order to form key region sub-windows. The key region sub-windows are then combined with the original basic layer extended window and enhanced layer extended window to form a content-aware key region extended window.

[0013] Optionally, the formation of the multiphase extended window sequence specifically includes: The key area extended windows under the same extended window number are paired one by one with the terminal stacked extended windows to form a window pairing sequence. The window pairing sequence is arranged in the order of the extended window numbers to generate a window pairing processing queue, which is used as the input of the window timing multiphase coding strategy and is composed of window pairing units. For each window pairing unit in the window pairing processing queue, the window start time parameter and the timing phase offset parameter are initialized. Using hierarchical weighting, the window start time parameter is weighted and normalized according to the time slice range covered by the window and the hierarchical weight of the terminal level. Through multi-phase offset allocation, the timing phase offset parameter is iteratively calculated and discretized according to the extended window number, the order of the terminal level and the minimum time interval constraint between windows. The calculated window start time parameter and the timing phase offset parameter with limited value range are written into the corresponding window pairing unit respectively. The window pairing processing queue is traversed, and all window pairing units are sorted according to the order of the window start time parameters. The window start time parameters and timing phase offset parameters are read sequentially. Using window timing multiphase encoding, time axis translation, phase misalignment and timing insertion operations are performed on the key area expansion window and the terminal stacked expansion window. Non-overlapping activation time slots are allocated to each window on the discrete time axis. Under the condition of ensuring that the minimum phase interval constraint is met between adjacent windows, the phase interleaving arrangement of all windows is completed, generating a multiphase expansion window sequence composed of multiple phase interleaved expansion windows.

[0014] Optionally, the generation of the joint redundancy symbol specifically includes: For each multiphase extended window sequence, establish a window cell number and record the time phase. Concatenate the base layer data symbol sequence and the enhancement layer data symbol sequence into a data symbol sequence. Record the total number of symbols, the ratio between the base layer and the enhancement layer, and the correspondence with the time phase to form the coded input set for the improved RS inequality error correction. Construct a weight sequence and nonlinear amplification coefficient based on the principle of prioritizing the base layer. Perform nonlinear amplification on the weight sequence of each data symbol and sum it within the current window to obtain the comprehensive importance of the window unit. Sum the comprehensive importance of all window units to obtain the total importance. Set the total budget for redundant symbols in this batch. Allocate redundant symbols according to the total importance of each window unit so that the redundancy ratio corresponding to the base layer is higher than that of the enhancement layer. Determine the sum of the number of data symbols and the number of redundant symbols in each window unit as the target code length to form a non-uniform redundancy allocation configuration that adapts to importance. For each window cell, an artificially perturbed RS generation matrix related to the time phase is constructed. The elements of the matrix are obtained by repeatedly multiplying the original meta-parameters associated with the time phase and level ratio of the current window cell. The data symbol sequence is encoded using RS encoding. The output codeword has the original data symbol at the beginning and the redundant symbol generated by this window cell at the end. The redundant symbols of the base layer and the enhancement layer are merged and a correspondence is established between the window cell number and the time phase to form a joint redundant symbol.

[0015] Optionally, the output of the recovered video public call data specifically includes: The joint redundancy symbols and hierarchical data blocks are encapsulated in multi-phase extended window units. Each unit is written with a window number, time phase and payload length identifier. They are arranged into a transmission sequence according to time phase priority and number order. The start and end phases and window index range are recorded in the sequence header, and verification information and timestamp are written. The receiving end parses the transmission sequence according to the time phase and window number, reconstructs the receiving set of each window unit, counts the number of successfully received data symbols and the number of joint redundant symbols, records the total number of original data symbols and the target code length, generates an erasure location list according to the arrival order, and establishes a joint redundancy pool indexed by phase and number. Decoding is initiated for each window unit. If the sum of the received data symbols and joint redundancy reaches the total number of original data symbols, direct decoding is performed. Otherwise, redundancy is compensated by allocating from the redundancy pool according to the total importance. Decoding is performed after the conditions are met, and the data of the base layer and enhancement layer are merged according to the time phase and the recovered video public voice data is output.

[0016] The beneficial effects of this invention are: This invention is based on constrained fuzzy hierarchical clustering and envelope-based layered construction, enabling terminals with different service priorities and network capabilities to obtain differentiated protection under the same redundancy budget. The window coverage and recovery probability of higher-level terminals are significantly improved, solving the problem of insufficient protection in concurrent scenarios of existing terminal-by-terminal adaptation. This invention uses a content-aware key region expansion window to explicitly aggregate and prioritize the protection of symbols corresponding to semantic key regions such as face and lip movements. The key region symbol density further drives redundancy allocation and phase scheduling, making it easier to achieve speech coherence and lip-sync under weak network conditions. The window-time multiphase coding of the present invention performs phase interleaving on the extended window on the time axis, breaks up the temporal correlation of symbols within the same time slice, significantly weakens the concentrated impact of sudden packet loss on a single window, and improves the recoverability of erase decoding. The joint redundancy cross-layer sharing mechanism of the present invention provides filling when there are insufficient symbols in the base layer or enhancement layer. Combined with the hierarchical recovery strategy of first base layer and then enhancement layer, it reduces the recovery threshold and the probability of stuttering, and improves the first frame presentation and continuous playback experience. The improved RS inequality error correction and preservation method of the present invention is compatible with the system-type decoding process, without the need to replace the receiver decoder, and can be quickly deployed in the existing network through policy upgrade. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 The flowchart below shows a video public phone intelligent communication system that supports multi-terminal interconnection, as proposed in this invention. Figure 2 This is a schematic diagram of the algorithm structure of a video public phone intelligent communication system that supports multi-terminal interconnection proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1-2 A video public phone intelligent communication system supporting multi-terminal interconnection includes: The time-slice hierarchical module is used to divide the video public call data stream according to preset time slices, forming a hierarchical data block set; The extended window construction module is used to combine hierarchical data blocks into initial extended windows of the base layer and enhancement layer covering different consecutive time slices based on the window length parameter, and generate hierarchical extended windows. The error correction strategy orchestration module is used to assemble three types of strategies: multi-terminal stacking of improved RS uneven error correction, priority protection of key areas, and window timing multiphase coding. The terminal overlay construction module is used to perform constrained fuzzy hierarchical clustering based on terminal-side service feature parameters and generate terminal overlay extended windows. The key region identification module is used to perform temporal saliency pre-screening, face detection and key point recognition on the video frames corresponding to the window to form an expanded window of the key region. The multiphase timing coding module is used to pair the key region expansion window with the terminal stacked expansion window to generate a multiphase expansion window sequence; The joint redundancy and recovery module is used to allocate redundancy budgets based on the multiphase extended window sequence and perform perturbation-type RS coding to generate joint redundancy symbols, thereby completing hierarchical data recovery.

[0020] In this embodiment, the modules are interconnected using the following method: The input data stream of the video public call service is obtained and divided into time slices to obtain hierarchical data blocks; An initial extended window covering different consecutive time slices is constructed based on hierarchical data blocks, and a hierarchical extended window is generated. The process of receiving a hierarchical expansion window and completing the preparation process for improved RS inequality error correction includes a multi-terminal stacked expansion window strategy, a content-aware key area priority protection strategy, and a window timing multiphase coding strategy. A multi-terminal overlay expansion window strategy is adopted for the hierarchical expansion window. Based on the terminal-side service characteristic parameters of the access terminals, the terminal dimensions are divided and overlaid to obtain a terminal overlay expansion window with differentiated coverage. Based on the key area priority protection strategy, content-aware analysis is performed on video frames to identify key feature areas including face areas and mouth areas, and data symbols are added to the key area sub-window of the hierarchical expansion window to form a content-aware key area expansion window. By configuring different window start times and timing phase offsets for the key region extended window and the terminal stacked extended window input window timing multiphase coding strategy, a multiphase extended window sequence with phase interleaving on the time axis is generated. Based on the multiphase extended window sequence, and using the improved RS unequal error correction, erasure coding is performed to generate joint redundant symbols. The joint redundancy symbols are encapsulated with hierarchical data blocks, and data recovery is performed at the receiving end. When there are insufficient symbols, the joint redundancy symbols are used for compensation, and the recovered video public voice data is output.

[0021] In this embodiment, obtaining the hierarchical data block specifically includes: The input data stream of the video public call service is obtained, the continuously arriving bit sequence in the input data stream is cached sequentially, and the data is divided in order according to the preset time slice length. During the division process, when the time length corresponding to the cached data reaches the preset time slice length parameter, the currently cached data is divided into a time slice, and a unique time slice number is assigned to each time slice. For each time slice, the cached data corresponding to the current time slice is classified and processed according to the preset hierarchical encoding rules. The part carrying voice information, character outline information and key control information is integrated into the basic layer data block, and the part carrying image detail texture information, resolution enhancement information and visual effect enhancement information is integrated into the enhancement layer data block. The base layer data blocks and enhancement layer data blocks corresponding to each time slice are organized in the order of time slice number to form hierarchical data blocks.

[0022] In this embodiment, the process of generating the hierarchical expansion window specifically includes: Read the hierarchical data blocks, organize the time slices in numerical order, and maintain the one-to-one correspondence between the base layer data blocks and the enhancement layer data blocks in the same time slice; Set the extended window length parameter and determine the number of consecutive time slices covered by each extended window. Select a time slice number from the hierarchical data block set as the starting number of the extended window. Starting from the starting number, sequentially select adjacent time slices to form the coverage range of the extended window. Repeat the starting number selection and coverage range determination operation for all time slices in turn to generate the time slice number range corresponding to multiple extended windows. For each time slice number range, the base layer data blocks belonging to the current time slice number range are sequentially retrieved in the hierarchical data blocks, and combined in the order of time slice numbers to form the base layer initial expansion window. At the same time, the enhancement layer data blocks are retrieved to form the enhancement layer initial expansion window, and each pair of base layer initial expansion windows and enhancement layer initial expansion windows is stored as a hierarchical expansion window.

[0023] In this embodiment, the preparation process for the improved RS inequality error correction specifically includes: Read the hierarchical expansion windows, organize the basic layer expansion windows and enhancement layer expansion windows corresponding to each expansion window number, establish an expansion window index table arranged in the order of expansion window numbers, and treat the basic layer expansion windows and enhancement layer expansion windows under the same number as a window unit to be processed. An improved RS unequal error correction execution strategy is set for each window unit to be processed. The improved RS unequal error correction refers to the differential window division, priority setting of important regions, and time phase coding scheduling while keeping the basic coding rules of Reed-Solomon erasure coding unchanged. This enables the basic layer data to obtain higher recovery priority and the enhancement layer data to obtain additional error correction supplementation capabilities. The error correction execution strategy is limited to three types of strategies in sequence: multi-terminal stacked extended window strategy, content-aware key region priority protection strategy, and window time-series multi-phase coding strategy. According to the preset strategy execution order, the three types of strategies in the improved RS inequality error correction are executed sequentially for the basic layer extended window and the enhancement layer extended window corresponding to each number in the extended window index table.

[0024] In this embodiment, obtaining the terminal's overlay extended window specifically includes: The system acquires terminal-side service characteristic parameters for each access terminal, normalizes network bandwidth, latency, packet loss rate, service priority, and processing capability, and uses constrained fuzzy hierarchical clustering to cluster the terminals. During the clustering iteration process, service priority constraints and minimum distinguishability constraints are applied simultaneously. The service priority constraints are based on the system's preset priority configuration for different service categories, and the minimum distinguishability constraints are based on the system's minimum difference requirement between different terminal levels. This results in a set of terminal levels with different level weights and target window coverage ratios, and each terminal level is assigned a corresponding level number and level weight. For each window unit to be processed in the hierarchical expansion window, the hierarchical number and weight of the serviceable terminal are read. An envelope optimization and overlay are used to construct a terminal stacked expansion window structure. The basic layer expansion window and the enhancement layer expansion window are used as the initial envelope. Under the condition of satisfying the target window coverage ratio of each terminal level, the window envelope expansion and overlay operations are performed sequentially from low to high according to the hierarchical weight. This ensures that the window corresponding to the higher-level terminal includes the window corresponding to the lower-level terminal in terms of time slice coverage and data protection range, thus generating a terminal stacked expansion window with differentiated coverage range.

[0025] By introducing fuzzy hierarchical clustering with business priority constraints and minimum discriminative constraints, and combining it with envelope optimization layering, differentiated protection is achieved based on terminal-side business characteristic parameters under the same redundancy budget, thereby improving fairness, latency stability and bandwidth utilization in concurrent scenarios.

[0026] In this embodiment, the formation of the key area expansion window specifically includes: Each terminal's overlay expansion window is associated with the covered time slice number range. Based on the time slice number range, the corresponding original video frame sequence is extracted from the input data of the video public call service. Each terminal's overlay expansion window is then associated with a set of consecutive video frame sequences as the input object for the critical area priority protection strategy. For each set of video frame sequences corresponding to the terminal's overlay extended window, content-aware analysis processing is performed on each frame. A preset face detection and facial key point recognition model is used. The facial key point recognition model is a pre-trained regression model used to locate multiple facial key points in the detected face area, including eyes, nose tip, corner of mouth, lip contour points, and chin contour points. The position of each key point in the image coordinate system is output to define the precise spatial range of the face area and mouth area, determine the spatial position of the face area and mouth area in the current frame image, and record the pixel set belonging to the face area and mouth area as the feature area set of the current frame. According to the video coding mapping relationship, the coding unit corresponding to each feature area is marked as the feature area data symbol set, so that the feature area data symbol set corresponding to each terminal overlay extended window can be determined. For each terminal's layered extended window, based on the set of feature region data symbols, basic layer data symbols and enhanced layer data symbols are extracted from the basic layer extended window and the enhanced layer extended window. The data symbols are grouped according to the time slice order to form key region sub-windows. The key region sub-windows are then combined with the original basic layer extended window and enhanced layer extended window to form a content-aware key region extended window.

[0027] Content-aware key region expansion windows map semantic regions such as faces and lip shapes to high-priority symbol sets, and aggregate and align them within the window to achieve differentiated protection of key speech information and lip-shape details. In weak network and sudden packet loss scenarios, it significantly improves recoverability and subjective clarity, and reduces mosaic and stuttering. At the same time, it guides redundancy to key regions with detection results, improves redundancy utilization efficiency and bandwidth utilization, and is compatible with RS decoding.

[0028] In this embodiment, the formation of the multiphase extended window sequence specifically includes: The key area extended windows under the same extended window number are paired one by one with the terminal stacked extended windows to form a window pairing sequence. The window pairing sequence is arranged in the order of the extended window numbers to generate a window pairing processing queue, which is used as the input of the window timing multiphase coding strategy and is composed of window pairing units. For each window pairing unit in the window pairing processing queue, the window start time parameter and the timing phase offset parameter are initialized. Using hierarchical weighting, the window start time parameter is weighted and normalized according to the time slice range covered by the window and the hierarchical weight of the terminal level. Through multi-phase offset allocation, the timing phase offset parameter is iteratively calculated and discretized according to the extended window number, the order of the terminal level and the minimum time interval constraint between windows. The calculated window start time parameter and the timing phase offset parameter with limited value range are written into the corresponding window pairing unit respectively. The window pairing processing queue is traversed, and all window pairing units are sorted according to the order of the window start time parameters. The window start time parameters and timing phase offset parameters are read sequentially. Using window timing multiphase encoding, time axis translation, phase misalignment and timing insertion operations are performed on the key area expansion window and the terminal stacked expansion window. Non-overlapping activation time slots are allocated to each window on the discrete time axis. Under the condition of ensuring that the minimum phase interval constraint is met between adjacent windows, the phase interleaving arrangement of all windows is completed, generating a multiphase expansion window sequence composed of multiple phase interleaved expansion windows.

[0029] By pairing key region windows with terminal overlay windows and setting start times and phase offsets according to hierarchical weighting, multi-phase interleaving and mutually exclusive time slot allocation are achieved on the time axis, breaking down temporal correlation and collision of symbols in the same chip, reducing the risk of sudden packet loss, improving the success rate of erasure decoding and the reachability of the first packet, prioritizing the continuity of basic voice and facial lip movements, reducing stuttering and heavy buffering, being compatible with low-cost deployment on the existing network, and improving the fairness and latency stability of multi-terminal concurrent scenarios.

[0030] In this embodiment, the generation of the joint redundancy symbol specifically includes: For each multiphase extended window sequence, establish a window cell number and record the time phase. Concatenate the base layer data symbol sequence and the enhancement layer data symbol sequence into a data symbol sequence. Record the total number of symbols, the ratio between the base layer and the enhancement layer, and the correspondence with the time phase to form the coded input set for the improved RS inequality error correction. Construct a weight sequence and nonlinear amplification coefficient based on the principle of prioritizing the base layer. Perform nonlinear amplification on the weight sequence of each data symbol and sum it within the current window to obtain the comprehensive importance of the window unit. Sum the comprehensive importance of all window units to obtain the total importance. Set the total budget for redundant symbols in this batch. Allocate redundant symbols according to the total importance of each window unit so that the redundancy ratio corresponding to the base layer is higher than that of the enhancement layer. Determine the sum of the number of data symbols and the number of redundant symbols in each window unit as the target code length to form a non-uniform redundancy allocation configuration that adapts to importance. For each window cell, an artificially perturbed RS generation matrix related to the time phase is constructed. The elements of the matrix are obtained by repeatedly multiplying the original meta-parameters associated with the time phase and level ratio of the current window cell. The data symbol sequence is encoded using RS encoding. The output codeword has the original data symbol at the beginning and the redundant symbol generated by this window cell at the end. The redundant symbols of the base layer and the enhancement layer are merged and a correspondence is established between the window cell number and the time phase to form a joint redundant symbol.

[0031] This invention uses a base layer priority weight and nonlinear amplification to drive redundant adaptive allocation, and introduces a Reed-Solomon generation matrix with time-varying phase perturbation to enhance codeword independence and cross-layer joint redundancy utilization. It improves recoverability and image continuity under sudden packet loss and asynchronous jitter, takes into account speech coherence and lip-sync, and maintains system-wide decoding compatibility, thereby reducing modification costs.

[0032] In this embodiment, the output of the recovered video public phone data specifically includes: The joint redundancy symbols and hierarchical data blocks are encapsulated in multi-phase extended window units. Each unit is written with a window number, time phase and payload length identifier. They are arranged into a transmission sequence according to time phase priority and number order. The start and end phases and window index range are recorded in the sequence header, and verification information and timestamp are written. The receiving end parses the transmission sequence according to the time phase and window number, reconstructs the receiving set of each window unit, counts the number of successfully received data symbols and the number of joint redundant symbols, records the total number of original data symbols and the target code length, generates an erasure location list according to the arrival order, and establishes a joint redundancy pool indexed by phase and number. Decoding is initiated for each window unit. If the sum of the received data symbols and joint redundancy reaches the total number of original data symbols, direct decoding is performed. Otherwise, redundancy is compensated by allocating from the redundancy pool according to the total importance. Decoding is performed after the conditions are met, and the data of the base layer and enhancement layer are merged according to the time phase and the recovered video public voice data is output.

[0033] Example 1: To verify the feasibility of this invention in practice, it was applied to a video-based public phone intelligent communication system for students, simultaneously supporting student identification, mental health monitoring, home-school communication and collaboration, and remote family interaction without mobile phone restrictions. The test environment was a typical hybrid access system of campus LAN and home broadband, with terminals including classroom public phone terminals, dormitory corridor monitoring terminals, dedicated terminals in the psychological counseling room, and remote video terminals for parents. Students did not carry mobile phones and relied on campus public phone equipment for identity verification, while parents accessed the authorization channel via a WeChat mini-program. Significant differences existed in network stability and terminal capabilities in the scenario, especially in the dormitory corridor area where bandwidth sharing fluctuated, requiring the psychological monitoring terminal to prioritize image clarity in key areas. This invention verifies the requirements of this type of educational scenario.

[0034] During application, when a student initiates a call request in front of a public phone terminal, the system first performs time-slicing on the input video public phone stream to obtain hierarchical data blocks of the base layer and enhancement layer, and then constructs hierarchical expansion windows across time in batches. Based on the improved RS unequal error correction strategy of this invention, the system reads service characteristic parameters from different terminal types, such as psychological counseling room terminals having the highest priority, parent terminals in the middle layer, and ordinary classroom terminals at the bottom layer. By constrained fuzzy hierarchical clustering, a terminal stacked expansion window with differentiated hierarchical weights is formed, maximizing the coverage of psychological terminals, followed by parent terminals, and minimizing the coverage of ordinary classroom terminals, thereby ensuring that key scenarios for psychological monitoring can still obtain the highest recovery capability even under weak network conditions.

[0035] Subsequently, the system automatically extracts the time-slice video frames corresponding to each terminal window, and uses face detection and key point recognition models to identify key psychological areas of concern, such as the student's face, eyes, and mouth, mapping these areas to a set of key region symbols. The key region expansion window constructed in this invention can protect local facial details of students with high priority. During the psychological state analysis stage, the system further calls natural language processing and emotion recognition models to extract keywords from the student's speech content, such as anxiety and stress, and analyzes them synchronously with facial expression features in the video, thus forming a multimodal psychological monitoring link.

[0036] Building upon this foundation, the critical region expansion window and terminal overlay window are encoded using a multi-phase temporal coding strategy. The system calculates the window start time and phase offset based on the weights of different terminal levels. By distributing the windows through time axis misalignment and phase interleaving, the system significantly reduces symbol concentration impairment within the same time slice, ensuring stable call quality for both the psychological and parent terminals even under weak network fluctuations. Subsequently, the system performs redundant symbol budgeting based on the base layer priority weights, the density of psychological critical regions, and multi-phase temporal characteristics. It then employs a generator matrix with time-varying phase perturbations to perform RS coding, generating joint redundant symbols. In the event of bandwidth fluctuations or sudden packet loss, the receiver utilizes the joint redundancy pool to prioritize the recovery of the student's face region and outputs high-fidelity video and audio recordings suitable for psychological state analysis after the call ends.

[0037] Traditional methods typically employ single-terminal video calls based on adaptive bitrates, coupled with statically allocated redundant RS (Recurrent Signals) at the coding layer and fixed-time interleaving. These methods only perform simple packet loss compensation and bandwidth adaptation, lacking the ability to jointly optimize based on student identity, psychologically critical regions, and multi-terminal priorities. To further demonstrate the pain points addressed by this invention in educational scenarios, the verification focuses on the following two typical stress scenarios: first, high-impact bandwidth competition in dormitories and corridors, manifested as continuous time-slice packet loss when students initiate home-school communication video calls during peak evening hours; second, the risk of call discontinuity and information misinterpretation due to network fluctuations on the parent's end. To ensure the reproducibility of the experiment, a unified network perturbation script was used to construct replayable link conditions. The key indicators of the traditional scheme and this invention were compared, and the experimental results are shown in Table 1. Table 1 Comparison of Key Indicators in Educational Scenarios

[0038] As shown in Table 1, this invention achieves significant benefits in multi-terminal interconnected educational scenarios: the success rate of student identification is increased to 99.1%, the image quality of the psychological expression area is improved by 5.7 dB, making emotional details easier for the psychological analysis engine to capture; the accuracy rate of emotional keyword extraction is improved by 10.5 percentage points, the continuity of home-school video calls is significantly enhanced, and the image recovery rate reaches 97.2% under weak network conditions; the first frame presentation time is reduced by 0.85 seconds, making remote interaction between parents and students smoother; the success rate of video calls in multi-person binding management scenarios is improved by 9.4 percentage points, while saving 4% of redundant budget, proving the engineering feasibility and core value of this invention in intelligent communication, psychological monitoring, and home-school interaction systems.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A video public phone intelligent communication system supporting multi-terminal interconnection, characterized in that, include: The time-slice hierarchical module is used to divide the video public call data stream according to preset time slices, forming a hierarchical data block set; The extended window construction module is used to combine hierarchical data blocks into initial extended windows of the base layer and enhancement layer covering different consecutive time slices based on the window length parameter, and generate hierarchical extended windows. The error correction strategy orchestration module is used to assemble three types of strategies: multi-terminal stacking of improved RS uneven error correction, priority protection of key areas, and window timing multiphase coding. The terminal overlay construction module is used to perform constrained fuzzy hierarchical clustering based on terminal-side service feature parameters and generate terminal overlay extended windows. The key region identification module is used to perform temporal saliency pre-screening, face detection and key point recognition on the video frames corresponding to the window to form an expanded window of the key region. The multiphase timing coding module is used to pair the key region expansion window with the terminal stacked expansion window to generate a multiphase expansion window sequence; The joint redundancy and recovery module is used to allocate redundancy budgets based on the multiphase extended window sequence and perform perturbation-type RS coding to generate joint redundancy symbols, thereby completing hierarchical data recovery.

2. The intelligent video public phone communication system supporting multi-terminal interconnection according to claim 1, characterized in that, The modules are connected in the following way: The input data stream of the video public call service is obtained and divided into time slices to obtain hierarchical data blocks; An initial extended window covering different consecutive time slices is constructed based on hierarchical data blocks, and a hierarchical extended window is generated. The process of receiving a hierarchical expansion window and completing the preparation process for improved RS inequality error correction includes a multi-terminal stacked expansion window strategy, a key area priority protection strategy, and a window timing multiphase coding strategy. A multi-terminal overlay expansion window strategy is adopted for the hierarchical expansion window, which divides and overlays the terminal dimensions to obtain a terminal overlay expansion window. Based on the key area priority protection strategy, content-aware analysis is performed on video frames to identify key feature areas and form a key area expansion window. The key region extended window and the terminal stacked extended window input window timing multiphase coding strategy are configured with different window start times and timing phase offsets to generate a multiphase extended window sequence. Based on the multiphase extended window sequence, and using the improved RS unequal error correction, erasure coding is performed to generate joint redundant symbols. The joint redundancy symbols are encapsulated with hierarchical data blocks, and data recovery is performed at the receiving end. When there are insufficient symbols, the joint redundancy symbols are used for compensation, and the recovered video public voice data is output.

3. The intelligent video public phone communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The acquisition of the hierarchical data blocks specifically includes: The input data stream of the video public call service is obtained, the continuously arriving bit sequence in the input data stream is cached sequentially, and the data is divided into segments according to a preset time slice length, and a unique time slice number is assigned to each time slice. For each time slice, the cached data corresponding to the current time slice is classified and processed according to the preset hierarchical encoding rules. The part carrying voice information, character outline information and key control information is integrated into the basic layer data block, and the part carrying image detail texture information, resolution enhancement information and visual effect enhancement information is integrated into the enhancement layer data block. The base layer data blocks and enhancement layer data blocks corresponding to each time slice are organized in the order of time slice number to form hierarchical data blocks.

4. The intelligent video public phone communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The process of generating the hierarchical expansion window specifically includes: Read the hierarchical data blocks, organize the time slices in numerical order, and maintain the one-to-one correspondence between the base layer data blocks and the enhancement layer data blocks in the same time slice; Set the extended window length parameter, select a time slice number from the hierarchical data block set as the starting number of the extended window, and sequentially select adjacent time slices to form the coverage range of the extended window, and repeat the starting number selection and coverage range determination operation for all time slices in turn, which is used for the time slice number range; For each time slice number range, the base layer data blocks belonging to the current time slice number range are sequentially retrieved in the hierarchical data blocks to form the base layer initial expansion window. At the same time, the enhancement layer data blocks are retrieved to form the enhancement layer initial expansion window. Each pair of base layer initial expansion windows and enhancement layer initial expansion windows is stored as a hierarchical expansion window.

5. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The preparation process for the improved RS inequality error correction specifically includes: Read the hierarchical expansion windows, organize the basic layer expansion windows and enhancement layer expansion windows corresponding to each expansion window number, establish an expansion window index table arranged in the order of expansion window numbers, and treat the basic layer expansion windows and enhancement layer expansion windows under the same number as a window unit to be processed. An improved RS unequal error correction execution strategy is set for each window unit to be processed. The improved RS unequal error correction refers to the differential window division, priority setting of important regions, and time phase coding scheduling while keeping the basic coding rules of Reed-Solomon erasure coding unchanged. According to the preset strategy execution order, the three types of strategies in the improved RS inequality error correction are executed sequentially for the basic layer extended window and the enhancement layer extended window corresponding to each number in the extended window index table.

6. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The specific steps involved in obtaining the terminal's overlay-style extended window are as follows: The terminal-side service characteristic parameters of each access terminal are obtained, and the network bandwidth, latency, packet loss rate, service priority and processing capability are normalized. Constrained fuzzy hierarchical clustering is used to cluster the terminals. During the clustering iteration process, service priority constraints and minimum discriminant constraints are applied simultaneously to obtain a set of terminal levels. A corresponding level number and level weight are assigned to each terminal level. For each window unit to be processed in the hierarchical expansion window, the hierarchical number and hierarchical weight of the serviceable terminal are read. The terminal stacked expansion window structure is constructed by envelope optimization and overlay. The basic layer expansion window and the enhancement layer expansion window are used as the initial envelope. The window envelope expansion and overlay operations are performed in sequence from low to high according to the hierarchical weight to generate a terminal stacked expansion window with differentiated coverage.

7. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The formation of the key area expansion window specifically includes: Each terminal's overlay expansion window is associated with the covered time slice number range. Based on the time slice number range, the corresponding original video frame sequence is extracted from the input data of the video public call service. A one-to-one correspondence is established between each terminal's overlay expansion window and a set of consecutive video frame sequences. For each set of video frame sequences corresponding to the terminal's overlay extended window, content-aware analysis processing is performed on each frame. A preset face detection and facial key point recognition model is used to determine the spatial location of the face region and mouth region in the current frame image. The set of pixels belonging to the face region and mouth region is recorded as the feature region set of the current frame. According to the video coding mapping relationship, the coding unit corresponding to each feature region is marked as the feature region data symbol set. For each terminal's layered extended window, based on the set of feature region data symbols, basic layer data symbols and enhanced layer data symbols are extracted from the basic layer extended window and the enhanced layer extended window. The data symbols are grouped according to the time slice order to form key region sub-windows. The key region sub-windows are then combined with the original basic layer extended window and enhanced layer extended window to form a content-aware key region extended window.

8. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The formation of the multiphase extended window sequence specifically includes: The key area extended windows under the same extended window number are paired one by one with the terminal stacked extended windows to form a window pairing sequence. The window pairing sequence is arranged according to the order of the extended window numbers to generate a window pairing processing queue, which is composed of window pairing units. For each window pairing unit in the window pairing processing queue, the window start time parameter and the timing phase offset parameter are initialized. Using hierarchical weighting, the window start time parameter is weighted and normalized according to the time slice range covered by the window and the hierarchical weight of the terminal level to which it belongs. The timing phase offset parameter is iteratively calculated and discretely quantized through multi-phase offset allocation. The calculated window start time parameter and the timing phase offset parameter with limited value range are written into the corresponding window pairing unit respectively. Traverse the window pairing processing queue, sort all window pairing units, read the window start time parameter and timing phase offset parameter in sequence, use window timing multiphase encoding, perform time axis translation, phase misalignment and timing insertion operations, allocate non-overlapping activation time slots to each window on the discrete time axis, and complete the phase interleaving arrangement of all windows to generate a multiphase extended window sequence.

9. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The generation of the joint redundancy symbol specifically includes: For each multiphase extended window sequence, establish a window cell number and record the time phase. Concatenate the base layer data symbol sequence and the enhancement layer data symbol sequence to form a data symbol sequence, thus forming a set of encoded inputs. Construct a weight sequence and nonlinear amplification coefficient based on the principle of prioritizing the base layer. Perform nonlinear amplification on the weight sequence of each data symbol and sum it within the current window to obtain the comprehensive importance of the window unit. Sum the comprehensive importance of all window units to obtain the total importance. Set the total budget for redundant symbols in this batch. Allocate redundant symbols according to the total importance of each window unit. Determine the sum of the number of data symbols and the number of redundant symbols in each window unit as the target code length to form a non-uniform redundancy allocation configuration. For each window cell, an artificially perturbed RS generation matrix related to the time phase is constructed. The data symbol sequence is encoded using RS encoding. Redundant symbols in the base layer and enhancement layer are merged. A correspondence is established between the window cell number and the time phase to form joint redundant symbols.

10. A video public phone intelligent communication system supporting multi-terminal interconnection according to claim 2, characterized in that, The output of the recovered video public call data specifically includes: The joint redundant symbols and hierarchical data blocks are encapsulated in multi-phase extended window units. Each unit is written with a window number, time phase, and payload length identifier, and arranged into a transmission sequence according to time phase priority and number order. The receiving end parses the transmission sequence according to the time phase and window number, reconstructs the receiving set of each window unit, generates an erasure location list according to the arrival order, and establishes a joint redundancy pool indexed by phase and number. Decoding is initiated for each window unit. The received data symbols and the sum of joint redundancy are directly decoded to reach the total number of original data symbols. The data is then mapped back to the time slice according to the time phase, and the base layer and enhancement layer data are merged. The recovered video public voice data is then output.

Images