A video backhaul adaptive code rate control method and system for underground coal mine image communication

Abstract

The present application relates to the field of coal mine underground communication and video transmission, and particularly relates to a video backhaul adaptive code rate control method for coal mine underground image communication, comprising: obtaining original video frames and generating layered backhaul video code streams; encapsulating and sending the layered backhaul video code streams; receiving end generating feedback messages and backhauling; sending end determining a switching window and predicting a discontinuity duration; generating a synchronization digest within the switching window and sending it with base layer code stream digital information packet; performing a check on the synchronization digest within the switching window; within the switching window and during the reconstruction of the closed loop, making the joint configuration vector satisfy the overhead budget constraint; after the end of the switching window, exiting cross-link replication transmission and control telegram repeated transmission, and encoding, encapsulating and sending subsequent original video frames according to the updated target code rate and key frame interval. The present application realizes continuous decodable transmission and fast synchronization recovery of base layer information under switching disturbance.

Description

Technical Field

[0001] This invention relates to the field of underground communication and video transmission in coal mines, and in particular to an adaptive bitrate control method and system for video backhaul in underground image communication in coal mines. Background Technology

[0002] Backhaul services for underground coal mine image communication typically require continuous video uplink transmission under conditions of limited bandwidth, significant latency, and packet loss fluctuations. To improve the availability of video transmission in weak network environments, existing technologies often employ scalable video coding to split the video into a base layer and an enhancement layer, and utilize multipath or multi-network bearer methods to improve the reliability of the base layer's arrival. When network conditions permit, an enhancement layer is then superimposed to improve video quality.

[0003] For example, the prior art, publication number CN103338410B, discloses a "Real-time Streaming Media Reassembly Algorithm Based on SVC under Multi-path Parallel Transmission." At the sending end, the video is split into basic layer data and enhancement layer data according to H.264 SVC format, and transmitted separately through different networks. At the receiving end, the received data is buffered and reassembled into frames to achieve real-time streaming media playback and image quality adjustment under multi-path parallel transmission conditions. This scheme focuses on the layered bitstream carrying on multiple paths and the receiver reassembly processing flow to improve playback continuity and support a certain degree of image quality adaptation.

[0004] However, in underground coal mine scenarios, backhaul links may experience short-term interruptions, sudden packet loss, and out-of-order arrivals due to factors such as switching between different access devices or backhaul links, wireless obstruction, and multipath fading. While existing technologies provide multipath transmission and receiver reassembly approaches for layered bitstreams, they do not address using advance notice of access device or backhaul link switching to determine the time window before and after the switch. Furthermore, they do not explicitly provide a mechanism for coordinated adjustment of critical transmission and encoding configurations during the backhaul process, supporting synchronous and rapid recovery, in response to short-term interruptions, sudden packet loss, and out-of-order arrivals caused by switching disturbances, under conditions of limited redundancy overhead. Consequently, before and after a switching disturbance, discontinuous transmission of basic information or untimely synchronous recovery may occur, affecting the continuous availability of backhaul services.

[0005] Therefore, the main technical problem this application aims to solve is: in the process of video backhaul for underground image communication in coal mines, when the backhaul link experiences or is about to experience a switch of access device or backhaul link, causing short-term interruption, sudden packet loss, and out-of-order delivery, how to ensure continuous and decodable transmission of grassroots information and support rapid synchronous recovery under the condition of limited redundancy overhead. Summary of the Invention

[0006] To overcome the aforementioned technical deficiencies, the present invention aims to provide a video backhaul adaptive bitrate control method and system for underground coal mine image communication. The present invention determines the switching window and predicts the interruption duration based on switching warning and receiver feedback, and selects a joint configuration (bitrate / keyframe / interleaving and hierarchical redundancy / cross-link replication / control telegram retransmission) from a candidate configuration set under overhead budget constraints. Combined with synchronization digest verification and reconstruction closed loop, the present invention achieves continuous decodeable transmission and rapid synchronization recovery of grassroots information under switching disturbances.

[0007] This invention discloses a video backhaul adaptive bitrate control method for underground coal mine image communication, comprising: S1. Acquire the original video frames captured by the underground camera device in the coal mine, and perform layered encoding to generate a layered return video stream. The layered return video stream includes at least a base layer stream and an enhancement layer stream. S2 encapsulates the layered backhaul video stream into digital information packets and sends them through the main backhaul link. At the same time, the sending end receives the handover announcement information sent by the downhole gateway or access device. S3, at the receiving end, a feedback message is generated and sent back based on the sequence number and arrival time of the digital information packet. The feedback message includes at least the acknowledgment of arrival rate and packet loss rate. The feedback message is then encapsulated into a control telegram in telegraphic short message format and sent. S4, the sending end determines the handover window based on the handover warning information and feedback messages. and predicted interruption duration and from the preset candidate configuration set Select the joint configuration vector used for switching windows:

[0008] in, For the target bitrate, For keyframe interval, For the depth of interleaving of the grassroots code stream, Redundancy ratio of the base layer code stream To enhance the redundancy ratio of the layer bitstream, The cross-link replication ratio of the grassroots code stream, To control the number of telegram repetitions and satisfy the following conditions: ; S5 generates a synchronization digest within the switching window and sends it in groups along with the digital information of the base layer bitstream, according to the cross-link replication ratio. The minimum set of digital information packets for repairing the base code stream is copied and sent on the backup backhaul link, and the control telegram is repeated according to the number of times the control telegram is repeated. Send repeatedly; S6, perform a check on the synchronization digest within the switching window. If the check fails, trigger the reconstruction closed loop: the receiver sends back a reconstruction request control telegram, the sender performs at least one copy transmission of the reconstruction key frame group on the main backhaul link and the backup backhaul link, and suspends the transmission of the enhancement layer code stream until the next key frame arrives. S7, during the switching window and the rebuild loop closure, enables the joint configuration vector Meet the expense budget constraint:

[0009] in, To convert the number of control telegram repetitions into a bandwidth overhead conversion factor, Set a preset cost limit threshold; S8, after the switching window ends, exits cross-link copy transmission and control telegram retransmission, and continues according to the updated target code rate. Interval with keyframe Encode, encapsulate, and send subsequent raw video frames; in, For control cycle number , This is the starting control cycle for switching windows. This is the termination control cycle for switching windows.

[0010] Preferably, the handover notification information includes at least the handover activation control period. Announcement of Interruption Duration With forecast confidence level And switching windows satisfies:

[0011] in, and It is an integer not less than 1, and is based on the forecast confidence level. Select from the preset set.

[0012] The feedback message should include at least a measure of the length of consecutive packet loss. And continuous packet loss length measurement In length The sequence number is calculated within the observation window as follows: Within the observation window, count the set of consecutive missing segment lengths formed by missing sequence numbers. And order:

[0013] in, For the preset window length, This represents the number of consecutive missing segments.

[0014] Preferably, the interruption duration is predicted. The confidence levels obtained by fusing forecast and measurement confidence levels satisfy the following:

[0015] in, The time interval is generated for grouping.

[0016] Preferably, the candidate configuration set It is a discrete set, and the sender is at the th... Within the switching window of each control cycle, the candidate configuration set is selected according to the principle of minimizing the cost function. Select candidate configuration vector And use the candidate configuration vector as the joint configuration vector. ,satisfy:

[0017] Wherein the cost function is:

[0018] in, Preset non-negative coefficients; , , , Candidate configuration vectors The corresponding component.

[0019] Preferably, the candidate configuration set Generate according to the following linked consistency constraints: In each candidate configuration vector, the keyframe interval Replication ratio satisfy:

[0020] in, The minimum replication ratio threshold. These are preset coefficients; and they are maintained in each candidate configuration vector:

[0021] in .

[0022] Preferably, when the cost budget constraint is not met, the joint configuration vector is degraded according to a preset degrading order. The restructuring within the budget, in the order of downgrades, includes: reducing... ,reduce ,reduce Increase Until the expenditure budget constraint is met, and maintain this in any downgrade step. and .

[0023] Preferably, after the reconstruction loop closure is triggered, the set of candidate configurations that minimize the cost function during the pause of the enhancement layer bitstream is determined. Restricted to include only those that satisfy and The candidate configuration vector, where To ensure a safe replication ratio threshold, The threshold for the safety keyframe interval.

[0024] Preferably, the synchronization digest consists of an identifier field and a verification field, wherein the verification field is a cyclic redundancy check. And the identification field includes at least the reference frame group number. With keyframe timestamps The reference frame group is a set of reference frames that start with a key frame and are located between adjacent key frames, and the synchronization digest is inserted into the parameter set group and the first key frame group of the base bitstream.

[0025] Preferably, the synchronized summary is sorted by the number of summary redundancy counts within the switching window. Repeated carryover, and number of times the abstract is redundant. Number of repetitions with control telegram satisfy:

[0026] in, This is the upper limit threshold for the number of times the summary is redundant.

[0027] Preferably, the control telegram in the telegraphic short message format is a fixed-length control frame. The fixed-length control frame includes at least a differential field and a check field. The differential field includes at least an acknowledgment arrival rate differential component, a packet loss rate differential component, and a reference frame group number. The difference component is the difference between corresponding quantities in adjacent control cycles, and the verification field is a cyclic redundancy check. .

[0028] Preferably, the number of telegram repetitions is controlled. Based on the predicted interruption duration The mapping is determined and satisfies:

[0029] in, The maximum number of repetitions threshold. The step duration threshold, This is the floor function.

[0030] Preferably, the repair minimum set of digital information packets is determined by the decoding dependency. The repair minimum set includes at least: key frame parameter set packets, key frame first packets, and the first packet of the first reference frame after the key frame. Furthermore, the repair minimum set is only copied and sent across links within the switching window.

[0031] Preferably, cross-link replication transmission employs time-staggered transmission, meaning that for the same repair minimum set of packets, after transmission on the primary backhaul link, there is a delay... Send its replicated packets on the backup backhaul link, and satisfy:

[0032] in, This is the maximum staggered duration threshold.

[0033] In view of this, the present invention also provides a video backhaul adaptive bitrate control system for underground coal mine image communication, comprising: The layered coding module is used to acquire the original video frames and generate a layered return video stream that contains at least the base layer bitstream and the enhancement layer bitstream; The handover warning receiving module is used to receive handover warning information; The feedback processing module is used to receive feedback messages and obtain the continuous packet loss length metric. The window determination and joint control module is used to determine the switching window and predict the interruption duration, and select from a preset candidate configuration set. Select joint configuration vector ; The synchronization digest module is used to generate a synchronization digest and send it along with the underlying bitstream. The replication scheduling module is used to perform cross-link replication and transmission of the repair minimum set digital information packets; The control telegraph module is used to send control telegraphs in telegraph-style short message format and to repeat the control telegraphs according to the number of repetitions. The reconstruction closed-loop module is used to trigger a reconstruction request and perform keyframe copying and pause the enhancement layer bitstream when the synchronous digest verification fails; The system is configured to execute the method described above.

[0034] Compared with existing technologies, the above technical solution has the following advantages: 1. Improved continuous availability under handover disturbances: Under conditions of short-term interruption, sudden packet loss and out-of-order arrival caused by the handover of access devices or backhaul links, the handover window is determined based on handover warning information and receiver feedback, and the interruption duration is estimated and predicted. This enables the backhaul process to implement targeted joint protection of the base layer information within the handover window, thereby improving the continuous decodeability of the base layer bitstream and reducing the frequency of screen freezes.

[0035] 2. Improved Synchronization Recovery Efficiency: By generating and carrying a synchronization digest within the switching window, performing synchronization digest verification, and triggering a reconstruction closed loop (reconstruction request, keyframe copying, pausing the enhancement layer until the next keyframe arrives) when verification fails, combined with the safety candidate set restriction in the recovery phase, the recovery process after synchronization failure is made more controllable, shortening the synchronization recovery time and improving recovery stability.

[0036] 3. Controllable and adaptive overhead under resource-constrained conditions: The redundancy ratio of the base code stream, the cross-link replication ratio of the base code stream, and the number of control telegram repetitions are incorporated into a unified overhead budget constraint. When the budget is not met, the budget is reorganized within the budget according to the preset degradation order, so that the combined overhead of redundancy and replication can be limited to the preset upper limit, while maintaining the principle of prioritizing the protection of the base layer, thereby adapting to the actual conditions of limited bandwidth in the well.

[0037] 4. Feasibility and consistency assurance of joint configuration selection: By pre-setting a set of candidate configurations and minimizing the cost function to select the joint configuration vector, combined with the linkage consistency constraints of key frame interval and replication ratio and the hierarchical redundancy difference threshold constraints, the multi-parameter collaborative control has a discrete selectable and fast decision-making implementation path, and avoids the instability of synchronous recovery caused by unreasonable parameter combinations.

[0038] 5. Enhanced reliability of control information: Feedback messages are encapsulated into control telegrams in telegraphic short message format and the number of repetitions is determined by mapping the predicted interruption duration. This enhances the probability of control information arrival within the switching window, enabling the sending end to obtain link status and recovery requests more promptly, thereby improving the timeliness and stability of the overall closed-loop control. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the execution environment of the video backhaul adaptive bit rate control method and system for underground coal mine image communication according to the present invention. Figure 2 This is a flowchart illustrating the adaptive bitrate control method and system for video backhaul in underground coal mine image communication according to the present invention. Figure 3 This is a schematic diagram of the continuous decodeable proportional change curve of the base layer before and after the switching disturbance; Figure 4 A schematic diagram showing the change curve of the combined overhead ratio of redundancy and duplication inside and outside the switching window and the upper limit threshold of the overhead; Figure 5 This is a schematic diagram illustrating the change in synchronization recovery time before and after the switching disturbance; Figure 6 This diagram illustrates the comparison of the number of times the screen freezes under different handover interruption scenarios. Detailed Implementation

[0040] The advantages of the present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments.

[0041] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0042] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0043] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0044] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0045] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0046] In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of the description of the invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0047] See Figure 2 As shown, this embodiment provides a video backhaul adaptive bitrate control method for underground coal mine image communication, including: S1, acquiring raw video frames captured by underground coal mine cameras and performing layered encoding to generate a layered backhaul video stream, the layered backhaul video stream including at least a base layer bitstream and an enhancement layer bitstream; S2, encapsulating the layered backhaul video stream into digital information packets and sending them via the main backhaul link, while simultaneously receiving handover warning information from an underground gateway or access device at the sending end; S3, generating and sending back a feedback message at the receiving end based on the sequence number and arrival time of the digital information packets, the feedback message including at least the acknowledgment of arrival rate and packet loss rate, and encapsulating the feedback message into a control telegram in telegraphic short message format and sending it; S4, determining the handover window at the sending end based on the handover warning information and the feedback message. and predicted interruption duration and from the preset candidate configuration set Select the joint configuration vector used for switching windows: ;in, For the target bitrate, For keyframe interval, For the depth of interleaving of the grassroots code stream, Redundancy ratio of the base layer code stream To enhance the redundancy ratio of the layer bitstream, The cross-link replication ratio of the grassroots code stream, To control the number of telegram repetitions and satisfy the following conditions: S5, generate a synchronization digest within the switching window and send it in groups along with the digital information of the base layer bitstream, according to the cross-link replication ratio. The minimum set of digital information packets for repairing the base code stream is copied and sent on the backup backhaul link, and the control telegram is repeated according to the number of times the control telegram is repeated. Repeated transmission; S6, within the switching window, perform a check on the synchronization digest. If the check fails, trigger the reconstruction loop closure: the receiver sends back a reconstruction request control telegram, the sender performs at least one duplicate transmission of the reconstruction keyframe group on the primary and backup backhaul links, and suspends the transmission of the enhancement layer bitstream until the next keyframe arrives; S7, within the switching window and during the reconstruction loop closure, enable the joint configuration vector Meet the expense budget constraint: ,in, To convert the number of control telegram repetitions into a bandwidth overhead conversion factor, Set a preset overhead limit threshold; S8, after the switching window ends, exit cross-link replication transmission and control telegram retransmission, and continue according to the updated target code rate. Interval with keyframe The subsequent raw video frames are encoded, encapsulated, and transmitted; among them, For control cycle number , This is the starting control cycle for switching windows. This is the termination control cycle for switching windows.

[0048] The following description, in conjunction with the accompanying drawings, further illustrates a specific embodiment of the present invention. This embodiment takes video backhaul service in underground coal mine image communication as the application object, and implements a video backhaul adaptive bitrate control method in a communication environment consisting of an underground camera device, a transmitting end, a receiving end, an underground gateway or access device, a main backhaul link, and a backup backhaul link. Figure 1 This diagram illustrates the execution environment of the method in this embodiment. Figure 2 The method flow diagram is shown. Figures 3 to 6 The curves and statistical graphs showing key performance indicators and comparative experimental results are presented. This embodiment is used to illustrate the implementation principle and beneficial effects of the present invention and does not constitute a limitation on the present invention.

[0049] To avoid ambiguity regarding technical names, this embodiment provides a complete explanation of key technical terms: Original video frame refers to the image sequence frame data output by the downhole camera device; Layered backhaul video stream refers to the encoded bitstream organized according to a layered structure, including at least a base layer bitstream and an enhancement layer bitstream, where the base layer bitstream provides basic decodeable video information, and the enhancement layer bitstream provides higher quality video information based on the base layer bitstream; Digital information packet refers to a transmission unit with a sequence number field and a timestamp field, which the receiving end can reassemble and statistically analyze based on; Switching warning information refers to a set of warning data issued by the downhole gateway or access device, used to characterize the time boundary and interruption characteristics of an upcoming or ongoing switching of the access device or backhaul link; Feedback message refers to a set of feedback fields generated by the receiving end based on the receiving status, including at least the acknowledgment arrival rate and packet loss rate; Control telegram refers to a fixed-length control frame in telegraphic short message format carrying the feedback message; Synchronization digest refers to the synchronization identifier and verification information carried with the key packets of the base layer bitstream, used to quickly determine the decoding synchronization status during switching disturbances; Repair minimum set digital information packet refers to the data generated by the decoding dependency... The relationship is determined, and it is the most critical set of basic groups for synchronization recovery; the handover window refers to the control period interval set around the handover effective time, within which joint configuration selection, redundancy protection, replication transmission, and reconstruction closed loop are performed; the predicted interruption duration is the estimate of the effective interruption duration caused by the handover disturbance; the preset candidate configuration set refers to the set of discrete candidate configuration vectors generated in advance, used to quickly select the joint configuration vector within the handover window; the joint configuration vector refers to the set of configuration quantities such as target bitrate, keyframe interval, basic bitstream interleaving depth, layered redundancy ratio, basic replication ratio, and control telegram repetition number determined simultaneously within the same control period; the overhead budget constraint refers to the comprehensive overhead upper limit constraint on the basic redundancy ratio, replication ratio, and control telegram repetition number; the budgeted reorganization refers to the process of adjusting the joint configuration vector according to the preset degradation order until the constraint is met when the overhead budget constraint is not met; the reconstruction closed loop refers to the recovery process of triggering a reconstruction request, keyframe replication, and pausing the enhancement layer until the next keyframe arrives when the synchronization digest verification fails; the safety candidate set refers to the subset of candidate configuration sets limited during the reconstruction closed loop, used to ensure the stability of configuration selection during the recovery phase.

[0050] Step S1: The method of acquiring the original video frames, the acquisition equipment and acquisition principle, and the implementation method of layered encoding: like Figure 2 As shown, the control cycle number is the control cycle number. Within the control cycle, the acquisition and layered encoding of the original video frames are performed first. The original video frames are acquired by the downhole camera device. In this embodiment, the downhole camera device preferably adopts an intrinsically safe or explosion-proof industrial camera that meets the requirements for explosion protection in underground mines, including a lens assembly, an image sensor assembly, an exposure control assembly, an illumination assembly, and a local time synchronization assembly. The lens assembly projects optical images of the tunnel environment onto the imaging surface of the image sensor. The image sensor assembly is preferably a CMOS image sensor with a global shutter or rolling shutter, which converts incident light into charge signals through photoelectric conversion, and then obtains digital pixel values ​​through column amplification and analog-to-digital conversion. The exposure control assembly controls the exposure time, analog gain, and digital gain to adapt to the low illumination and high dust scattering environment underground. The illumination assembly is preferably infrared or white light supplementary lighting, and can use PWM dimming to reduce power fluctuations. The local time synchronization assembly generates timestamps for the video frame acquisition time, which can be synchronized on an industrial Ethernet network via a time base sent by the underground gateway or via PTP (Precision Time Protocol) / NTP (Network Time Protocol), thereby ensuring the consistency of time-related statistics between the sending and receiving ends.

[0051] In this embodiment, the raw video frames can be organized in YUV 4:2:0 format, with each frame containing luminance and chrominance components, along with a frame number and acquisition timestamp field. The downhole camera device can send the raw video frames to the transmitting end via MIPI CSI-2 (Mobile Industry Processor Interface Camera Serial Interface 2), LVDS (Low-Voltage Differential Signaling), or Gigabit Ethernet. The transmitting end can be an downhole edge computing box, an industrial control computer, or an embedded main controller with hardware encoding capabilities, which includes a CPU, memory, and a video encoding unit. After receiving the raw video frames, the transmitting end performs layered encoding to generate a layered return video stream.

[0052] In this embodiment, layered coding is preferably implemented using a scalable video coding method, specifically H.264 SVC, H.265 SHVC, or an equivalent layered coding system. For ease of explanation, H.264 SVC is used as an example. Layered coding can simultaneously include one or more of spatial scalability, temporal scalability, and quality scalability. This embodiment emphasizes ensuring "basic decodability" during handover disturbances; therefore, the base layer bitstream preferably uses a layer with lower resolution, lower frame rate, or lower quality but with independent decoding capabilities. The enhancement layer bitstream preferably carries relatively incremental information to improve resolution, frame rate, or image quality. One implementation of this embodiment is: the base layer bitstream uses a spatially scalable base layer, for example, downsampling the original video frame to 640×360 and encoding it at 25fps or 15fps; the enhancement layer bitstream uses a higher resolution or higher quality enhancement layer, for example, residual information or upsampling enhancement information corresponding to 1280×720. Another implementation is: the base layer bitstream uses a temporally scalable base layer, encoding only keyframes and low frame rate prediction frames; the enhancement layer bitstream supplements intermediate frames to improve the frame rate. Alternatively, a quality-scalable approach can be adopted, where the base layer bitstream is used as a coarse-quantized bitstream and the enhancement layer bitstream is used as a fine-grained residual.

[0053] During the encoding process, the transmitting end performs prediction structure construction, transformation and quantization, entropy coding, and outputs network abstraction layer units for each frame. Prediction structure construction includes the selection of keyframes and non-keyframes, the establishment of inter-frame reference relationships, and motion estimation; transformation and quantization include integer transformation of residual blocks and quantization according to quantization parameters; entropy coding includes CAVLC (Context-Adaptive Variable Length Coding) or CABAC (Context-Adaptive Binary Arithmetic Coding); network abstraction layer units include parameter set units such as sequence parameter sets and image parameter sets, as well as actual video data units. To facilitate the subsequent identification and replication of the "repaired minimum set of digital information groups," this embodiment retains the boundary information between parameter set units and the first unit of the keyframe during encapsulation, so that these key units can be divided into independent digital information groups or located in identifiable positions within the digital information groups.

[0054] The generation of layered return video streams can be performed layer by layer: the base stream outputs the set of network abstraction layer units (NAT units) of the base layer, and the enhancement layer stream outputs the set of NAT units of the enhancement layer. To ensure the independent decodeability of the base stream, it contains necessary parameter set information and keyframe information, and periodically inserts parameter set units when necessary to resist parameter set loss caused by packet loss. Meanwhile, the enhancement layer stream can be sent when network conditions permit; however, during handover disturbances or when budgetary restructuring requires reducing overhead, the transmission rate of the enhancement layer can be reduced or temporarily suspended.

[0055] Step S2: Determining the encapsulation method, sequence number and timestamp fields, and group generation time interval for digital information groups: After the layered backhaul video stream is generated, the sending end performs encapsulation, encapsulating the layered backhaul video stream into digital information packets and sending them via the main backhaul link. In this embodiment, the digital information packets can adopt an RTP-style packet format or an equivalent format, including a fixed header and payload fields. The fixed header includes a sequence number field, a timestamp field, a payload type field, a layer identifier field, and necessary extended fields. The sequence number field uniquely identifies the packet order; the receiving end can use the sequence number field to determine loss or out-of-order packets. The timestamp field identifies the sampling time or frame time corresponding to the packet; the receiving end can use the timestamp field for jitter buffering and statistics. The payload type field identifies whether the packet carries a parameter set unit, a keyframe data unit, or a normal prediction frame data unit. The layer identifier field identifies whether the packet belongs to a base layer stream or an enhancement layer stream. Extended fields can be used to carry information such as synchronization digests or digest redundancy markers.

[0056] The group generation time interval refers to the time interval between the generation of adjacent digital information groups, which is denoted as the group generation time interval in this embodiment. Group generation time interval The time interval can be determined based on the frame rate and the fragmentation strategy. For example, if a fixed-duration fragmentation strategy is used, generating one or more packets every 20ms, then the packet generation time interval... It can be approximated as 0.02s; if grouping is adopted according to the boundary of the network abstraction layer unit, then the group generation time interval is... This can be obtained statistically based on the encoding output rhythm and grouping strategy. Group generation time interval. It is used to convert the continuous packet loss length metric into a time-scale estimate and is an important parameter for predicting the duration of interruptions.

[0057] The primary backhaul link can be an underground wireless backhaul link, an underground industrial Ethernet link, or a hybrid link. The backup backhaul link can be a separate link from the primary backhaul link, or a logical link with independent queues and different priorities on the same physical medium. During handover disruptions, the backup backhaul link is primarily used to carry out the replication and retransmission of critical data packets at the grassroots level and the repeated transmission of control telegrams, thereby improving the accessibility of critical control and data.

[0058] Step S3: Generation of feedback messages, statistics of acknowledgment arrival rate and packet loss rate, encapsulation and reliability design of control telegrams: After receiving digital information packets, the receiving end performs reassembly and statistics, and generates a feedback message. The feedback message includes at least the acknowledgment arrival rate and the packet loss rate. The acknowledgment arrival rate can be calculated using a statistical window, which can be a fixed-duration window or a fixed-packet-count window. Taking a fixed-duration window as an example, the window duration... Within this timeframe, the receiver counts the number of arriving payload bits and divides it by the window duration. The confirmed arrival rate is obtained. The packet loss rate can be obtained by counting the number of missing packets by the sequence number field and dividing by the number of packets that should have arrived.

[0059] To more accurately characterize the characteristics of sudden packet loss, this embodiment calculates a continuous packet loss length metric. This continuous packet loss length metric is denoted as the continuous packet loss length metric. The calculation method is as follows: within a window of the preset length... Within the sequence number observation window, the lengths of consecutive missing segments formed by statistically analyzing missing sequence numbers are set. And define a continuous packet loss length metric that satisfies:

[0060] The number of consecutive missing segments is The continuous packet loss length metric is used to measure the length of the most severe burst of packet loss, thus providing a basis for "stability estimation when handover forecast is uncertain".

[0061] After the feedback message is generated, the receiving end encapsulates it into a control telegram in telegraphic short message format and sends it. In this embodiment, the control telegram uses a fixed-length control frame format and includes at least a differential field and a check field. The differential field carries the difference in acknowledgment arrival rate, packet loss rate, and reference frame group number, where the differential components are the differences between corresponding values ​​in adjacent control cycles; the check field uses cyclic redundancy check. These are used to detect errors in control telegram transmission. The advantage of fixed-length control frames is that they facilitate stable scheduling under low bandwidth or high packet loss conditions downhole, and the probability of arrival can be improved through repeated transmission. As a carrier of control information, control telegrams can have a higher transmission priority than ordinary data packets, and can be repeatedly transmitted on both the primary and backup backhaul links to improve reliability.

[0062] Step S4: Determining the switching window, calculating the confidence fusion of predicted interruption duration, and providing numerical examples: After receiving the handover warning information and control telegram, the sending end determines the handover window and the predicted interruption duration. The handover warning information includes at least the handover activation control cycle. Announcement of Interruption Duration With forecast confidence level Switch windows to switch the effective control cycle. Set start and end boundaries for the center and switch the window start control cycle. With switching window termination control cycle satisfy:

[0063] Preset window span parameters With preset window span parameters It is an integer not less than 1, and is based on the forecast confidence level. Select from the preset set. When the forecast confidence is high, a smaller window span can be selected to reduce unnecessary redundancy overhead; when the forecast confidence is low, a larger window span can be selected to avoid the protection window deviating due to switching boundary errors.

[0064] The predicted interrupt duration is denoted as the predicted interrupt duration. In this embodiment, the predicted interruption duration is calculated by fusing the confidence levels of the warning and the measurement. The predicted interruption duration satisfies the following:

[0065] The group generation time interval This is the generation time interval for adjacent digital information packets. This calculation reflects the greater trust in the forecast interruption duration when the forecast confidence is high, and the greater trust in a metric derived from the consecutive packet loss length when the forecast confidence is low.

[0066] To clarify the calculation process, a numerical example is provided: assuming a switching effective control period. Forecast confidence level Select , Then switch windows to Assuming the predicted interruption duration... s, the set of missing segment lengths within the sequence number observation window is Continuous packet loss length measurement Group generation time interval If s, then the predicted interruption duration is:

[0067] It should be noted that step S4 will be explained again in this embodiment, including a detailed explanation of the construction of the preset candidate configuration set, the definition of the candidate configuration vector, and the linkage consistency constraint: Within the switching window, the sending end needs to determine multiple configuration values ​​simultaneously. To avoid delays caused by complex optimizations in real time, this embodiment uses a preset candidate configuration set. Preset candidate configuration set As a discrete set, it can be pre-generated and fixed during the deployment phase based on equipment capabilities, link bandwidth range, and downhole service requirements, or it can be dynamically updated according to configuration files. Pre-set candidate configuration set. Each candidate configuration vector in the dataset is defined as a candidate configuration vector according to a uniform structure. Among them, the candidate target bit rate Candidate bitrate value, candidate keyframe interval Candidate keyframe interval value, candidate base layer bitstream interleaving depth Candidate interleaving depth value, candidate base layer bitstream redundancy ratio The candidate FEC (Forward Error Correction) redundancy ratio value represents the candidate enhancement layer bitstream redundancy ratio. The candidate enhancement layer redundancy ratio value represents the cross-link replication ratio of the candidate base layer bitstream. Candidate replication ratio value, candidate control telegram repetition count. This represents the candidate repetition count. Once a candidate configuration vector is selected, it is used as the control cycle number. The corresponding joint configuration vector, written as:

[0068] To further illustrate this, a set of feasible candidate values ​​are provided: candidate target bitrate From the set Selection; Candidate keyframe interval From the set Selection (unit: frames); candidate base layer bitstream interleaving depth From the set Selection; Redundancy ratio of candidate base-level code streams From the set Selection; Redundancy ratio of candidate enhancement layer bitstream From the set Selection; Cross-link replication ratio of candidate grassroots code streams From the set Selection; Number of repetitions of candidate control telegrams From the set Select. The above set can be adjusted according to device and link capabilities.

[0069] To ensure structural consistency among candidate configuration vectors, a pre-defined candidate configuration set is used. Linkage consistency constraints are applied during generation. One of these constraints is the candidate keyframe interval. Cross-link replication ratio of candidate grassroots code streams The coupling constraints between them satisfy:

[0070] The minimum replication ratio threshold As a preset lower limit, the coupling coefficient The second constraint on consistency is the redundancy ratio of candidate base-level code streams, which is a preset coefficient. Redundancy ratio of candidate enhancement layer bitstream The hierarchical redundancy difference threshold constraint between them satisfies:

[0071] Among them, the hierarchical redundancy difference threshold .

[0072] It should be noted that step S4 will be explained in detail again in this embodiment, with a detailed explanation and numerical examples of the cost function minimization selection: Within the switching window, the sending end selects from the preset candidate configuration set according to the principle of minimizing the cost function. Candidate configuration vectors are selected and used as the joint configuration vector. The joint configuration vector satisfies:

[0073] Cost function satisfy:

[0074] Here's a clearer example of candidate comparison: Suppose we're predicting the duration of the interruption. Conversion factor Weighting , , , , Let the candidate configuration vector A be... The candidate configuration vector B is The candidate configuration vector C is First, check the linkage consistency constraints, and then... , , For A, the lower limit of the replication ratio is:

[0075] A's Satisfy; and Satisfied. For B, the lower limit of the replication ratio is:

[0076] B's Satisfy; and Satisfied. For C, the lower bound of the replication ratio is:

[0077] C satisfy; Satisfaction is achieved. The cost function is then calculated:

[0078]

[0079]

[0080] Therefore, candidate configuration vector C is selected as the joint configuration vector for this control cycle. .

[0081] Step S5: Generation of synchronization digests, definition of reference frame groups, and an example of coupling digest redundancy counts with control telegram repetition counts: Within the switching window, the sending end generates a synchronization digest and sends it along with the base layer bitstream digital information in packets. The synchronization digest consists of an identifier field and a check field, with the check field employing cyclic redundancy check. The identification field must include at least the reference frame group number. With keyframe timestamps In this embodiment, a reference frame group is defined as: a set of reference frames starting with a keyframe and located between adjacent keyframes. A synchronization digest is inserted into the parameter set group and the first keyframe group of the base bitstream.

[0082] Within the switching window, synchronize the summary by the number of summary redundancy counts. Repeated carrying, the number of times the summary is redundant and the number of times the control telegram is repeated satisfy:

[0083] The upper limit threshold for the number of times the abstract is redundancy is: .

[0084] It should be noted that step S5 will be explained again in this embodiment, including the method for determining the minimum set of digital information groups, the copying and sending strategy, and the significance of the overhead: Within the switching window, the sending end replicates the underlying bitstream across links according to the cross-link replication ratio. The minimum set of repair digital information packets for the base-level bitstream is replicated and transmitted on the backup backhaul link. The minimum set of repair digital information packets is determined by decoding dependencies and includes at least the keyframe parameter set packet, the first packet of the keyframe, and the first packet of the first reference frame after the keyframe. The minimum set of repair digital information packets are replicated across links only within the handover window to concentrate replication overhead on the protection most needed during handover disturbances, while reducing redundancy outside the handover window.

[0085] It should be noted that step S5 is also described in this embodiment, including the control telegram format, the meaning of the differential field, the mapping of the number of repetitions, and examples: The control telegram is a fixed-length control frame, which includes at least a differential field and a check field. The differential field includes at least an acknowledgment arrival rate difference component, a packet loss rate difference component, and a reference frame group number difference component. The difference components are the differences between corresponding values ​​in adjacent control cycles. The check field is a cyclic redundancy check (CRC).

[0086] Control telegrams are counted according to the number of times the control telegram is repeated. Repeated transmission: The number of times the control telegram is repeated is determined by the predicted interrupt duration mapping, satisfying:

[0087] The maximum repetition threshold is: The step duration threshold is .

[0088] Step S7: Explanation and Examples of "Step-by-Step Actions" for Expenditure Budget Constraints and On-Budget Restructuring: This embodiment applies overhead budget constraints within the switching window and during the loop reconstruction period. The overhead budget constraints satisfy:

[0089] Where the upper limit threshold for overhead is The conversion factor is When the overhead budget constraint is not met, in-budget refactoring is performed. In-budget refactoring reduces the cross-link replication ratio of the base layer bitstream, reduces the redundancy ratio of the enhancement layer bitstream, reduces the target bitrate, and increases the keyframe interval in a preset degradation order until the overhead budget constraint is met. During the in-budget refactoring process, the redundancy ratio of the base layer bitstream is kept greater than that of the enhancement layer bitstream and the layer redundancy difference threshold constraint is met.

[0090] Example given: Let , If a certain control cycle adopts , , The total cost is then:

[0091] Triggering on-budget restructuring; After being reduced to 0.31:

[0092] The budget constraints have been met, and the on-budget restructuring is now complete.

[0093] Steps S6 and S8: Synchronous digest verification, loop reconstruction, security candidate set restriction and exit mechanism: When the receiver fails to verify the synchronization digest within the switching window, it triggers a reconstruction loop closure and sends back a reconstruction request control telegram. Upon receiving the reconstruction request control telegram, the transmitter performs at least one copy transmission of the reconstruction keyframe group on both the primary and backup backhaul links, and suspends the transmission of the enhancement layer bitstream until the next keyframe arrives. To improve the stability of the anomaly recovery phase, a preset candidate configuration set is used during the period after the reconstruction loop closure is triggered and the enhancement layer bitstream is paused. Restricted to include only those that satisfy and The candidate configuration vector, where the secure replication ratio threshold is The safety key frame interval threshold is After the switching window ends, the sending end exits cross-link copy transmission and control telegram retransmission, and continues to encode, encapsulate, and transmit subsequent raw video frames according to the updated target bitrate and keyframe interval.

[0094] Experimental and comparative data: To verify the continuous availability and synchronous recovery capability of backhaul under handover disturbance conditions in this embodiment, an underground backhaul simulation / semi-physical platform was constructed. The primary backhaul link is an underground wireless backhaul link, and the backup backhaul link is an underground industrial Ethernet link or another independent wireless backhaul link. The underground gateway simulates the issuance of handover warning information and injects short-term interruptions, sudden packet loss, and out-of-order packets through a link simulator. The comparative scheme is implemented based on the layered SVC multipath bearer and receiver reassembly approach disclosed in the background technology, that is, the base layer bitstream and the enhancement layer bitstream are transmitted through different links and reassembled at the receiver, but without introducing the handover window joint configuration selection driven by handover warning information, the budget reassembly without overhead budget constraints, and the reconstruction closed-loop security candidate set restriction triggered by asynchronous digest verification.

[0095] The test video was 1280×720, 25fps, with a control cycle of 200ms, a preset window length W=50 groups, and a step duration threshold. ms, maximum number of repetitions threshold Overhead limit threshold Three handover disturbance scenarios were set up: Scenario A was a 300ms interruption accompanied by sudden packet loss; Scenario B was a 600ms interruption accompanied by significant out-of-order packet loss; and Scenario C was a 900ms interruption accompanied by a mixture of sudden packet loss and out-of-order packet loss. Statistical indicators included: the proportion of continuously decodable base layers (measured as the percentage of time the base layer could be continuously decodable within a 1-second sliding window), synchronization recovery time (the time from the first verification failure to recovery to a decodable state), the combined overhead of redundancy and replication (the percentage of extra bits sent by the sender out of the total sent bits), and the number of times the screen froze within the handover window (number of times / minute). The experimental results are shown in Table 1. Figure 3 , Figure 4 , Figure 5 , Figure 6 The comparative trends of the proportion of continuously decodable components at the grassroots level, overall overhead, synchronization recovery time, and number of freezes are presented respectively. Figure 3 The continuous decodeable ratio curve at the grassroots level in the middle shows a valley near the handover disturbance. In this embodiment, the valley is shallower and recovers faster. This is because the joint configuration vector is selected by a preset candidate configuration set and linkage consistency constraints are applied within the handover window. At the same time, the digital information of the repair minimum set is copied and sent to enhance the reliability of control telegrams, thereby improving the probability of arrival of key synchronization information and key data at the grassroots level. Figure 4 The overall overhead curve rises near the switching window but does not cross the threshold, reflecting the effects of overhead budget constraints and in-budget realignment: when the overall overhead approaches or exceeds... Prioritize reducing and These items will be quickly brought back into the budget. Figure 5 The synchronization recovery time curve in the previous scheme showed a higher peak near the handover, while the peak value in this embodiment was significantly reduced. This is because the synchronization digest verification failure triggered the reconstruction loop closure and restricted the security candidate set during the pause of the enhancement layer bitstream, thus improving the recovery phase. and It remains within a conservative range, thus accelerating recovery. Figure 6 The comparison of the number of freezes demonstrates the stability effect of this embodiment in reducing the number of freezes by switching window joint control and closed-loop recovery under different interruption length scenarios.

[0096] Table 1. Comparison results between this embodiment and the comparative scheme under switching disturbances.

[0097] Table 1 presents the comparison results between this embodiment and the comparative scheme in four indicators: "continuously decodable proportion of the base layer, synchronization recovery time, proportion of redundancy and replication overhead, and number of screen freezes" under three types of handover disturbance scenarios (Scenario A: 300ms interruption with sudden packet loss; Scenario B: 600ms interruption with significant out-of-order packet loss; Scenario C: 900ms interruption with a mixture of sudden packet loss and out-of-order packet loss). Among them, the continuously decodable proportion of the base layer reflects the continuous availability of the base layer bitstream during the handover disturbance; the synchronization recovery time reflects the recovery speed after synchronization failure; the proportion of redundancy and replication overhead reflects the additional transmission overhead introduced to ensure availability; and the number of screen freezes reflects the degree to which the terminal-side experience is affected by the disturbance.

[0098] As shown in Table 1, with the increase in interruption duration and the accompanying enhancement of out-of-order / burst packet loss, the proportion of continuously decodable base layers in the comparative schemes decreased significantly, the synchronization recovery time increased significantly, and the number of screen freezes increased. This indicates that relying solely on layered multipath bearers and receiver reassembly can easily lead to missing synchronization information and unreachable key packets near the switching window, resulting in a long recovery waiting time. In contrast, this embodiment exhibits a higher proportion of continuously decodable base layers, a shorter synchronization recovery time, and a lower number of screen freezes in all three scenarios. This demonstrates that the combined strategy of this embodiment within the switching window—"determining the switching window and estimating the predicted interruption duration through switching forecast and feedback—selecting a joint configuration vector from a preset candidate configuration set—performing duplicate transmission of the repair minimum set of digital information packets—repeatedly transmitting control telegrams according to the predicted interruption duration mapping—triggering a reconstruction loop and limiting the security candidate set upon synchronization digest verification failure"—can significantly enhance the reachability of key synchronization information and key base layer data and accelerate the recovery process. Meanwhile, the proportion of redundancy and replication overhead in this embodiment is higher than that in the comparative scheme in Table 1, indicating that the improvement in availability comes at the cost of increasing redundancy and replication. However, this overhead is still controllable under the constraints of the overhead upper limit threshold and the control of the budgeted reorganization mechanism, thereby achieving the technical effect of "significantly improved availability and controlled resource consumption" under handover disturbances.

[0099] It should be noted that this embodiment will provide a detailed explanation of the selection and relationship between the control cycle, statistical window, and key timing parameters: In downhole video backhaul services, to ensure the stable operation of the "switching window determination—predicted interruption duration calculation—candidate configuration selection—budgeted refactoring—reconstruction closed loop" process under limited computing resources, this embodiment typically organizes configuration updates and feedback statistics to be executed within a fixed control cycle. The control cycle length can be 100ms to 500ms; this embodiment preferably uses 200ms to ensure sufficient update frequency before and after switching disturbances, while avoiding excessive control message overhead. The control cycle sequence number is denoted as the control cycle sequence number. At the end of each control cycle, the feedback statistics and forecast information of the previous cycle are summarized as the basis for configuration in the current cycle.

[0100] Continuous packet loss length measurement Serial number observation window length It can be selected in conjunction with the control cycle: for example, under an encapsulation strategy of 25fps and an average of 2 digital information packets per frame, approximately 50 digital information packets are generated per second, in which case the following can be selected: ,make The statistical scope is on the same time scale as the "proportion of continuously decodable grassroots units in a 1-second sliding window", which makes it easy to correlate the statistical quantity with the availability index.

[0101] Group generation time interval The determination can be made in two ways: First, by statistically analyzing the average grouping rhythm of the encapsulation and transmission module, for example, by statistically analyzing continuous... The generation timestamps of each digital information group are used, and the average interval is taken as the data. Secondly, a conservative estimate with a fixed upper limit is adopted, for example, taking a smaller value based on the "most dense grouping rhythm". ,make Do not underestimate the measurement items, thus providing a more robust level of protection even when the forecast confidence level is low.

[0102] Step duration threshold With the maximum number of repetitions threshold Used to control the number of times a telegram is repeated. The mapping range. Generally, a suitable range is... It can be the control cycle length or an integer multiple thereof. For example, when the control cycle is 200ms, it can be taken as... Maximum repetition threshold It can be set by combining "acceptable control overhead limit" and "switching window length limit", for example, take This is to enhance the reliability of control information even when the predicted interruption duration is long, while avoiding excessive control telegram usage.

[0103] Expenditure cap threshold The selection should match the actual available uplink bandwidth underground. For example, when the primary backhaul link has a normal available uplink bandwidth of approximately 1.5 Mbps, and the service side allows 30%–40% of the bandwidth to be used for redundancy and replication within the switching window, the appropriate bandwidth can be selected. It is around 0.65; when the downhole bandwidth is more limited (e.g., 800kbps), it can be taken as... It is around 0.55, and relies on budgeted restructuring to prioritize reducing replication and enhancement layer protection to meet the budget.

[0104] It should be noted that, in this embodiment, the generation and confidence level of the handover warning information will be discussed. The engineering method for determining: In downhole networks, handover warning information can be generated and distributed by the downhole gateway or access device based on link measurements and access control status. Common handover triggering criteria include: received signal strength indication, signal-to-noise ratio estimation, increasing packet loss statistics, uplink retransmission count, increased queuing delay on the access side, and handover control indications on the access device side. Handover activation control cycle. The predicted interruption duration can be obtained by converting the "estimated handover effective time on the access side" into the control cycle boundary. It can be given by the handover process on the access side (e.g., the expected duration of link establishment, authentication, or route switching).

[0105] Forecast confidence level In engineering implementation, this can be determined based on "consistency of multi-source evidence." For example, when the access side clearly enters the handover preparation state, and the link quality indicators show a consistent deterioration trend over multiple consecutive control cycles, it can be set as follows: Take a larger value (e.g., 0.8–0.95); when only short-term fluctuations occur and the access side has not entered the handover preparation state, it can be set to... Choose a smaller value (e.g., 0.3–0.6). This approach provides the sender with a usable dimension of "preview confidence" without introducing additional complex reasoning, thereby supporting the prediction of outage duration. The confidence level is calculated by fusion.

[0106] It should be noted that, in this embodiment, the specific implementation methods of the hierarchical coding parameters and keyframe intervals will be explained (hereinafter referred to as...). A detailed description is provided regarding the establishment of an executable mapping with encoder parameters: In layered coding, keyframe interval Used to constrain the frequency of keyframe occurrences. Keyframe intervals can be adjusted during project implementation. This is mapped to the encoder's keyframe period parameters. For example, when the base stream encoding frame rate is 25fps, the keyframe interval... This means that a keyframe is inserted every 50 frames, corresponding to a refresh approximately every 2 seconds; when At this time, a refresh occurs approximately every 1.2 seconds. Due to the need for faster synchronization recovery during handover disturbances, the safety critical frame interval threshold... You can select no more than 50 (e.g., 40 or 50) to avoid excessively sparse keyframes that would cause long recovery wait times after reconstruction and loop closure.

[0107] Grassroots code stream interleaving depth In engineering, this can be implemented as a depth parameter in a packet interleaver. This involves rearranging consecutive base-level bitstream digital information into groups according to the interleaving depth before retransmission, thus breaking up sudden packet loss into approximately random packet loss and improving the probability of redundancy code repair. Base-level bitstream redundancy ratio. Redundancy ratio of enhancement layer bitstream The redundancy ratio parameter can be implemented as a forward error correction coding parameter. For example, when using block code or fountain code structures, the redundancy ratio can correspond to the ratio of "redundant group number / source group number"; while maintaining the redundancy ratio of the base code stream. Greater than the redundancy ratio of the enhancement layer bitstream And keep the difference no less than the hierarchical redundancy difference threshold. This is to ensure that the basic level can be decoded when budget is limited.

[0108] It should be noted that, in this embodiment, the "fixed-length example" of the control telegram field and the specific calculation example of the differential field will be described in detail: The control telegram uses a fixed-length control frame format. In one feasible example, the control telegram can be fixed at 32 bytes or 48 bytes (depending on the link MTU and reliability requirements). The differential field and check field can be implemented using a "field + quantization" approach. For example, the acknowledgment rate difference component can quantize the "change in acknowledgment rate" into an integer codeword, the packet loss rate difference component can quantize the "change in packet loss rate" into an integer codeword, the reference frame group number difference component can be represented by a small integer (usually 0 or 1), and the check field uses cyclic redundancy check. The calculated check codeword.

[0109] The specific calculation example for the differential field "difference between adjacent control cycles" is as follows: If the first The confirmation arrival rate per control cycle is the confirmation arrival rate. , No. The confirmation arrival rate per control cycle is the confirmation arrival rate. Then the arrival rate difference component is confirmed to be If the first The packet loss rate per control cycle is the packet loss rate. , No. The packet loss rate per control cycle is the packet loss rate. The difference in packet loss rate is then... If the reference frame group number is determined by the reference frame group number... Change to reference frame group number If the reference frame group number difference component is 1, then the above difference component can be further quantized into a fixed-length control telegram field in a fixed-point format.

[0110] It should be noted that in this embodiment, multiple sets of calculation examples (different) will be performed. ,different ,different exhibit , Changes with switching windows: To further clarify the predicted outage duration The calculation results are presented in Table 2, showing the relationship between forecast confidence, consecutive packet loss length metric, and packet generation time interval. Multiple examples are used in the examples. The forecast interruption duration is consistently used in all examples. Step duration threshold Maximum repetition threshold .

[0111] Table 2 shows multiple sets of calculation examples for predicted interruption duration and control telegram repetition count.

[0112] Further examples of switching windows are provided: assuming a switching activation control period. .when Time to take , ;when Time to take , ;when Time to take , Then, at high confidence levels, the window switching is... Switch the window to medium confidence level. Switch windows when confidence is low. .

[0113] Table 2 shows the duration of the unified warning interruption. Unified step duration threshold With the maximum number of repetitions threshold Under different forecast confidence levels Different consecutive packet loss length metrics Different group generation time intervals Predicted interruption duration corresponding to the combination Number of repetitions with control telegram The calculation results are used to illustrate the linkage between the predicted interruption duration and the number of control telegram repetitions, as well as its adaptability to the severity of handover disturbances.

[0114] As can be seen from Table 2, when the forecast confidence level At higher values ​​(e.g., 0.90), the predicted interruption duration is... Mainly determined by the duration of the predicted interruption Dominant, even as the length of consecutive packet loss increases, The range of change is also relatively limited, reflecting a higher level of trust in the forecast information; when the forecast confidence level is low (e.g., 0.30). The proportion of measurements consisting of continuous packet loss length and packet generation time interval is higher, making it more effective in situations where sudden packet loss is more severe. This can be increased accordingly, resulting in a higher number of control telegram repetitions. This improves the accessibility of control information during handover disturbances. On the other hand, when the packet generation time interval... When the packet length is smaller (the packets are denser), even if the consecutive packet loss length metric is the same, the measurement item... It will also become smaller, thus reducing and This demonstrates that the calculation mechanism can reflect the changes in the time scale of disturbances caused by differences in encapsulation rhythm, avoiding overestimation of interruption duration in densely packed scenarios. Table 2 thus verifies that the calculation logic of "fusion of warning and measurement confidence" and "control of telegram repetition frequency based on predicted interruption duration" in this embodiment has clear adaptability and interpretability.

[0115] It should be noted that in this embodiment, different Example of in-budget reorganization triggering and action path calculation: The triggering and action path of on-budget refactoring are crucial in engineering implementation. To illustrate intuitively how on-budget refactoring can produce different final configurations based on the same candidate configuration when different cost cap thresholds are applied, the following example is provided. Let the discount factor be... The initial configuration for a certain control cycle is as follows: , , , , , And satisfy .

[0116] The total cost at this point is:

[0117] when When the budget is met; When triggered by budgetary restructuring, reduce the following order first: For example, ,but: Meet the budget and end; when You can continue to reduce the budget item by item in the same order until the budget is met.

[0118] It should be noted that, in this embodiment, the shrinkage of the "safe candidate set" after the reconstruction loop closure is triggered will be described in detail: When the synchronization digest verification fails and a reconstruction loop is triggered, the preset candidate configuration set will be used. Restricted to include only those that satisfy and The candidate configuration vector. A shrinkage example is given: Let... , The original preset candidate configuration set contains candidate configuration vectors. After filtering, only candidate configuration vectors that meet the conditions are retained to reduce the probability of failure to verify again during the abnormal recovery phase and improve the stability of synchronous recovery.

[0119] Candidate configuration vector Specifically as follows: ; ; ; ; It should be noted that this embodiment will provide multiple sets of extended experimental data, as shown in Tables 3 and 4: According to the conditions of the safe candidate set and filter: : and Remove; : Satisfied, but Remove; : and ,reserve; : and ,reserve.

[0120] Table 3. Comparison results under different forecast confidence levels (Scenario B: 600ms interruption + obvious out-of-order delivery)

[0121] Table 3 presents the forecast confidence levels for different prediction confidence levels under the same handover disturbance scenario (Scenario B: 600ms interruption with significant out-of-order delivery). The comparison results between this embodiment and the comparative scheme on three indicators, namely "proportion of continuously decodable grassroots, synchronous recovery time, and number of times the screen is frozen", are used to illustrate that this embodiment can still maintain a stable improvement in usability when the credibility of the preview information is different.

[0122] As shown in Table 3, as the forecast confidence level decreases from 0.90 to 0.30, the comparative scheme exhibits a decrease in the proportion of continuously decodable base layers, an increase in synchronization recovery time, and an increase in the number of freezes. This indicates that when forecast information is unreliable or cannot be effectively adjusted in advance, relying solely on receiver reassembly is insufficient to address the critical packet unreachability problem caused by the superposition of out-of-order and interruption events. In contrast, this embodiment maintains a significantly better proportion of continuously decodable base layers and synchronization recovery time than the comparative scheme at different forecast confidence levels, and the number of freezes is significantly reduced. This is because this embodiment does not rely solely on forecast information, but rather uses a confidence fusion mechanism based on the predicted interruption duration to increase the proportion of measurement items under low confidence conditions. This allows for the selection of a more robust joint configuration vector within the switching window and enhances control telegram protection and repair of the minimum set replication. Therefore, even with a low forecast confidence level, this embodiment can still achieve good stability through a measurement-driven protection and recovery closed loop. Table 3 thus verifies that this embodiment is adaptable to fluctuations in forecast information reliability and can maintain an effective response to switching disturbances in downhole environments with uncertain forecasts.

[0123] Table 4. Comparison results under different overhead upper limit thresholds (Scenario A: 300ms interruption + sudden packet loss)

[0124] Table 4 presents the overhead upper limit thresholds for different scenarios under the same handover disturbance (Scenario A: 300ms interruption accompanied by sudden packet loss). The comparison results between this embodiment and the comparative scheme on three indicators—"proportion of continuously decodable components at the grassroots level, synchronous recovery time, and proportion of combined overhead of redundancy and replication"—are used to illustrate that under conditions of tighter or more abundant bandwidth resources, this embodiment achieves a balance between "improved availability and controlled overhead" through overhead budget constraints and in-budget restructuring.

[0125] As can be seen from Table 4, when the overhead limit threshold is reached... At a lower threshold (e.g., 0.55), the combined overhead of redundancy and replication in this embodiment is close to and limited by this threshold, yet it can still significantly improve the proportion of continuously decodable layers at the base level and reduce synchronization recovery time. This demonstrates that the budgeted restructuring mechanism can prioritize reducing replication and enhancement layer-related overhead according to a preset degradation order when resources are scarce, while maintaining priority protection for the base level, thus achieving significant availability benefits even under strict budget constraints. When the overhead limit threshold is increased (e.g., 0.65, 0.75), this embodiment allows for more redundancy and replication overhead to be invested within the switching window, resulting in a higher proportion of continuously decodable layers at the base level and a shorter synchronization recovery time, reflecting the performance ceiling improvement when resources are more abundant. The comparison schemes are implemented in different... The lack of significant changes in the indicators indicates the absence of a "joint configuration selection and dynamic reorganization mechanism under budget constraints." This prevents the effective utilization of additional budgets to enhance critical protection within the switching window and hinders the formation of a clear protection priority strategy when the budget tightens. Table 4 thus verifies that the overhead budget constraints and in-budget reorganization in this embodiment not only limit the overall overhead to no more than the threshold but also prioritize the use of limited budgets to improve the decoding and synchronous recovery capabilities of the grassroots level, achieving "adjustable performance under controllable budget."

[0126] The following combination Figure 1 Another embodiment of the present invention will be described. This embodiment provides a video backhaul adaptive bitrate control system for underground coal mine image communication. The system is deployed on the underground backhaul sending side or underground edge computing node, and is used to adaptively configure and synchronize the backhaul process under conditions of short-term interruption, sudden packet loss, and out-of-order delivery caused by access device or backhaul link switching. This embodiment is used to illustrate the system structure and its functional coordination relationship, and does not constitute a limitation on the present invention.

[0127] like Figure 1 As shown, the system includes a hierarchical coding module, a handover warning receiving module, a feedback processing module, a window determination and joint control module, a synchronization digest module, a copy scheduling module, a control telegram module, and a closed-loop reconstruction module.

[0128] The layered encoding module is used to acquire the raw video frames collected by the downhole camera device and to perform layered encoding on the raw video frames to generate a layered return video stream that contains at least a base layer bitstream and an enhancement layer bitstream. The base layer bitstream is used to provide basic decodeable video information, and the enhancement layer bitstream is used to provide higher quality video information based on the base layer bitstream.

[0129] The handover warning receiving module is used to receive handover warning information sent by the downhole gateway or access device. The handover warning information includes at least the handover effective control cycle, the warning interruption duration and the warning confidence level, and provides the handover warning information to the window determination and joint control module.

[0130] The feedback processing module is used to receive control telegrams transmitted back from the receiving end and parse them to obtain feedback messages. The feedback messages include at least the arrival rate and packet loss rate. The feedback processing module is also used to calculate the continuous packet loss length based on the sequence number and arrival time of digital information packets, and provide the continuous packet loss length measurement to the window determination and joint control module.

[0131] The window determination and joint control module is used to determine the handover window and predict the interruption duration based on the handover forecast information and feedback messages, and select a joint configuration vector from the preset candidate configuration set to determine the target bit rate, keyframe interval, base stream interleaving depth, base stream redundancy ratio, enhancement layer stream redundancy ratio, base stream cross-link replication ratio, and control telegram repetition count. The window determination and joint control module is also used to apply overhead budget constraints to the joint configuration vector, and perform in-budget refactoring according to the preset degradation order when the budget is not met.

[0132] The synchronization summary module is used to generate a synchronization summary within the switching window and send it along with the key packets of the base code stream. The synchronization summary includes at least the reference frame group number, the key frame timestamp, and the cyclic redundancy check field, and can be carried repeatedly according to the number of summary redundancy.

[0133] The replication scheduling module is used to perform cross-link replication transmission of the repair minimum set digital information packets of the base code stream within the switching window to improve the reachability of critical packets during handover disturbances.

[0134] The control telegram module is used to encapsulate feedback messages into control telegrams in telegraphic short message format and send them. It also repeats the control telegrams within the switching window according to the number of times the control telegram is repeated, so as to enhance the reliability of control information under sudden packet loss conditions.

[0135] The reconstruction closed-loop module is used to trigger the reconstruction closed-loop operation when the synchronization digest verification fails. This includes the receiver sending back a reconstruction request control telegram, the sender performing at least one copy transmission of the reconstruction keyframe group, and pausing the transmission of the enhancement layer bitstream until the next keyframe arrives. The reconstruction closed-loop module is also used to impose a security candidate set restriction on the candidate configuration set during the pause of the enhancement layer bitstream, so that the copy ratio and keyframe interval in the recovery phase meet the preset security threshold conditions.

[0136] Through the collaboration of the above modules, the system can perform switching window determination, joint configuration selection, overhead constraints and closed-loop recovery control according to the preset control cycle, thereby improving the continuous decoding capability of the base code stream under switching disturbance conditions and shortening the synchronization recovery time, while keeping redundancy and replication overhead within the preset upper limit constraint range.

[0137] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A video backhaul adaptive bitrate control method for underground coal mine image communication, characterized in that, include: S1, acquire the original video frames collected by the underground camera device in the coal mine, and perform layered encoding to generate a layered return video stream, wherein the layered return video stream includes at least a base layer stream and an enhancement layer stream; S2, the layered backhaul video stream is encapsulated into digital information packets and sent via the main backhaul link. Simultaneously, the sending end receives handover announcement information from the downhole gateway or access device. This handover announcement information includes at least the handover activation control period. Announcement of Interruption Duration With forecast confidence level ; S3, at the receiving end, a feedback message is generated and transmitted back based on the sequence number and arrival time of the digital information group. The feedback message includes at least the acknowledgment of arrival rate and packet loss rate. The feedback message is then encapsulated into a control telegram in telegraphic short message format and sent. S4, the sending end determines the handover window based on the handover warning information and the feedback message. and predicted interruption duration And according to the principle of minimizing the cost function, select from the preset candidate configuration set Select the joint configuration vector for the switching window, satisfying... ,in Cost function: in, For the target bitrate, For keyframe interval, For the depth of interleaving of the grassroots code stream, Redundancy ratio of the base layer code stream To enhance the redundancy ratio of the layer bitstream, The cross-link replication ratio of the grassroots code stream, The number of times the control telegram is repeated, and satisfies the following conditions: ; S5, within the switching window, a synchronization summary is generated and sent in groups along with the digital information of the base stream, according to the cross-link replication ratio. The minimum set of digital information packets for repairing the base code stream is copied and sent on the backup backhaul link, and the control telegram is repeated a certain number of times. Send repeatedly; S6, perform verification on the synchronization summary within the switching window, and trigger reconstruction closed loop when verification fails: the receiving end sends back reconstruction request control telegram, the sending end performs at least one copy transmission of reconstruction key frame group on the main backhaul link and the backup backhaul link, and suspends the transmission of enhancement layer code stream until the next key frame arrives. S7, during the switching window and the reconstruction loop closure, the joint configuration vector is... Meet the expense budget constraint: in, To convert the number of control telegram repetitions into a bandwidth overhead conversion factor, Set a preset cost limit threshold; S8, after the switching window ends, exit the cross-link copy transmission and the repeated transmission of the control telegram, and continue according to the updated target code rate. Interval with the keyframe Encode, encapsulate, and send subsequent raw video frames; in, For control cycle number ; This is the starting control period for the switching window. This is the termination control cycle for the switching window.

2. The method according to claim 1, characterized in that, in, and A preset window span parameter that is an integer not less than 1, and based on the aforementioned forecast confidence level. Select from the preset set.

3. The method according to claim 2, characterized in that, The feedback message includes at least a continuous packet loss length metric. And the continuous packet loss length metric In length The sequence number is calculated within the observation window as follows: Within the observation window, the length set of consecutive missing segments formed by counting missing sequence numbers is calculated. And order: in, For the preset window length, This represents the number of consecutive missing segments.

4. The method according to claim 3, characterized in that, The predicted interruption duration The predicted confidence level is obtained by fusing it with a measurement term derived from the continuous packet loss length metric, satisfying the following: in, The time interval for generating adjacent digital information groups.

5. The method according to claim 1, characterized in that, The candidate configuration set It is a discrete set, and the sending end is in the first... Within the switching window of each control cycle, the candidate configuration set is selected according to the principle of minimizing the cost function. Select candidate configuration vector The candidate configuration vector is then used as the joint configuration vector. ,satisfy: Wherein the cost function is: in, The preset non-negative coefficients; , , , The candidate configuration vectors are respectively The corresponding component.

6. The method according to claim 5, characterized in that, The candidate configuration set Generate according to the following linked consistency constraints: In each candidate configuration vector, the keyframe interval Replication ratio satisfy: in, The minimum replication ratio threshold. These are preset coefficients; and they are maintained in each candidate configuration vector: in ,and This is the threshold for the difference in layered redundancy.

7. The method according to claim 6, characterized in that, When the overhead budget constraint is not met, the joint configuration vector is degraded according to a preset degrading order. The downgrade order for the on-budget restructuring includes: reducing... ,reduce ,reduce Increase Until the aforementioned cost budget constraint is met, and maintained during any degradation step. and .

8. The method according to claim 1, characterized in that, The repair minimum set of digital information packets is determined by the decoding dependency. The repair minimum set includes at least: key frame parameter set packets, key frame first packets, and the first packet of the first reference frame after the key frame. The repair minimum set is only copied and sent across links within the switching window.

9. The method according to claim 8, characterized in that, The cross-link replication transmission employs time-staggered transmission, meaning that for the same repair minimum set of packets, after transmission on the primary backhaul link, there is a delay... Send its replicated packets on the backup backhaul link, and satisfy: in, This is the maximum staggered duration threshold.

10. A video backhaul adaptive bitrate control system for underground coal mine image communication, characterized in that, include: The layered coding module is used to acquire the original video frames and generate a layered return video stream that contains at least the base layer bitstream and the enhancement layer bitstream; The handover warning receiving module is used to receive handover warning information; The feedback processing module is used to receive feedback messages and obtain the continuous packet loss length metric. The window determination and joint control module is used to determine the switching window and predict the interruption duration, and select from a preset candidate configuration set. Select joint configuration vector ; The synchronization digest module is used to generate a synchronization digest and send it along with the underlying bitstream. The replication scheduling module is used to perform cross-link replication and transmission of the repair minimum set digital information packets; The control telegraph module is used to send control telegraphs in telegraph-style short message format and to repeat the control telegraphs according to the number of repetitions. The reconstruction closed-loop module is used to trigger a reconstruction request and perform keyframe copying and pause the enhancement layer bitstream when the synchronous digest verification fails; The system is configured to perform the method according to any one of claims 1 to 9.

Images