Audio and video emergency backhaul method and system based on multi-source heterogeneous device cooperation

Abstract

This invention relates to the field of emergency communication and audio / video transmission technology, and discloses a method and system for collaborative audio / video emergency backhaul using multi-source heterogeneous devices. The method includes receiving raw sensor data sets from multiple independent devices, including uncompressed audio / video streams, sensor readings, and network probe data packets. Based on the network probe data packets, multi-link survivability assessment is performed, and a transmission channel is constructed in real time, generating a multi-modal network fusion transmission command. Based on this command and the raw audio / video stream, content-aware analysis is performed to dynamically derive coding parameters, generating an adaptive coding control signal. Combining device status reports and network situational awareness, a multi-device collaborative management command is generated to coordinate device protocol stacks and resource allocation. The above commands and information are integrated to form a backhaul execution decision, scheduling multiple devices to collaboratively complete the adaptive backhaul of audio / video data. This invention improves the link robustness and content transmission efficiency of audio / video backhaul in emergency scenarios.

Description

Technical Field

[0001] This invention relates to the field of emergency communication and audio / video transmission technology, specifically to a method and system for emergency audio / video backhaul using multi-source heterogeneous devices. Background Technology

[0002] In emergency command and disaster relief scenarios, transmitting audio and video data back using multiple independent devices on-site is crucial for obtaining on-site information. Existing technologies typically rely on a pre-set single communication link or a simple multi-link backup and switching mechanism. These methods primarily make decisions based on static network quality indicators, and encoding strategies generally employ fixed bitrate adjustments based on network bandwidth estimation, or simple resolution adaptation.

[0003] Conventional solutions have significant limitations when dealing with the complex and ever-changing network environments at emergency sites. On one hand, network infrastructure at emergency sites is frequently damaged, public and private networks may experience intermittent outages, and various heterogeneous links, such as satellite and ad hoc networks, coexist. Existing technologies lack dynamic assessment of the continuous connectivity and resilience of multiple links under real-world harsh environments. They cannot intelligently predict and construct a survivable logical transmission channel composed of complementary links before one link deteriorates or breaks, leading to frequent interruptions in backhaul services. On the other hand, traditional encoding adjustments only respond to changes in network conditions and fail to consider the differences in the content value of audio and video data. When bandwidth is limited, using the same compression strategy for key personnel's action areas and static backgrounds either results in the loss of important information or causes limited channel resources to be occupied by non-critical content, affecting backhaul efficiency and information effectiveness. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for emergency audio and video transmission through multi-source heterogeneous devices, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a method for emergency audio and video transmission via multi-source heterogeneous device collaboration, the method comprising:

[0006] Receive raw sensing data sets from multiple independent devices, the raw sensing data sets including uncompressed video data streams, uncompressed audio data streams, device-built-in sensor readings, and network probe data packets;

[0007] Based on the original sensing data set, a multimodal network fusion transmission command is generated, and the estimated available bandwidth range is determined through link parameter optimization during the generation process of the multimodal network fusion transmission command.

[0008] Based on the multimodal network fusion transmission command and the original sensing data set, content-aware analysis is performed on the uncompressed video data stream and the uncompressed audio data stream. Based on the content-aware analysis results and the estimated available bandwidth range in the multimodal network fusion transmission command, coding parameters are dynamically derived through a bitrate allocation model to generate an adaptive coding control signal.

[0009] Based on the adaptive coding control signal and the device status reports from the multiple independent devices, a multi-device collaborative management instruction is generated. The multi-device collaborative management instruction is used to coordinate the protocol stacks and resource allocation strategies of different devices.

[0010] The system integrates the multi-device collaborative management instructions, the adaptive encoding control signals, and the current network situation information to generate a backhaul execution decision, and schedules the multiple independent devices to complete the backhaul of audio and video data based on the backhaul execution decision.

[0011] Preferably, the step of generating multimodal network fusion transmission instructions based on the original sensing data set includes:

[0012] The network probe data packets are parsed to extract satellite beacon strength, ad hoc network neighbor table status, and mobile communication network slice identifier;

[0013] The satellite beacon strength, the ad hoc network neighbor table status, and the mobile communication network slice identifier are input into the transmission link decision engine, and the transmission link decision engine outputs a preliminary link selection scheme and a handover trigger threshold.

[0014] The preliminary link selection scheme is compared with the preset redundancy strategy, which defines the mapping relationship and activation conditions between the primary link and at least two backup links. A link redundancy configuration table is generated based on the comparison results.

[0015] Based on the geographical location information and motion vectors from the readings of the built-in sensors of the device, the link redundancy configuration table is corrected in real time to generate an enhanced link configuration table;

[0016] By combining the real-time latency and jitter data in the network probe data packets, the enhanced link configuration table is finally optimized to generate a multimodal network fusion transmission instruction that includes specific link parameters, switching logic, and fault avoidance rules.

[0017] Preferably, the preliminary link selection scheme is compared with a preset redundancy strategy. The redundancy strategy defines the mapping relationship and activation conditions between the primary link and at least two backup links. Based on the comparison result, a link redundancy configuration table is generated, including:

[0018] Query the preset strategy database to obtain a redundant strategy template that matches the current task scenario. The redundant strategy template includes a link priority sequence, bandwidth reservation ratio, and failure detection cycle.

[0019] The matching degree of the candidate links in the preliminary link selection scheme is calculated with the link priority sequence in the redundancy strategy template to generate a matching degree score for each candidate link.

[0020] Candidate links are sorted according to the matching score, and logical channels are allocated to each link according to the bandwidth reservation ratio in the redundancy strategy template to form an initial redundancy configuration.

[0021] By incorporating device attitude data from the device's built-in sensor readings, the stability of the initial redundancy configuration under device motion conditions is evaluated, and the allocation weights of the logical channels are dynamically adjusted.

[0022] The dynamically adjusted logical channel allocation results, the failure detection period, and the link health check rules are encapsulated to generate the link redundancy configuration table.

[0023] Preferably, the step of generating an adaptive coding control signal based on the multimodal network fusion transmission command and the original sensing data set includes:

[0024] The content feature extraction module is invoked to perform scene complexity analysis and motion intensity assessment on the uncompressed video data stream, generating a video content feature vector;

[0025] The audio feature extraction module is invoked to perform spectral analysis and loudness detection on the uncompressed audio data stream, generating an audio content feature vector;

[0026] The video content feature vector and the audio content feature vector are fused to generate a comprehensive content descriptor;

[0027] The integrated content descriptor and the estimated available bandwidth range in the multimodal network fusion transmission instruction are input together into the bitrate allocation model, and the target video bitrate, target audio bitrate and keyframe interval are output.

[0028] A set of encoding parameters is generated based on the target video bitrate, target audio bitrate, and keyframe interval. The set of encoding parameters is then fine-tuned based on abnormal jitter data from the device's built-in sensor readings to form the final adaptive encoding control signal.

[0029] Preferably, the step of inputting the integrated content descriptor and the estimated available bandwidth range in the multimodal network fusion transmission instruction into the bitrate allocation model, and outputting the target video bitrate, target audio bitrate, and keyframe interval, includes:

[0030] Establish a first mapping relationship with the comprehensive content descriptor as the independent variable and the visual quality prediction score as the dependent variable;

[0031] Establish a second mapping relationship with the comprehensive content descriptor as the independent variable and the audio quality prediction score as the dependent variable;

[0032] Under the constraint of the estimated available bandwidth range, the visual quality prediction score and the audio quality prediction score are weighted and summed to obtain the maximum weighted sum.

[0033] Based on the maximum weighted sum, determine the optimal bitrate allocation pair of the uncompressed video data stream and the uncompressed audio data stream within the available bandwidth.

[0034] Based on the target video bitrate, the target audio bitrate, and the keyframe interval, the bitrate allocation model is output.

[0035] Preferably, the step of generating multi-device collaborative management instructions based on the adaptive coding control signal and device status reports from the multiple independent devices includes:

[0036] Parse the device status report to obtain the identifier, supported protocol list, current remaining energy level, and physical interface status of each device;

[0037] Based on the list of supported protocols and the predefined protocol conversion rule library, negotiate and determine the protocol conversion path for each pair of devices that need to interact with data, and generate a protocol mapping relationship table between devices.

[0038] Based on the encoding format requirements in the adaptive encoding control signal, the inter-device protocol mapping table is verified to ensure that the encoding format can be supported by the protocol conversion path, and a verified protocol mapping relationship is generated.

[0039] Based on the current remaining energy level and task priority, a distributed energy dispatch plan is formulated, which specifies the power supply mode switching timing and load balancing strategy for each device.

[0040] The verified protocol mapping relationship, the distributed energy scheduling plan, and the display control commands generated based on the physical interface status are summarized and encapsulated to form the multi-device collaborative management instruction.

[0041] Preferably, parsing the device status report to obtain the identifier, supported protocol list, current remaining energy level, and physical interface status of each device includes:

[0042] The received device status report is parsed to identify the structured data fields and unstructured log information in the report;

[0043] The device's unique identifier string is extracted from the structured data fields and matched with the pre-registered device information database to verify the legitimacy of the device's identity.

[0044] The supported protocol list is parsed, broken down into specific communication protocol names and version number combinations, and the local protocol capability matrix is ​​queried to evaluate the compatibility level of each protocol in the current network environment.

[0045] The estimated sustainable operating time of the equipment is calculated by reading the value of the current remaining energy level and combining it with the benchmark energy consumption parameters corresponding to the equipment model.

[0046] The physical interface status is detected, and the connection status, negotiation rate, and error count of the interface are identified to generate an interface health score.

[0047] Preferably, the distributed energy dispatch plan is formulated by combining the current remaining energy level and task priority. The distributed energy dispatch plan specifies the power supply mode switching timing and load balancing strategy for each device, including:

[0048] Establish a two-dimensional decision matrix with the equipment's remaining energy level as the rows and the task priority as the columns, and assign an initial energy scheduling weight coefficient to each cell in the matrix;

[0049] The energy scheduling weight coefficient is dynamically adjusted based on the key indicators of the real-time backhaul task.

[0050] Based on the adjusted energy dispatch weight coefficient, the recommended operating power threshold for each device in the next dispatch cycle is calculated, and the triggering conditions for power supply mode switching are determined.

[0051] Construct a load balancing objective function with the optimization objectives of minimizing total system energy consumption and maximizing task completion rate, and solve for the load distribution ratio among each device.

[0052] The suggested operating power threshold, the power supply mode switching trigger condition, and the load allocation ratio are encapsulated into an energy dispatch strategy table, which serves as the core output of the distributed energy dispatch plan.

[0053] Preferably, the step of integrating the multi-device collaborative management instructions, the adaptive encoding control signals, and the current network situation information to generate a backhaul execution decision, and scheduling the multiple independent devices to complete the backhaul of audio and video data based on the backhaul execution decision, includes:

[0054] The multi-device collaborative management instructions are parsed to extract the verified inter-device protocol mapping relationship and distributed energy dispatch plan;

[0055] Read the adaptive coding control signal to obtain the final determined target video bitrate, target audio bitrate, and keyframe interval parameters;

[0056] The current network status information is obtained in real time from the network probe, including the bandwidth utilization, end-to-end latency, and packet loss rate statistics of each available link;

[0057] A backhaul decision engine is constructed, using the inter-device protocol mapping relationship, distributed energy scheduling plan, coding parameter set, and current network status information as input feature vectors.

[0058] The rule reasoning module in the backhaul decision engine and the machine learning model make joint decisions to output specific device scheduling sequences, data distribution paths and transmission timing control commands.

[0059] The device scheduling sequence, data distribution path, and transmission timing control command are sent to the corresponding independent devices, driving each device to synchronously start the encoding and transmission of audio and video data according to the specified encoding parameters and communication protocol.

[0060] Preferably, when the processor executes the computer program, it implements the steps of the multi-source heterogeneous device collaborative audio and video emergency transmission method as described in any one of the above-described methods.

[0061] Compared with the prior art, the beneficial effects of the present invention are:

[0062] By conducting multi-link survivability assessments on network probe data packets, a composite transmission channel can be constructed in real time based on the assessment results. This allows for dynamic selection and splicing of one or more link combinations, such as satellite, cellular, and ad hoc networks, to form a logically unified and more resilient data path. This enables the backhaul process to proactively adapt to sudden link interruptions and drastic quality fluctuations, maintaining uninterrupted data flow or rapid reconstruction in extremely unstable network environments, thus improving the overall robustness and service continuity of the backhaul path.

[0063] Content-aware analysis is performed on uncompressed raw audio and video streams to directly extract valuable information such as key regions, motion features, and audio events within the scene. Based on this analysis, a dynamically derived set of differentiated encoding parameters is generated, achieving a tight coupling between encoding strategies and data content value. This enables the system to automatically allocate more resources to high-value information regions when network bandwidth is limited, thereby optimizing the allocation efficiency of limited channel resources while maintaining overall bitrate control, and improving the information fidelity and effectiveness of the returned content. Attached Figure Description

[0064] Figure 1 This is a schematic diagram illustrating the working principle of the multi-source heterogeneous device collaborative audio and video emergency transmission method described in this invention.

[0065] Figure 2 A flowchart for generating multimodal network fusion transmission instructions;

[0066] Figure 3 A flowchart for generating adaptive coded control signals;

[0067] Figure 4 A diagram showing the combination of energy and load distribution for multi-source heterogeneous devices;

[0068] Figure 5 This is a modified intensity heatmap for the state fusion mapping. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] Please see Figure 1 This invention provides a method for emergency audio and video backhaul using multi-source heterogeneous devices. The method includes: integrating raw sensor data sets from multiple independent devices to achieve dynamic network fusion, adaptive coding, and multi-device collaborative scheduling, thereby achieving efficient and reliable backhaul of audio and video data in emergency scenarios. The raw sensor data set includes uncompressed video data streams, uncompressed audio data streams, device-built-in sensor readings, and network probe data packets. The method first generates a multimodal network fusion transmission command based on the raw sensor data set. The generation process of this command includes multi-link survivability assessment of the network probe data packets and real-time construction of the transmission channel. Next, an adaptive coding control signal is generated based on the multimodal network fusion transmission command and the raw sensor data set. The generation process of this signal includes content-aware analysis and dynamic derivation of coding parameters for the uncompressed video and audio data streams. Finally, a multi-device collaborative management command is generated based on the adaptive coding control signal and device status reports from multiple independent devices. This command is used to coordinate the protocol stacks and resource allocation strategies of different devices. Finally, the multi-device collaborative management instructions, adaptive encoding control signals, and current network status information are integrated to generate a backhaul execution decision, and multiple independent devices are scheduled to complete the backhaul of audio and video data based on the backhaul execution decision.

[0071] In one embodiment of the present invention, see [reference] Figure 2The network probe data packets in the original sensing data set are parsed to extract satellite beacon strength, ad hoc network neighbor table status, and mobile communication network slice identifiers. These data are then input into the transmission link decision engine, which outputs a preliminary link selection scheme and a handover trigger threshold. The preliminary link selection scheme is compared with a preset redundancy strategy. The redundancy strategy defines the mapping relationship and activation conditions between the primary link and at least two backup links. Based on the comparison results, a link redundancy configuration table is generated. Combining real-time geographic location information collected by the device's built-in sensors, the geographical environment characteristics of the transmission nodes corresponding to each link are determined, identifying scenarios such as obstruction or signal shielding that may affect link transmission quality. Simultaneously, motion vector data is used to analyze the device's movement trend, speed, and direction, predicting the potential impact of device location changes on the stability of each link connection. Based on the analysis results of geographical environment characteristics and device movement trends, the priority ranking and bandwidth allocation ratio of each link in the link redundancy configuration table are dynamically adjusted, prioritizing the allocation of link resources with stronger transmission stability in the current and predicted geographical environment. This results in an enhanced link configuration table adapted to the device's location status and movement trends. The real-time latency and jitter data in the network probe packets are combined to perform final optimization on the enhanced link configuration table, generating a multimodal network fusion transmission command that includes specific link parameters, switching logic, and fault avoidance rules.

[0072] A pre-defined strategy database is queried to retrieve a redundancy strategy template matching the current task scenario. This template includes a link priority sequence, bandwidth reservation ratio, and failure detection cycle. The candidate links in the initial link selection scheme are compared with the link priority sequence in the redundancy strategy template to calculate a matching score for each candidate link. The matching scores are used to rank the candidate links, and logical channels are allocated to each link according to the bandwidth reservation ratio in the redundancy strategy template, forming an initial redundancy configuration. Device attitude data from built-in sensors is used to evaluate the stability of the initial redundancy configuration under device motion conditions, dynamically adjusting the allocation weights of the logical channels. The dynamically adjusted logical channel allocation results, failure detection cycle, and link health check rules are encapsulated to generate a link redundancy configuration table.

[0073] In practical implementation, this can be described using a specific example scenario. In this scenario, multiple independent devices are deployed at the emergency site. The devices' built-in sensor readings indicate that the devices are in a mobile state, and network probe data packets are continuously captured from satellite, ad hoc network, and mobile communication network interfaces. The process of parsing the network probe data packets extracts example data including a satellite beacon strength of -85dBm, the ad hoc network neighbor table status showing three available neighbor nodes and their link quality, and the mobile communication network slice identifier "eMBB-Slice-1". The satellite beacon strength, ad hoc network neighbor table status, and mobile communication network slice identifier are input into the transmission link decision engine. The transmission link decision engine outputs a preliminary link selection scheme based on a preset link quality assessment algorithm. In the example, the priority order of the scheme is mobile communication network, ad hoc network, and satellite link, and the handover trigger threshold is set to a mobile communication network latency exceeding 200 milliseconds.

[0074] In some embodiments, the preliminary link selection scheme is compared with a preset redundancy strategy. The redundancy strategy defines the mapping relationship and activation conditions between the primary link and at least two backup links. For example, the primary link is a mobile communication network, the first backup link is an ad hoc network, and the second backup link is a satellite link. The activation conditions are latency exceeding the limit and continuous packet loss, respectively. A link redundancy configuration table is generated based on the comparison results. The configuration table entries include primary and backup link identifiers, logical channel allocation, and health check parameters. After querying the preset strategy database and obtaining a redundancy strategy template that matches the current task scenario, the redundancy strategy template includes a link priority sequence, bandwidth reservation ratio, and failure detection period. The matching degree between the candidate links in the preliminary link selection scheme and the link priority sequence in the redundancy strategy template is calculated. One calculation method can be implemented using the following formula:

[0075]

[0076] in: This represents the matching score of the j-th candidate link. This represents the weight factor of the i-th priority attribute. This represents the measured value of the j-th link on the i-th attribute. This represents the threshold of the i-th attribute in the redundancy policy template. It is a comparison function. This represents the total number of attributes involved in the matching process. Candidate links are ranked based on their matching scores, and logical channels are allocated to each link according to the bandwidth reservation ratio in the redundancy strategy template, forming an initial redundancy configuration. Device attitude data from built-in sensor readings, such as accelerometer readings indicating high-speed movement, is incorporated to evaluate the stability of the initial redundancy configuration under these conditions. The allocation weights of logical channels are dynamically adjusted, increasing the weight of links insensitive to latency jitter. The dynamically adjusted logical channel allocation results, failure detection cycles, and link health check rules are encapsulated to generate the final link redundancy configuration table.

[0077] In practice, the link redundancy configuration table is updated in real time based on the geographic location information and motion vectors from the device's built-in sensor readings. For example, if GPS data indicates that the device is entering a building-covered area, and motion vectors predict signal attenuation, an enhanced link configuration table is generated to prioritize backup links in advance. Combining real-time latency and jitter data from network probe packets—for example, if mobile network latency rises to 190 milliseconds and jitter intensifies—the enhanced link configuration table is optimized to generate a multimodal network fusion transmission command containing specific link parameters, switching logic, and fault avoidance rules. The command explicitly initiates a smooth handover process to the ad hoc network when latency reaches 195 milliseconds.

[0078] In one embodiment of the present invention, see [reference] Figure 3 The content feature extraction module performs scene complexity analysis and motion intensity assessment on the uncompressed video data stream, generating a video content feature vector. The audio feature extraction module performs spectral analysis and loudness detection on the uncompressed audio data stream, generating an audio content feature vector. The video and audio content feature vectors are fused to generate a comprehensive content descriptor. This comprehensive content descriptor, along with the estimated available bandwidth range from the multimodal network fusion transmission command, is input to the bitrate allocation model. The bitrate allocation model outputs the target video bitrate, target audio bitrate, and keyframe interval. The target video bitrate, target audio bitrate, and keyframe interval generate a coding parameter set, which is then fine-tuned based on abnormal jitter data from the device's built-in sensors to form the final adaptive coding control signal.

[0079] A first mapping relationship is established with the comprehensive content descriptor as the independent variable and the visual quality prediction score as the dependent variable. A second mapping relationship is established with the comprehensive content descriptor as the independent variable and the audio quality prediction score as the dependent variable. A joint optimization problem is constructed under the constraint of the estimated available bandwidth range. The objective of the joint optimization problem is to maximize the weighted sum of the visual quality prediction score and the audio quality prediction score. Solving the joint optimization problem yields the optimal bitrate allocation pair for the uncompressed video data stream and the uncompressed audio data stream within the available bandwidth range. The optimal bitrate allocation pair contains the numerical values ​​of the target video bitrate and the target audio bitrate. The motion intensity evaluation value in the video content feature vector is used as the basis. The keyframe interval corresponding to the motion intensity evaluation value is determined by a lookup table method, completing the output of the bitrate allocation model.

[0080] In practical implementation, this can be described using a specific example scenario. In this scenario, the uncompressed video data stream is a 1080p resolution raw video from a live camera, and the uncompressed audio data stream is PCM audio from a microphone array. The estimated available bandwidth range in the multimodal network fusion transmission command is a dynamic range of 1.5Mbps to 2.5Mbps. The content feature extraction module is invoked to perform scene complexity analysis and motion intensity assessment on the uncompressed video data stream, generating a video content feature vector. In the example, the scene complexity is calculated as 7.2 by measuring the texture entropy of the image frames, and the motion intensity assessment is calculated as 45 pixels / frame by averaging the motion vector size between consecutive frames. The audio feature extraction module is invoked to perform spectral analysis and loudness detection on the uncompressed audio data stream, generating an audio content feature vector. In the example, the spectral analysis yields an average spectral flatness of 0.15, and the loudness detection yields a current average loudness of -20dBFS. The video content feature vector and the audio content feature vector are fused to generate a comprehensive content descriptor, which is a multidimensional vector containing numerical features.

[0081] In some embodiments, the comprehensive content descriptor and the estimated available bandwidth range in the multimodal network fusion transmission instruction are jointly input into the bitrate allocation model. The bitrate allocation model outputs the target video bitrate, the target audio bitrate, and the keyframe interval. A first mapping relationship is established with the comprehensive content descriptor as the independent variable and the visual quality prediction score as the dependent variable. This first mapping relationship is represented by a pre-trained neural network model. A second mapping relationship is established with the comprehensive content descriptor as the independent variable and the audio quality prediction score as the dependent variable. This second mapping relationship is represented by a multinomial regression model. Under the constraint of the estimated available bandwidth range, a joint optimization problem is constructed. The objective of the joint optimization problem is to maximize the weighted sum of the visual quality prediction score and the audio quality prediction score, which is mathematically expressed as maximizing the objective function:

[0082]

[0083] in: This represents the overall quality target value. and These are the weighting coefficients for visual quality and audio quality, respectively, and they satisfy... , Indicates video bitrate The corresponding visual quality prediction score, Indicates audio bitrate Corresponding audio quality prediction score, decision variables and Constraints must be met , This represents the currently estimated upper limit of available bandwidth, while and It has a lower bound determined by encoder performance. Solving the joint optimization problem yields the optimal bitrate allocation pair for the video data stream and the uncompressed audio data stream within the available bandwidth. In the example, the target video bitrate is 1850 kbps, and the target audio bitrate is 128 kbps. Based on the motion intensity evaluation value in the video content feature vector, the keyframe interval corresponding to the motion intensity evaluation value is determined by a lookup table. When the motion intensity evaluation value is 45 pixels / frame, the lookup table shows a keyframe interval of 2 seconds, completing the output of the bitrate allocation model.

[0084] In practice, an encoding parameter set is generated based on the target video bitrate, target audio bitrate, and keyframe interval. This set includes the quantization parameters of the video encoder and the bitrate mode of the audio encoder. The encoding parameter set is then fine-tuned based on abnormal jitter data from the device's built-in sensors. For example, if the gyroscope reading indicates a sudden and severe shaking of the device, the fine-tuning process temporarily shortens the keyframe interval to 1 second and reserves more for forward error correction overhead, resulting in the final adaptive encoding control signal.

[0085] Optionally, the content feature extraction module can be implemented by combining a deep learning-based scene classification network with an optical flow estimation algorithm. It can be understood that the spectral analysis in the audio feature extraction module can calculate the Mel frequency cepstral coefficients and their dynamic characteristics. The bitrate allocation model can be solved using the Lagrange multiplier method. In solving the joint optimization problem of the bitrate allocation model, the specific implementation of the Lagrange multiplier method involves combining bandwidth constraints with a weighted sum of visual quality prediction scores and audio quality prediction scores as the objective, constructing an optimization function containing constraints by introducing Lagrange multipliers. Subsequently, by calculating the rate of change of this function with respect to video bitrate, audio bitrate, and Lagrange multipliers and finding the equilibrium point, the optimal bitrate allocation pair—the video and audio bitrate values ​​that maximize the weighted quality score within the available bandwidth—is solved. The weight coefficients in the joint optimization problem... and Settings can be configured based on the task configuration file, for example, in scenarios where voice command is the primary method. Greater than .

[0086] In one embodiment of the present invention, the device status report is parsed to obtain the identifier, supported protocol list, current remaining energy level, and physical interface status of each device. Based on the supported protocol list and a predefined protocol conversion rule base, a protocol conversion path is negotiated and determined for each pair of devices requiring data interaction, generating a device-to-device protocol mapping table. The encoding format requirements in the adaptive encoding control signal are used to verify the device-to-device protocol mapping table, ensuring that the encoding format is supported by the protocol conversion path, generating a verified protocol mapping relationship. The current remaining energy level is combined with task priority to formulate a distributed energy scheduling plan, which specifies the power supply mode switching timing and load balancing strategy for each device. The verified protocol mapping relationship, the distributed energy scheduling plan, and the explicit control commands generated based on the physical interface status are summarized and encapsulated into a multi-device collaborative management instruction.

[0087] In practice, the implementation can be described with reference to a specific example scenario in which device status reports are received from three independent devices, and the adaptive encoding control signal has determined the encoding format to be H.264 video encoding and AAC audio encoding. The process of parsing device status reports yields the following results: The first report identifies the device as "UAV_Terminal_01," supports protocols including "RTP / UDP / IP," "HTTP / 2overQUIC," and "Custom Mesh Protocol v2," has a current remaining energy level of 78%, and displays "Ethernet Port: Connected, 1Gbps" and "Wireless NIC 1: Connected, 802.11ac." The second report identifies the device as "Individual Soldier Device_02," supports protocols including "RTP / UDP / IP," "RTMP," and "Custom Mesh Protocol v1," has a current remaining energy level of 45%, and displays "Wireless NIC: Connected, 802.11n." The third report identifies the device as "Vehicle Repeater_03," supports protocols including "SRT / UDP / IP," "RTP / UDP / IP," and "Custom Mesh Protocol v2," has a current remaining energy level of 90%, and displays "Satellite Modem: Connected, 2Mbps" and "4G Module: Connected, LTE."

[0088] Based on the list of supported protocols and a predefined protocol conversion rule base, a protocol conversion path is negotiated and determined for each pair of devices that need to interact with each other. For example, video data transmission is required between "UAV_Terminal_01" and "Vehicle Relay_03". The intersection of their supported protocol lists includes "RTP / UDP / IP" and "Custom Mesh Protocol v2". The protocol conversion rule base defines "Custom Mesh Protocol v2" as having lower encapsulation overhead, so a protocol mapping table between devices is generated. The entries in the table specify that the transmission from "UAV_Terminal_01" to "Vehicle Relay_03" uses "Custom Mesh Protocol v2" and requires no conversion. Based on the encoding format requirements in the adaptive encoding control signal, such as the payload format of H.264 over RTP, the protocol mapping table between devices is verified to ensure that the encoding format is supported by the protocol conversion path. In the example, the protocol documentation for "Custom Mesh Protocol v2" declares support for encapsulating H.264 NALU units, thus generating a verified protocol mapping relationship. Based on the current remaining energy level and task priority, and with the task priority configuration file indicating a "high" priority task, a distributed energy scheduling plan is formulated. This plan specifies the timing of power mode switching and load balancing strategies for each device. For example, the plan stipulates that "Individual Soldier Device_02," due to its low remaining energy level, will switch to low-power receiving mode in the next scheduling cycle, while "Vehicle-Mounted Repeater_03" will bear more data forwarding load. The verified protocol mapping relationships, the distributed energy scheduling plan, and the display control commands generated based on physical interface status are summarized. These display control commands indicate that the Ethernet port status of "UAV_Terminal_01" is prioritized as the primary physical port, and are encapsulated into multi-device collaborative management instructions.

[0089] In some embodiments, the protocol conversion rule base is a database containing a protocol conversion capability matrix and conversion overhead costs. It can be understood that the verified protocol mapping table is a structured list recording the source device, destination device, application data format, the finally negotiated transport protocol, and the necessary conversion gateway addresses. The formulation of a distributed energy scheduling plan can introduce a decision formula to calculate the load allocation ratio, such as the forwarding load allocated to device i. This can be achieved through calculation:

[0090]

[0091] in: This indicates the current remaining energy level of device i. It is an energy sensitivity coefficient greater than 0. This represents the overall connectivity score of device i. This indicates the total load that the system needs to allocate. It represents the total number of devices participating in the collaboration.

[0092] Optionally, the display control commands generated for physical interface status may include commands instructing the device to switch to the active physical network interface. Device status reports can be structured and encapsulated in a predefined JSON or Protocol Buffers format for easy parsing. The conversion overhead costs defined in the protocol conversion rule base may include quantitative indicators of additional latency, header overhead bytes, and computational resource consumption. Task priorities can be dynamically issued from the central control center or inferred locally by field devices based on the perceived type of emergency event.

[0093] See Figure 4 This is a combined chart of energy and load distribution across multi-source heterogeneous devices, representing a professional data visualization result from the device collaborative management phase of an audio / video emergency backhaul system. Load distribution is positively correlated with remaining energy levels, consistent with the strategy of "prioritizing high-energy devices" in emergency backhaul. Individual device_02, with its lower energy (45%), shows a significantly reduced load distribution ratio, reflecting low-power protection logic. This chart is used to verify the rationality of the energy-sensitive load scheduling algorithm, ensuring the system's continuous operation capability in emergency scenarios. Balanced load distribution (with similar loads for drones and vehicle-mounted relays) prevents single-device overload and improves system stability.

[0094] In one embodiment of the present invention, the received device status report is parsed to identify structured data fields and unstructured log information. A unique device identifier string is extracted from the structured data fields and matched against a pre-registered device information database to verify the legitimacy of the device's identity. Protocol list parsing is supported, breaking it down into specific communication protocol names and version numbers, and querying the local protocol capability matrix to assess the compatibility level of each protocol in the current network environment. The current remaining energy level is read and combined with the baseline energy consumption parameters corresponding to the device model to calculate the estimated sustainable operating time of the device. The calculation formula is as follows:

[0095]

[0096] in: This indicates the estimated continuous operating time of the equipment. This indicates the current remaining energy level of the device (normalized value). This indicates the baseline battery capacity parameter corresponding to the device model. This indicates the average power consumption parameter of the device in the current operating mode.

[0097] Physical interface status detection identifies the interface's connection status, negotiation rate, and error count, generating an interface health score. The calculation formula is as follows:

[0098]

[0099] in: This indicates the interface health score. , , These represent the weighting coefficients for connection state, negotiation rate, and error count, respectively. This indicates the connection state score. Indicates the actual negotiated rate of the interface. Indicates the maximum speed supported by the interface. This indicates the cumulative error count for the interface. This indicates the threshold for interface error counting.

[0100] A two-dimensional decision matrix is ​​established with equipment remaining energy levels as rows and task priorities as columns. An initial energy scheduling weight coefficient is assigned to each cell in the matrix. Key performance indicators (KPIs) of the tasks are transmitted in real-time, and the energy scheduling weight coefficients are dynamically adjusted accordingly. These KPIs include data timeliness requirements and content importance levels. Based on the adjusted energy scheduling weight coefficients, the recommended operating power threshold for each device in the next scheduling cycle is calculated, and the triggering conditions for power supply mode switching are determined. A load balancing objective function is constructed, with the optimization objectives of minimizing total system energy consumption and maximizing task completion rate, to determine the load distribution ratio among devices. The recommended operating power threshold, power supply mode switching triggering conditions, and load distribution ratio are encapsulated into an energy scheduling strategy table, serving as the core output of the distributed energy scheduling plan.

[0101] In practical implementation, this can be described using a specific example scenario. In this scenario, a device status report is received from a device with the identifier "Individual Soldier Device_05," and its data payload is a JSON-formatted text file. The received device status report is parsed to identify the structured data fields and unstructured log information. The structured data fields include "deviceID," "protocolList," "batteryLevel," and "interfaceStatus," while the unstructured log information contains a message: "Warning: Ethernetportnegotiationretrycountexceeded3." The unique device identifier string "Individual Soldier Device_05" is extracted from the structured data fields and matched against a pre-registered device information database. The pre-registered device information database stores the public key fingerprint corresponding to this identifier. Upon successful verification, the legitimacy of the device's identity is confirmed. The "protocolList" field, representing the supported protocols, is parsed and broken down into specific communication protocol names and version numbers, resulting in a list of ["RTP / v2.0", "SRT / v1.4", "MQTT / v3.1.1"]. The local protocol capability matrix is ​​then queried to assess the compatibility level of each protocol in the current network environment. For example, in a network environment characterized by high packet loss, the local protocol capability matrix shows that the SRT protocol has a compatibility level of "high," while the MQTT protocol has a level of "medium." The remaining energy level, "batteryLevel," is read; its value is 45. Combined with the baseline energy consumption parameters corresponding to the device model, the estimated continuous operating time of the device is calculated to be approximately 5.6 hours. The "interfaceStatus" field of the physical interface status descriptor is checked, identifying the interface's connection status as "eth0:up," negotiation rate as "1000Mbps," and error count as "rx_errors:12." An interface health score is generated, using the error count as a negative factor and the negotiation rate as a positive factor in the scoring model, resulting in a health score of 85 for this interface.

[0102] In some embodiments, the process of developing a distributed energy dispatch plan by combining the current remaining energy level with task priority involves collaborative energy management among multiple devices. A two-dimensional decision matrix is ​​established, with the device's remaining energy level as rows and task priority as columns. Task priority is divided into three levels: "high," "medium," and "low." An initial energy dispatch weight coefficient is assigned to each cell in the matrix. For example, for a cell with a remaining energy level in the range of (40%, 60%) and a task priority of "high," the initial weight coefficient is set to 0.8. The energy dispatch weight coefficient is dynamically adjusted based on the key indicators of the real-time transmitted tasks. Key indicators include data timeliness requirements and content importance levels. For example, when the content importance level is upgraded from "normal" to "critical," the weight coefficient of the corresponding cell is increased by 0.15. Based on the adjusted energy... The scheduling weight coefficient is used to calculate the recommended operating power threshold for each device in the next scheduling cycle and determine the trigger condition for power supply mode switching. For example, for "Individual Soldier Device_05", the calculated recommended operating power threshold is 6.5 watts, and the trigger condition is that if the real-time power consumption exceeds this threshold for 10 consecutive seconds, it will switch to a low frame rate encoding mode. A load balancing objective function is constructed, with the optimization objectives of minimizing total system energy consumption and maximizing task completion rate, to determine the load distribution ratio among devices. The recommended operating power threshold, power supply mode switching trigger condition, and load distribution ratio are encapsulated into an energy scheduling strategy table, which serves as the core output of the distributed energy scheduling plan. See Table 1.

[0103] Table 1: Energy Dispatch Strategy Table

[0104]

[0105] In practical implementation, the dynamic adjustment of the energy dispatch weight coefficient can be achieved through an incremental calculation, with the adjustment amount... Determined by the formula:

[0106]

[0107] in: This represents the increment of the weight coefficient adjustment for the cell corresponding to the i-th energy level interval and the j-th task priority level. It is a factor related to data timeliness requirements. It is a normalized timeliness urgency index. It is a content importance ranking factor. It is a normalized content importance index. The new weight coefficients are constrained within the range of [0,1], and are the sum of the original weight coefficients and the weight coefficient adjustment increments, thereby realizing the dynamic adjustment of the energy dispatch weight coefficients.

[0108] In one embodiment of the invention, multi-device collaborative management instructions are parsed to extract verified inter-device protocol mapping relationships and distributed energy scheduling plans. Adaptive encoding control signals are read to obtain the final determined target video bitrate, target audio bitrate, and keyframe interval parameters. Network probes acquire real-time current network status information, including bandwidth utilization, end-to-end latency, and packet loss rate statistics for each available link. A backhaul decision engine is constructed, using inter-device protocol mapping relationships, distributed energy scheduling plans, encoding parameter sets, and current network status information as input feature vectors. The rule reasoning module and machine learning model in the backhaul decision engine perform joint decision-making to output specific device scheduling sequences, data distribution paths, and transmission timing control commands. The device scheduling sequences, data distribution paths, and transmission timing control commands are issued to the corresponding independent devices, driving each device to synchronously initiate audio and video data encoding and transmission operations according to specified encoding parameters and communication protocols.

[0109] In practical implementation, this can be described using a specific example scenario. In this scenario, multi-device collaborative management instructions have been encapsulated and generated, including verified inter-device protocol mapping relationships and a distributed energy scheduling plan. The adaptive coding control signal has determined the target video bitrate to be 2.1 Mbps, the target audio bitrate to be 96 kbps, and the keyframe interval to be 30 frames. The process of parsing the multi-device collaborative management instructions involves extracting the verified inter-device protocol mapping relationships and the distributed energy scheduling plan. For example, the plan stipulates that "UAV relay D" will stop its relay mission and enter hibernation when its battery level is below 30%. The adaptive coding control signal is read to obtain the finally determined target video bitrate of 2.1 Mbps, target audio bitrate of 96 kbps, and keyframe interval parameter of 30 frames. These parameters constitute the coding parameter set. The network probe obtains real-time information on the current network status. In the example, the information includes the bandwidth utilization of the available link "4G_Link_1" at 85%, the end-to-end latency at 110ms, and the packet loss rate at 0.5%, and the bandwidth utilization of the available link "Satellite_Link_2" at 40%, the end-to-end latency at 800ms, and the packet loss rate at 2%.

[0110] A backhaul decision engine is constructed, which takes the protocol mapping relationship between devices, the distributed energy scheduling plan, the coding parameter set and the current network status information as the input feature vector. The input feature vector is a combined data structure containing discrete category fields and continuous numerical fields. The decision-making process is jointly conducted by the rule reasoning module and the machine learning model in the backhaul decision engine. The rule reasoning module handles explicit policy logic, such as the rule "if the end-to-end delay of any path exceeds 500ms, then forward error correction is enabled". The machine learning model handles complex nonlinear relationships, such as a trained gradient boosting tree model used to predict the overall transmission success probability under different distribution paths. The output of the joint decision is a specific device scheduling sequence, data distribution path and transmission timing control command. In the example, the output device scheduling sequence is ["Handheld acquisition terminal A starts encoding and sending", "Vehicle aggregation gateway B starts receiving, transcoding and forwarding", "UAV relay D starts listening and backup forwarding"]. The data distribution path is "Handheld acquisition terminal A -> (SRT over UDP) -> Vehicle aggregation gateway B -> (Private protocol over QUIC) -> Command center server C". The transmission timing control command includes "Vehicle aggregation gateway B starts sending to command center server C after receiving the 5th video data packet". The device scheduling sequence, data distribution path, and transmission timing control commands are issued to the corresponding independent devices. The instructions are reliably transmitted through the device management channel, driving each device to synchronously start the encoding and transmission of audio and video data according to the specified encoding parameters and communication protocols. For example, after receiving the command, "Handheld Acquisition Terminal A" immediately sets its video encoder target bitrate to 2.1Mbps and sends a data stream to the specified IP and port of "Vehicle Aggregation Gateway B" based on the SRT protocol stack.

[0111] In some embodiments, the machine learning model portion of the backhaul decision engine may employ a feedforward neural network whose output layer provides a decision value. The calculation can be expressed as:

[0112]

[0113] in: It represents a scalar score for a candidate path. It is the Sigmoid activation function. , , , These are the weight matrices and bias vectors for the first and second layers of the neural network, respectively. It is a preprocessed input feature vector. It is a modified linear unit activation function. It can be understood that the rule-based reasoning module operates based on a set of "condition-action" rules, for example, the condition being "IF link packet loss rate > 1% AND path hop count > 2 THEN action = enable application layer retransmission".

[0114] Optionally, network probes can be implemented using software modules deployed on gateway devices or independent probe nodes, periodically sending probe packets to the target address and collecting statistical information. Device management channels can be based on a separate, high-priority, low-speed control link, such as a signaling channel using LoRa or a cellular network. Transmission timing control commands can include precise timestamps or actions triggered based on specific packet sequence numbers to achieve operational synchronization between devices. During joint decision-making, the output of the rule-based reasoning module and the output of the machine learning model can be fused through a weighted voting mechanism to form the final scheduling instructions.

[0115] See Figure 5 This is a heatmap of the correction intensity of state fusion mapping, a professional visualization chart used in multi-dimensional data fusion scenarios (such as mobile antenna tracking and multi-source device collaboration). The chart shows the distribution of correction intensity in the spatiotemporal dimensions of the state fusion mapping. Higher correction intensity indicates greater state deviation in the corresponding spatiotemporal region, requiring stronger feedback correction to ensure data accuracy. This is commonly seen in systems requiring spatiotemporal feature fusion, used to locate spatiotemporal regions with large state deviations, guiding subsequent control strategy optimization. Regions with high correction intensity typically correspond to moments of strong system disturbance and can serve as key areas of focus for fault diagnosis and algorithm optimization.

[0116] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0117] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An emergency audio and video transmission method based on multi-source heterogeneous device collaboration, characterized in that, The method includes: Receive raw sensing data sets from multiple independent devices, the raw sensing data sets including uncompressed video data streams, uncompressed audio data streams, device-built-in sensor readings, and network probe data packets; Based on the original sensing data set, a multimodal network fusion transmission command is generated, and the estimated available bandwidth range is determined through link parameter optimization during the generation process of the multimodal network fusion transmission command. Based on the multimodal network fusion transmission command and the original sensing data set, content-aware analysis is performed on the uncompressed video data stream and the uncompressed audio data stream. Based on the content-aware analysis results and the estimated available bandwidth range in the multimodal network fusion transmission command, coding parameters are dynamically derived through a bitrate allocation model to generate an adaptive coding control signal. Multi-device collaborative management instructions are generated based on the adaptive coding control signal and device status reports from the multiple independent devices; The system integrates the multi-device collaborative management instructions, the adaptive encoding control signals, and the current network situation information to generate a backhaul execution decision, and schedules the multiple independent devices to complete the backhaul of audio and video data based on the backhaul execution decision; The step of generating multimodal network fusion transmission instructions based on the original sensing data set includes: The network probe data packets are parsed to extract satellite beacon strength, ad hoc network neighbor table status, and mobile communication network slice identifier; The satellite beacon strength, the ad hoc network neighbor table status, and the mobile communication network slice identifier are input into the transmission link decision engine, and the transmission link decision engine outputs a preliminary link selection scheme. The preliminary link selection scheme is compared with the preset redundancy strategy, which defines the mapping relationship and activation conditions between the primary link and at least two backup links. A link redundancy configuration table is generated based on the comparison results. Based on the geographical location information and motion vectors from the readings of the built-in sensors of the device, the link redundancy configuration table is corrected in real time to generate an enhanced link configuration table; By combining the real-time latency and jitter data in the network probe data packets, the enhanced link configuration table is finally optimized to generate a multimodal network fusion transmission instruction that includes specific link parameters, switching logic, and fault avoidance rules.

2. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 1, characterized in that, The preliminary link selection scheme is compared with a preset redundancy strategy. The redundancy strategy defines the mapping relationship and activation conditions between the primary link and at least two backup links. Based on the comparison results, a link redundancy configuration table is generated, including: Query the preset strategy database to obtain a redundant strategy template that matches the current task scenario. The redundant strategy template includes a link priority sequence, bandwidth reservation ratio, and failure detection cycle. The matching degree of the candidate links in the preliminary link selection scheme is calculated with the link priority sequence in the redundancy strategy template to generate a matching degree score for each candidate link. Candidate links are sorted according to the matching score, and logical channels are allocated to each link according to the bandwidth reservation ratio in the redundancy strategy template to form an initial redundancy configuration. By incorporating device attitude data from the device's built-in sensor readings, the stability of the initial redundancy configuration under device motion conditions is evaluated, and the allocation weights of the logical channels are dynamically adjusted. The dynamically adjusted logical channel allocation results, the failure detection period, and the link health check rules are encapsulated to generate the link redundancy configuration table.

3. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 1, characterized in that, The process of generating an adaptive coding control signal based on the multimodal network fusion transmission command and the original sensing data set includes: The content feature extraction module is invoked to perform scene complexity analysis and motion intensity assessment on the uncompressed video data stream, generating a video content feature vector; The audio feature extraction module is invoked to perform spectral analysis and loudness detection on the uncompressed audio data stream, generating an audio content feature vector; The video content feature vector and the audio content feature vector are fused to generate a comprehensive content descriptor; The integrated content descriptor and the estimated available bandwidth range in the multimodal network fusion transmission instruction are input into the bitrate allocation model to output the target video bitrate, target audio bitrate, and keyframe interval. A set of encoding parameters is generated based on the target video bitrate, target audio bitrate, and keyframe interval. The set of encoding parameters is then fine-tuned based on abnormal jitter data from the device's built-in sensor readings to form the final adaptive encoding control signal.

4. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 3, characterized in that, The step of inputting the integrated content descriptor and the estimated available bandwidth range from the multimodal network fusion transmission instruction into the bitrate allocation model, and outputting the target video bitrate, target audio bitrate, and keyframe interval, includes: Establish a first mapping relationship with the comprehensive content descriptor as the independent variable and the visual quality prediction score as the dependent variable; Establish a second mapping relationship with the comprehensive content descriptor as the independent variable and the audio quality prediction score as the dependent variable; Under the constraint of the estimated available bandwidth range, the visual quality prediction score and the audio quality prediction score are weighted and summed to obtain the maximum weighted sum. Based on the maximum weighted sum, determine the optimal bitrate allocation pair of the uncompressed video data stream and the uncompressed audio data stream within the available bandwidth. Based on the target video bitrate, the target audio bitrate, and the keyframe interval, the bitrate allocation model is output.

5. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 1, characterized in that, The step of generating multi-device collaborative management instructions based on the adaptive coding control signal and device status reports from the multiple independent devices includes: Parse the device status report to obtain the identifier, supported protocol list, current remaining energy level, and physical interface status of each device; Based on the list of supported protocols and the predefined protocol conversion rule library, negotiate and determine the protocol conversion path for each pair of devices that need to interact with data, and generate a protocol mapping relationship table between devices. Based on the encoding format requirements in the adaptive encoding control signal, the inter-device protocol mapping table is verified to ensure that the encoding format can be supported by the protocol conversion path, and a verified protocol mapping relationship is generated. Based on the current remaining energy level and task priority, a distributed energy dispatch plan is formulated, which specifies the power supply mode switching timing and load balancing strategy for each device. The verified protocol mapping relationship, the distributed energy scheduling plan, and the display control commands generated based on the physical interface status are summarized and encapsulated to form the multi-device collaborative management instruction.

6. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 5, characterized in that, The process of parsing the device status report to obtain the identifier, supported protocol list, current remaining energy level, and physical interface status of each device includes: The received device status report is parsed to identify the structured data fields and unstructured log information in the report; The device's unique identifier string is extracted from the structured data fields and matched with the pre-registered device information database to verify the legitimacy of the device's identity. The supported protocol list is parsed, broken down into specific communication protocol names and version number combinations, and the local protocol capability matrix is ​​queried to evaluate the compatibility level of each protocol in the current network environment. The estimated sustainable operating time of the equipment is calculated by reading the value of the current remaining energy level and combining it with the baseline energy consumption parameters corresponding to the equipment model. The physical interface status is detected, and the connection status, negotiation rate, and error count of the interface are identified to generate an interface health score.

7. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 5, characterized in that, The plan, which combines the current remaining energy level with task priority, is used to formulate a distributed energy dispatch plan. This plan specifies the timing of power supply mode switching and load balancing strategies for each device, including: Establish a two-dimensional decision matrix with the equipment's remaining energy level as the rows and the task priority as the columns, and assign an initial energy scheduling weight coefficient to each cell in the matrix; The energy scheduling weight coefficient is dynamically adjusted based on the key indicators of the real-time backhaul task. Based on the adjusted energy dispatch weight coefficient, the recommended operating power threshold for each device in the next dispatch cycle is calculated, and the triggering conditions for power supply mode switching are determined. Construct a load balancing objective function with the optimization objectives of minimizing total system energy consumption and maximizing task completion rate, and solve for the load distribution ratio among each device. The suggested operating power threshold, the power supply mode switching trigger condition, and the load allocation ratio are encapsulated into an energy dispatch strategy table, which serves as the core output of the distributed energy dispatch plan.

8. The audio and video emergency transmission method based on multi-source heterogeneous device collaboration according to claim 1, characterized in that, The process of integrating the multi-device collaborative management instructions, the adaptive encoding control signals, and the current network situation information to generate a backhaul execution decision, and scheduling the multiple independent devices to complete the backhaul of audio and video data based on the backhaul execution decision, includes: The multi-device collaborative management instructions are parsed to extract the verified inter-device protocol mapping relationship and distributed energy dispatch plan; Read the adaptive coding control signal to obtain the final determined target video bitrate, target audio bitrate, and keyframe interval parameters; The current network status information is obtained in real time from the network probe, including the bandwidth utilization, end-to-end latency, and packet loss rate statistics of each available link; A backhaul decision engine is constructed, using the inter-device protocol mapping relationship, distributed energy scheduling plan, coding parameter set, and current network status information as input feature vectors. The rule reasoning module in the backhaul decision engine and the machine learning model make joint decisions to output specific device scheduling sequences, data distribution paths and transmission timing control commands. The device scheduling sequence, data distribution path, and transmission timing control command are sent to the corresponding independent devices, driving each device to synchronously start the encoding and transmission of audio and video data according to the specified encoding parameters and communication protocol.

9. An audio and video emergency transmission system based on multi-source heterogeneous device collaboration, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the audio and video emergency transmission method for multi-source heterogeneous devices as described in any one of claims 1 to 8.

Images