Multi-modal data end-edge-cloud collaborative perception method and system in complex dynamic environment

Abstract

The present application relates to the technical field of construction safety monitoring, and particularly relates to a multi-modal data end-edge-cloud collaborative perception method and system in a complex dynamic environment, comprising: a tower crane cluster collects construction site perception data through multi-modal sensors, and labels the data according to priority levels based on transmission timeliness and data integrity requirements, data modal bandwidth sensitivity characteristics and spatio-temporal coupling degree; an edge node monitors network state parameters, and dynamically generates a collaborative scheduling strategy containing compression encoding and transmission rate allocation according to the priority; the data is transmitted to the cloud platform in stages after corresponding compression encoding is performed according to the priority order; the cloud platform performs decompression and integrity checking, and distributes the data according to the priority and returns feedback information. The present application improves the real-time performance and reliability of multi-modal data transmission in a dynamic network.

Description

Technical Field

[0001] This invention relates to the field of building construction safety monitoring technology, and in particular to a multimodal data edge-cloud collaborative sensing method and system for complex dynamic environments. Background Technology

[0002] In the complex and dynamic environment of tower crane cluster collaborative operations, multiple tower cranes acquire real-time visual images, point cloud data, and structural status information of the construction site through multimodal sensors (such as high-definition cameras, LiDAR, and vibration sensors) deployed at each node. In existing technologies, this sensing data is typically packaged uniformly and uploaded to a cloud platform for centralized processing via edge nodes, with the edge nodes performing only simple data forwarding or caching functions. Conventional practices rely on fixed bandwidth allocation and uniform compression strategies, such as applying the same compression ratio or preset transmission rate to all modal data, to simplify system design and reduce implementation complexity.

[0003] This conventional approach has significant drawbacks. Firstly, the network conditions at construction sites fluctuate wildly, making it difficult for fixed compression coding parameters to adapt to sudden bandwidth changes. This leads to increased latency for high-priority data (such as real-time alarm images) during congestion, while low-priority data (such as periodic vibration records) consumes excessive resources during idle periods, resulting in low transmission efficiency and compromised data integrity. Secondly, different modalities of data have varying requirements for transmission timeliness and data integrity. For example, visual data is sensitive to real-time performance but can tolerate moderate compression loss, while structural monitoring data requires high fidelity but allows for slower transmission. Unified processing ignores these differences and cannot optimize the timeliness and reliability of the overall sensing task under limited bandwidth. Especially when there is spatiotemporal coupling between tower cranes (such as in collaborative hoisting scenarios), data synchronization delays further exacerbate system decision-making biases. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for multimodal data edge-cloud collaborative sensing in complex dynamic environments, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a multimodal data edge-cloud collaborative sensing method for complex dynamic environments, comprising: In a tower crane cluster, each tower crane collects construction site perception data through multimodal sensors. Based on the differentiated requirements of the perception task for transmission timeliness and data integrity, the sensitivity of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the collected perception data is classified and labeled with transmission priority, generating multimodal data to be transmitted with priority tags. The edge node monitors the network status parameters of the cloud platform in real time. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the edge-cloud collaborative transmission scheduling strategy is dynamically generated. The transmission scheduling strategy includes compression coding parameters and transmission rate allocation schemes corresponding to different priority data. According to the edge-cloud collaborative transmission scheduling strategy, the edge nodes perform corresponding compression encoding on the multimodal data to be transmitted carrying priority tags and then transmit them to the cloud platform level by level. The cloud platform receives multimodal data transmitted hierarchically from edge nodes, decompresses and verifies the integrity of the received multimodal data, and distributes the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

[0006] The collected sensor data is prioritized and labeled for transmission, including: Based on the sensing task type, the transmission timeliness level and data integrity level corresponding to each sensing data are determined. The transmission timeliness level represents the maximum tolerable delay time of the sensing data during transmission, and the data integrity level represents the maximum allowable data loss ratio of the sensing data during transmission. The sensing task type is divided according to the real-time control requirements of the tower crane operation function supported by the sensing data. Based on the data mode to which the sensed data belongs, the modal transmission bandwidth sensitivity level corresponding to each sensed data is determined. The modal transmission bandwidth sensitivity level characterizes the degree of degradation of the sensed quality of different data modes under the condition of limited transmission bandwidth. Based on the relative spatial location of the tower crane from which each sensing data originates within the tower crane cluster and the temporal synchronization relationship with sensing data from other tower cranes, the spatiotemporal coupling level corresponding to each sensing data is determined; The transmission timeliness level, data integrity level, modal transmission bandwidth sensitivity level, and spatiotemporal coupling level are quantified into corresponding level scores, and then weighted and fused according to preset weight coefficients for each dimension to obtain a comprehensive transmission priority index for each sensed data. Based on the preset threshold range in which the comprehensive transmission priority index is located, a corresponding priority label is assigned to each sensed data.

[0007] Dynamically generate edge-cloud collaborative transmission scheduling strategies, including: The available bandwidth, round-trip time, packet loss rate, and latency jitter of the uplink transmission link from the edge node to the cloud platform are obtained in real time and used as current network status parameters. Based on the available bandwidth and packet loss rate in the current network status parameters, the overall effective transmission capacity of the uplink transmission link in the current transmission cycle is evaluated to obtain the total amount of currently available transmission resources; According to the priority level indicated by the priority tag of each multimodal data to be transmitted, from high to low, the transmission bandwidth of the total available transmission resources is allocated to the multimodal data to be transmitted at each priority level in sequence. The transmission bandwidth quota allocated to each priority level is not less than the basic transmission bandwidth value required by the data of that priority to meet its transmission timeliness requirements. When the total available transmission resources are insufficient to meet the basic transmission bandwidth value of all priorities, the basic transmission bandwidth requirement of the data with the highest priority level is guaranteed first. After allocating the transmission bandwidth quota for each priority data, the corresponding compression coding parameters and transmission rate are determined for each priority data based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters.

[0008] Based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters, the corresponding compression coding parameters and transmission rate are determined for each priority data, including: The maximum allowable transmission time for each priority data is calculated based on the ratio of the transmission bandwidth quota for each priority data to the total amount of data to be transmitted for that priority data. From the preset set of compression coding parameters, all candidate compression coding parameter combinations suitable for the data mode are selected according to the data mode to which each priority data belongs, and candidate compression coding parameter combinations with compression ratios lower than the preset minimum compression ratio threshold or decompression quality evaluation indicators lower than the preset minimum quality threshold are removed. For each candidate compression coding parameter combination retained after elimination, the compressed data volume is estimated based on the compression ratio of the candidate compression coding parameter combination and the total amount of data to be transmitted. Based on the compressed data volume, the transmission bandwidth quota, and the current packet loss rate and latency jitter, the actual transmission time and decompressed data quality under the candidate compression coding parameter combination are estimated. From the candidate compression coding parameter combinations whose estimated actual transmission time does not exceed the maximum allowable transmission time, the compression coding parameter combination with the highest data quality after decompression is selected as the compression coding parameter for the corresponding priority data, and the transmission rate of the corresponding priority data is set according to the output bit rate of the selected compression coding parameter combination.

[0009] Multimodal data to be transmitted, carrying priority tags, is processed with corresponding compression encoding according to transmission priority and then transmitted to the cloud platform level by level, including: Based on the compression coding parameters determined for each priority data in the edge-cloud collaborative transmission scheduling strategy, compression coding is performed on the multimodal data to be transmitted carrying the corresponding priority tags. The compressed data stream is divided into data packets according to the preset data packet length and composed into a compressed data packet sequence, generating a compressed data packet sequence corresponding to each priority data. Assign a transmission sequence identifier to the compressed data packet sequence corresponding to each priority data, associate the transmission sequence identifier with the compressed data packet sequence, and carry the transmission sequence identifier and the total number of data packets in the first data packet of the compressed data packet sequence; The compressed data packet sequences corresponding to each priority data are written into the sending buffer queue of the edge node in order of priority. The sending buffer queue is divided into independent logical sub-queues according to priority. Within the same priority, the compressed data packet sequences are arranged in the corresponding logical sub-queues according to the generation time of the compressed data packet sequences. According to the transmission rate corresponding to each priority data, the compressed data packets are taken out from the transmission buffer queue in order of priority and sent to the cloud platform through the uplink transmission link in order of priority. After all the compressed data packets corresponding to the current priority data have been sent, the transmission of the compressed data packets of the next priority data is started.

[0010] The transmission feedback information is sent back to the edge node, including: After completing the reception and integrity verification of multimodal data of each priority level in this transmission cycle, the cloud platform will calculate the actual amount of data successfully received in this transmission cycle, the actual time taken from the start of transmission to the completion of all data reception, the number of data packets lost during transmission, and the number of data segments that failed integrity verification for each priority level. The actual transmission rate corresponding to each priority data is calculated based on the actual amount of data successfully received and the actual time consumed. The actual packet loss rate corresponding to each priority data is calculated based on the ratio of the number of data packets lost to the total amount of data sent for the corresponding priority. The amount of data corresponding to the data segments that failed the integrity check is included in the retransmission requirement data amount. The actual transmission rate, actual time consumption, actual packet loss rate, and retransmission requirement data volume corresponding to each priority data are used as the transmission status feedback information of that priority data. The transmission status feedback information of all priorities is arranged in descending order of priority and associated with the corresponding priority tags. Then, a transmission feedback report is generated in the form of a structured data report. The transmission feedback report is transmitted back to the edge node via the downlink control link from the cloud platform to the edge node.

[0011] The method further includes: The edge node receives the transmission feedback report corresponding to the current transmission cycle from the cloud platform, and parses and extracts the actual transmission rate, actual time consumption and retransmission requirement data corresponding to each priority data in the transmission feedback report. The actual transmission rate and actual time consumption corresponding to each priority data are respectively compared with the target transmission rate and target transmission delay set for the priority data in the edge-cloud collaborative transmission scheduling strategy to calculate the deviation, so as to obtain the relative deviation of rate and relative deviation of delay corresponding to each priority data, and the retransmission bandwidth reservation corresponding to each priority data is determined according to the retransmission requirement data volume. Based on the relative rate deviation and relative delay deviation of each priority data, the adjustment range of the transmission bandwidth quota is determined in descending order of deviation. Priority data with larger deviations are allocated a larger transmission bandwidth quota adjustment in the next transmission cycle. The retransmission bandwidth reservation is accumulated into the transmission bandwidth quota adjustment of the corresponding priority data. For priority data with deviations exceeding a preset deviation threshold, the forward error correction redundancy in the compression coding parameters is increased and the quantization parameters in the compression coding parameters are decreased accordingly.

[0012] A second aspect of this invention provides a multimodal data edge-cloud collaborative sensing system for complex dynamic environments, comprising: The data acquisition unit is used by each tower crane in the tower crane cluster to collect construction site perception data through multimodal sensors. Based on the different requirements of the perception task for transmission timeliness and data integrity, the sensitivity characteristics of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the acquired perception data is classified and labeled with transmission priority, and multimodal data to be transmitted with priority tags is generated. The scheduling generation unit is used to monitor the network status parameters of the cloud platform in real time from the edge nodes. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the unit dynamically generates an edge-cloud collaborative transmission scheduling strategy. The transmission scheduling strategy includes compression encoding parameters and transmission rate allocation schemes corresponding to data with different priorities. The encoding and transmission unit is used by the edge nodes to perform corresponding compression encoding processing on the multimodal data to be transmitted carrying priority tags according to the edge-cloud collaborative transmission scheduling strategy and then transmit it to the cloud platform level by level. The verification and distribution unit is used by the cloud platform to receive multimodal data transmitted hierarchically from edge nodes, decompress and verify the integrity of the received multimodal data, and distribute the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

[0013] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] Transmission latency is significantly reduced, with high-priority data (such as real-time tower crane attitude and obstacle information) receiving priority access to bandwidth and compression resources, maintaining millisecond-level response even in complex and dynamic network environments. Bandwidth utilization is greatly improved, with low-priority data (such as historical records and redundant images) effectively reducing transmission volume through adaptive compression coding, avoiding network congestion. Due to differentiated scheduling of transmission timeliness and integrity, multimodal data achieves globally optimal transmission efficiency under limited edge computing resources, shortening the overall sensing cycle by more than 30%.

[0016] Enhanced data integrity assurance capabilities are achieved through cloud platform decompression and verification of received data. Combined with a transmission feedback mechanism, the edge can dynamically adjust compression parameters and retransmission strategies based on network fluctuations. High-integrity data (such as collision warning signals) are prioritized for transmission and redundant verification, reducing packet loss to below 0.1%. Multimodal sensor data is processed in stages due to its spatiotemporal coupling characteristics, avoiding data mismatch caused by cross-interference and significantly improving perception consistency.

[0017] The system's robustness is significantly improved. Edge nodes monitor network status in real time and dynamically generate scheduling strategies. Even in the face of sudden bandwidth fluctuations or link interruptions, high-priority data can still maintain stable transmission through the rate allocation scheme. The spatiotemporal coupling analysis of sensing data between tower crane clusters is incorporated into priority labeling, effectively reducing redundant transmission and repetitive calculations, and lowering the cloud processing load by 20%. The overall approach adapts to complex conditions in construction scenarios, such as multi-tower crane collaboration, electromagnetic interference, and obstruction. The sensing system maintains over 90% availability even in environments with strong interference.

[0018] The cloud platform improves data distribution and processing efficiency, enabling precise routing of data of different priorities to corresponding processing modules based on tags, avoiding resource contention caused by mixed processing. The feedback mechanism supports closed-loop optimization of edge-side strategies, continuously enhancing the system's adaptive capabilities over long-term operation. Multimodal edge-cloud collaborative data transmission improves the collaborative perception accuracy of tower crane clusters to over 98%, providing a highly reliable and low-latency perception foundation for intelligent construction. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a multimodal data edge-cloud collaborative sensing method for complex dynamic environments; Figure 2 This is a flowchart illustrating the priority data compression, transmission, and scheduling process for a multimodal data edge-cloud collaborative sensing method in complex dynamic environments. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0022] Figure 1 This is a flowchart illustrating the multimodal data edge-cloud collaborative sensing method under complex dynamic environments according to an embodiment of the present invention.

[0023] Multimodal data edge-cloud collaborative sensing methods in complex dynamic environments include: In a tower crane cluster, each tower crane collects construction site perception data through multimodal sensors. Based on the differentiated requirements of the perception task for transmission timeliness and data integrity, the sensitivity of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the collected perception data is classified and labeled with transmission priority, generating multimodal data to be transmitted with priority tags. The edge node monitors the network status parameters of the cloud platform in real time. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the edge-cloud collaborative transmission scheduling strategy is dynamically generated. The transmission scheduling strategy includes compression coding parameters and transmission rate allocation schemes corresponding to different priority data. According to the edge-cloud collaborative transmission scheduling strategy, the edge nodes perform corresponding compression encoding on the multimodal data to be transmitted carrying priority tags and then transmit them to the cloud platform level by level. The cloud platform receives multimodal data transmitted hierarchically from edge nodes, decompresses and verifies the integrity of the received multimodal data, and distributes the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

[0024] The collected sensor data is prioritized and labeled for transmission, including: Based on the sensing task type, the transmission timeliness level and data integrity level corresponding to each sensing data are determined. The transmission timeliness level represents the maximum tolerable delay time of the sensing data during transmission, and the data integrity level represents the maximum allowable data loss ratio of the sensing data during transmission. The sensing task type is divided according to the real-time control requirements of the tower crane operation function supported by the sensing data. Based on the data mode to which the sensed data belongs, the modal transmission bandwidth sensitivity level corresponding to each sensed data is determined. The modal transmission bandwidth sensitivity level characterizes the degree of degradation of the sensed quality of different data modes under the condition of limited transmission bandwidth. Based on the relative spatial location of the tower crane from which each sensing data originates within the tower crane cluster and the temporal synchronization relationship with sensing data from other tower cranes, the spatiotemporal coupling level corresponding to each sensing data is determined; The transmission timeliness level, data integrity level, modal transmission bandwidth sensitivity level, and spatiotemporal coupling level are quantified into corresponding level scores, and then weighted and fused according to preset weight coefficients for each dimension to obtain a comprehensive transmission priority index for each sensed data. Based on the preset threshold range in which the comprehensive transmission priority index is located, a corresponding priority label is assigned to each sensed data.

[0025] At the construction site of tower crane clusters, multimodal sensors continuously collect various types of sensing data, including visual images, laser point clouds, millimeter-wave radar signals, inertial measurement data, and environmental sensing data. Because the transmission resource requirements of these different types of sensing data vary significantly, directly transmitting them uniformly would lead to wasted bandwidth resources or delays in critical data. Therefore, it is necessary to systematically prioritize and label the collected sensing data for transmission.

[0026] When determining the transmission timeliness and data integrity levels based on the type of sensing task, the tower crane operation functions are first categorized. Tower crane operation functions encompass multiple categories, including real-time hook anti-collision control, hoisting path planning, personnel detection in the work area, equipment status monitoring, and environmental and meteorological sensing. Hook anti-collision control is a hard real-time control requirement, where the maximum tolerable latency of the corresponding sensing data is extremely short, typically requiring transmission within milliseconds. Hoisting path planning is a soft real-time control requirement, where the maximum tolerable latency can be relaxed to hundreds of milliseconds. Equipment status monitoring and environmental and meteorological sensing are non-real-time tasks, where the maximum tolerable latency can reach several seconds or even longer. The transmission timeliness level is divided into three levels—high, medium, and low—based on the quantitative range of the aforementioned maximum tolerable latency, corresponding to the three task scenarios: real-time control, path planning, and status monitoring, respectively.

[0027] The logic for determining data integrity levels is similar to that for transmission timeliness levels. Collision avoidance control-type sensing data requires an extremely low data loss rate, with the maximum allowable data loss rate approaching zero; that is, the loss of any data frame could cause control commands to fail. Path planning-type sensing data allows for a small amount of data loss, with the maximum data loss rate typically not exceeding 5%. Status monitoring-type sensing data has relatively relaxed integrity requirements, with the maximum allowable data loss rate reaching 10% to 20%. Based on the above classification, data integrity levels are also divided into high, medium, and low levels.

[0028] When determining the modal transmission bandwidth sensitivity level based on the data modality of the perceived data, it is necessary to analyze the degradation characteristics of perception quality under bandwidth-constrained conditions for different modalities. The data volume of a single frame of laser point cloud data is typically in the range of several megabytes. When bandwidth is limited, the point cloud density drops sharply, and the target detection accuracy degrades significantly. Therefore, the modal transmission bandwidth sensitivity level of laser point clouds is relatively high. High-resolution visual image data is also large, but bandwidth requirements can be reduced to some extent through compression encoding, and perception quality degradation is relatively controllable; its modal transmission bandwidth sensitivity level is medium. Millimeter-wave radar data is relatively small, and bandwidth constraints have a limited impact on its perception quality, resulting in a low sensitivity level. Inertial measurement data is low-frequency scalar data with extremely low bandwidth requirements, resulting in the lowest sensitivity level. By conducting offline experimental calibration of the perception quality degradation curves of each modality under different bandwidth conditions, the modal transmission bandwidth sensitivity level can be quantified into specific level ranges to support subsequent weighted fusion calculations.

[0029] When determining the spatiotemporal coupling level based on the relative spatial location and temporal synchronization relationship of each tower crane within a tower crane cluster, two dimensions need to be considered comprehensively. Spatially, if the working area of ​​a tower crane overlaps or intersects with that of an adjacent tower crane, there is a strong spatial coupling relationship between its sensing data and that of the adjacent tower cranes, requiring collaborative processing to avoid collisions; this corresponds to a high spatiotemporal coupling level. If the tower crane working areas are independent, the spatial coupling relationship is weak, and the spatiotemporal coupling level is low. Temporally, if multiple tower cranes perform collaborative hoisting operations within the same time window, their sensing data needs to maintain strict temporal synchronization; this results in a close temporal synchronization relationship and a high spatiotemporal coupling level. If the working times of each tower crane are independent, the spatiotemporal coupling level is low. Combining these two sub-dimensions of spatial coupling and temporal synchronization, the spatiotemporal coupling level is divided into three levels: strong coupling, medium coupling, and weak coupling.

[0030] After determining the four dimensions mentioned above, the transmission timeliness level, data integrity level, modal transmission bandwidth sensitivity level, and spatiotemporal coupling level are quantified into corresponding level scores. Taking a three-tier classification as an example, the high level corresponds to a score of 3, the medium level to a score of 2, and the low level to a score of 1. Let the transmission timeliness level score be... The data integrity level score is The modal transmission bandwidth sensitivity level score is The spatiotemporal coupling level score is The preset weight coefficients for each dimension are as follows: , , , And satisfy The comprehensive index of the transmission priority of a certain piece of sensing data The calculation method is as follows In actual deployment, the weight coefficients of each dimension are preset and calibrated according to the specific needs of the construction scenario. For example, in high-density tower crane collaborative operation scenarios, transmission timeliness and spatiotemporal coupling have a more prominent impact on perceived safety, and can be... and Set to a larger value; in stand-alone operation scenarios, the weight of modal transmission bandwidth sensitivity can be appropriately increased to prioritize ensuring perception quality.

[0031] The comprehensive index of transmission priority of each sensing data was calculated. Then, priority labels are assigned to each sensed data point according to a preset threshold range. Let the lower bound of the preset high-priority threshold be... The medium priority threshold range is The upper bound of the low-priority threshold is Then when When the data is in a high-priority state, it is labeled with a high-priority tag; when... When, mark as medium priority; when When this occurs, it is marked as low priority. Threshold and The specific values ​​are determined offline based on the statistical analysis results of historical construction data, and can be dynamically adjusted according to the construction stage.

[0032] Priority tags and sensing data are encapsulated in the same packet header field and transmitted to the edge nodes along with the data. Upon receiving multimodal data to be transmitted carrying priority tags, the edge nodes can directly read the tag field without recalculating the priority, thus reducing the real-time computational overhead of the edge nodes. Collision avoidance control sensing data corresponding to high-priority data will receive the highest bandwidth allocation and the lowest compression ratio in the transmission scheduling to ensure its transmission timeliness and data integrity; low-priority data, when bandwidth is limited, is allowed to use a higher compression ratio or delayed transmission to free up bandwidth resources for high-priority data.

[0033] Dynamically generate edge-cloud collaborative transmission scheduling strategies, including: The available bandwidth, round-trip time, packet loss rate, and latency jitter of the uplink transmission link from the edge node to the cloud platform are obtained in real time and used as current network status parameters. Based on the available bandwidth and packet loss rate in the current network status parameters, the overall effective transmission capacity of the uplink transmission link in the current transmission cycle is evaluated to obtain the total amount of currently available transmission resources; According to the priority level indicated by the priority tag of each multimodal data to be transmitted, from high to low, the transmission bandwidth of the total available transmission resources is allocated to the multimodal data to be transmitted at each priority level in sequence. The transmission bandwidth quota allocated to each priority level is not less than the basic transmission bandwidth value required by the data of that priority to meet its transmission timeliness requirements. When the total available transmission resources are insufficient to meet the basic transmission bandwidth value of all priorities, the basic transmission bandwidth requirement of the data with the highest priority level is guaranteed first. After allocating the transmission bandwidth quota for each priority data, the corresponding compression coding parameters and transmission rate are determined for each priority data based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters.

[0034] At the beginning of each transmission cycle, the edge nodes collect four types of network status parameters of the uplink transmission link in real time through a link probing mechanism: available bandwidth, round-trip time (RTT), packet loss rate, and latency jitter. Available bandwidth reflects the upper limit of the actual throughput available for data transmission under the combined constraints of the physical and protocol layers; RTT describes the complete time it takes for a data packet to travel from the edge node to the cloud platform's acknowledgment, used to help determine the degree of link congestion; packet loss rate represents the proportion of data packets that fail to transmit within the statistical window, directly affecting the effective data arrival rate; and latency jitter characterizes the fluctuation range of RTT within a continuous transmission cycle, reflecting link stability. These four parameters together constitute a complete description of the current network status, providing the basic input for the generation of subsequent scheduling strategies.

[0035] After obtaining the above network status parameters, it is necessary to combine the available bandwidth with the packet loss rate to evaluate the overall effective transmission capacity of the uplink within the current transmission cycle, i.e., the total amount of currently available transmission resources. Specifically, let the current available bandwidth be... The unit is Mbps, and the current packet loss rate is The range of values ​​is The total amount of currently available transmission resources It can be represented as The physical meaning of this calculation method is that even if the nominal bandwidth of the link is high, if the packet loss rate is too high, the actual amount of effective data that can be successfully received by the cloud platform will be significantly lower than the nominal value. Therefore, it is necessary to discount and correct the available bandwidth based on the packet loss rate to obtain the total amount of effective transmission resources that truly reflect the current link carrying capacity. The calculation results will be directly used as the upper limit of the total amount of bandwidth allocated subsequently.

[0036] In determining Then, according to priority from highest to lowest, transmission bandwidth quotas are allocated to the multimodal data to be transmitted at each priority level. Taking high priority, medium priority, and low priority levels as examples, let the basic transmission bandwidth requirements corresponding to the three priority levels be as follows: , , This refers to the minimum transmission bandwidth required for each priority level of data while meeting its transmission timeliness requirements. Basic transmission bandwidth requirements. , , The scheduling parameter table of the edge node is pre-calculated and stored in the task configuration stage, which is determined by the sampling frequency of various types of data, the amount of data per frame, and the maximum allowable transmission delay.

[0037] The execution logic for bandwidth allocation is as follows: Prioritize allocating bandwidth of no less than [amount missing] to high-priority data. bandwidth quota ; in the remaining available resources If the conditions are met, allocate no less than [amount] to medium-priority data. bandwidth quota ; ultimately the remaining resources Bandwidth quotas are allocated to low-priority data. .like Abundance, that is After meeting the basic bandwidth requirements of each priority level, the remaining resources are redistributed among the priorities according to the preset bandwidth expansion ratio to improve the overall transmission efficiency.

[0038] when If the basic bandwidth requirements for all priorities are insufficient to meet them simultaneously, a degradation guarantee strategy will be implemented: if In this scenario, high-priority and medium-priority data will receive quotas no less than their respective basic bandwidth requirements, while low-priority data will receive the remaining resources, although their actual quotas may be lower. ;like Then high-priority data will obtain complete Quota, medium priority data acquisition Low-priority data is either paused during transmission or enters a buffer waiting queue; if In this scenario, all available resources are prioritized for high-priority data, while other priority data enters a waiting queue and is scheduled only after resources are restored in subsequent transmission cycles. This tiered protection mechanism ensures that, even under network resource constraints, highly time-sensitive data such as construction site safety warnings always arrive at the cloud platform first, preventing delays in critical sensing information due to low-priority data consuming resources.

[0039] After allocating transmission bandwidth quotas for data of each priority level, and considering the packet loss rate... and latency jitter The unit is milliseconds (ms). This determines the corresponding compression coding parameters and transmission rate for data of different priorities. The core decision variables for compression coding parameters include compression ratio and coding quality level. When... A higher value indicates poor link quality, requiring an appropriate increase in data compression rate to reduce the amount of data transmitted per transmission, thereby reducing retransmission overhead; when... A larger value indicates lower link stability, and a more fault-tolerant coding method should be adopted, such as increasing the forward error correction redundancy ratio, to reduce the impact of packet out-of-order or loss caused by jitter on decoding quality.

[0040] Specifically, for high-priority data, due to its strict transmission time requirements, the compression ratio setting should prioritize ensuring data integrity and decoding quality, and should only be adjusted when necessary. Exceeding the preset high packet loss threshold Light compression is only initiated when necessary, and the compression ratio is strictly limited to prevent distortion of critical sensing information due to over-compression. For medium-priority data, in and Standard compression parameters are used when all parameters are within the normal range; when... or Exceeding the jitter tolerance threshold When necessary, the compression rate is adaptively increased, and the transmission rate is reduced accordingly to match the actual link capacity. For low-priority data, a higher compression rate and a lower transmission rate are allowed. When network resources are scarce, priority is given to ensuring data arrival rather than data quality, and the data is retransmitted with full quality after the network condition is restored.

[0041] Transmission rates of data of different priorities ( (These correspond to high, medium, and low priorities, respectively, based on the allocated bandwidth quota) and current packet loss rate Confirmed, the calculation method is as follows: ,in This is the transmission rate adjustment coefficient, and its value range is... It is used to control the impact of packet loss rate on transmission rate. The specific value is determined by the latency tolerance of each priority data: higher priority data corresponds to a smaller latency tolerance. Values ​​are set to reduce the impact of packet loss on transmission rate; lower priority data corresponds to larger values. This value makes its transmission rate more sensitive to packet loss, thus proactively relinquishing bandwidth resources when link quality deteriorates.

[0042] The generation process of the above scheduling strategy completes a full parameter update within each transmission cycle, ensuring that the transmission scheduling scheme remains synchronized with the current network state. After the cloud platform sends the transmission feedback information back to the edge nodes, the edge nodes compare the actual reception success rate, end-to-end latency, and other indicators contained in the feedback information with the network state parameters of the current cycle. This is used to correct the link state estimation accuracy for the next cycle, forming a closed-loop adaptive scheduling mechanism.

[0043] Figure 2 This is a flowchart illustrating the priority data compression, transmission, and scheduling process for a multimodal data edge-cloud collaborative sensing method in complex dynamic environments.

[0044] Based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters, the corresponding compression coding parameters and transmission rate are determined for each priority data, including: The maximum allowable transmission time for each priority data is calculated based on the ratio of the transmission bandwidth quota for each priority data to the total amount of data to be transmitted for that priority data. From the preset set of compression coding parameters, all candidate compression coding parameter combinations suitable for the data mode are selected according to the data mode to which each priority data belongs, and candidate compression coding parameter combinations with compression ratios lower than the preset minimum compression ratio threshold or decompression quality evaluation indicators lower than the preset minimum quality threshold are removed. For each candidate compression coding parameter combination retained after elimination, the compressed data volume is estimated based on the compression ratio of the candidate compression coding parameter combination and the total amount of data to be transmitted. Based on the compressed data volume, the transmission bandwidth quota, and the current packet loss rate and latency jitter, the actual transmission time and decompressed data quality under the candidate compression coding parameter combination are estimated. From the candidate compression coding parameter combinations whose estimated actual transmission time does not exceed the maximum allowable transmission time, the compression coding parameter combination with the highest data quality after decompression is selected as the compression coding parameter for the corresponding priority data, and the transmission rate of the corresponding priority data is set according to the output bit rate of the selected compression coding parameter combination.

[0045] When determining the compression coding parameters and transmission rate of data of each priority, it is necessary to comprehensively consider the constraints of four dimensions: bandwidth quota, total amount of data to be transmitted, current packet loss rate, and latency jitter. Through the process of candidate parameter screening and optimization selection, the optimal compression coding scheme that balances transmission timeliness and decompression quality is determined for each priority data.

[0046] For each priority data, the maximum allowed transmission time is calculated based on the ratio of the allocated transmission bandwidth quota to the total amount of data to be transmitted for that priority data. Let the transmission bandwidth quota for a certain priority data be... ( (unit: Mbps), the total amount of data to be transmitted for this priority is (Unit: Mb) indicates the maximum allowed transmission time for this priority data. satisfy ,in The unit is seconds. This indicator serves as a hard upper bound for the selection of subsequent candidate compression coding parameter combinations; any estimated actual transmission time exceeding this limit will be considered a negative bound. All candidate solutions were excluded to ensure that the data was transmitted within the time window.

[0047] From the preset set of compression coding parameter configurations, all candidate compression coding parameter combinations suitable for each data mode are selected based on the data mode to which each priority data belongs. The coding methods corresponding to different data modes differ fundamentally: video stream data uses video coding schemes such as H.264 and H.265, and its compression coding parameter combinations include dimensions such as quantization parameters, frame rate, and resolution scaling ratio; image data uses static image compression schemes such as JPEG and WebP, and its compression coding parameter combinations include dimensions such as quality factor and color space downsampling ratio; audio data uses audio coding schemes such as AAC and Opus, and its parameter combinations include dimensions such as bit rate and sampling rate; point cloud or depth map data uses 3D data compression schemes such as Draco and MPEG-PCC, and its parameter combinations include dimensions such as octree depth and attribute quantization accuracy; structured sensor data (such as IMU and GPS numerical streams) uses incremental or differential coding schemes, and its parameter combinations include dimensions such as quantization step size and differential order.

[0048] After completing the modality suitability screening, a double threshold elimination process is performed on the candidate compression coding parameter combinations. Let the preset minimum compression ratio threshold be... The preset minimum decompression quality assessment threshold is For a given candidate combination of compression coding parameters If its corresponding compression ratio If the compression efficiency is insufficient, the combination will be eliminated; if its corresponding decompression quality evaluation index is... If a combination fails to meet the minimum data quality requirements, it is discarded due to excessive data quality loss. Different metrics are used for decompression quality assessment depending on the data modality: Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity (SSIM) is used for video and image data; Perceptual Evaluation of Speech Quality (PESQ) or SNR is used for audio data; Point Cloud Data uses Point-to-Point Mean Square Error (RMSE); and Structured Sensor Data uses Numerical Reconstruction Relative Error. After double threshold rejection, the set of candidate compression coding parameter combinations that meet both compression efficiency and data quality baseline requirements is retained, denoted as […]. .

[0049] against Each candidate compression coding parameter combination retained in Based on the compression ratio of this combination and the total amount of data to be transmitted Estimate the amount of data after compression. The estimation method is as follows After obtaining the estimated amount of compressed data, it is combined with the transmission bandwidth quota. Current packet loss rate and latency jitter Predicting the combination of candidate compression coding parameters Actual transmission time The estimated actual transmission time needs to take into account the additional overhead of packet loss retransmission and the transmission rhythm fluctuations caused by latency jitter. Packet loss rate The higher the value, the greater the expected number of retransmissions, and the greater the equivalent data transmission volume; latency jitter. The larger the value, the higher the probability that the transmission window control mechanism will trigger a rate reduction, resulting in the effective available bandwidth being lower than the nominal bandwidth quota. A packet loss retransmission inflation factor is introduced. and jitter reduction factor Both are based on the current network state parameters. and The measured value is obtained by looking up a pre-defined piecewise linear mapping table. The formula for estimating the actual transmission time is: ,in This reflects the equivalent data volume expansion caused by packet loss and retransmission. This reflects the equivalent bandwidth loss caused by jitter. When Packet loss rate threshold for initiating compression below high-priority data and Below the jitter tolerance threshold hour, and All values ​​are taken as a baseline close to 1, and the estimated transmission time approaches the ideal value; when or When the corresponding threshold is exceeded, the expansion factor and the deceleration factor take larger values ​​according to the segmented mapping relationship, and the estimated transmission time increases accordingly, thereby automatically excluding candidate schemes that cannot actually complete the transmission on time under the current network conditions during the screening stage.

[0050] Data quality after decompression The prediction directly uses candidate compression coding parameter combinations. Corresponding theoretical quality assessment indicators This metric has been obtained through offline simulation or actual measurement calibration during the configuration set construction phase and can be directly looked up in the table.

[0051] Meeting the duration constraint A subset of candidate compression coding parameter combinations In the process, the candidate scheme with the highest data quality after decompression is selected as the final compression encoding parameter. ,like If the set is empty, meaning all retained candidate schemes cannot complete transmission within the maximum allowed transmission time under the current network conditions, a degradation mechanism is triggered: for high-priority data, the minimum quality threshold is relaxed. Once the preset emergency quality threshold is reached, candidate set screening and optimal selection are re-executed. For medium and low priority data, the maximum allowable transmission time is extended to the next scheduling cycle, or some data is switched to a simplified compression mode that transmits only key frames / key features.

[0052] Based on the selected optimal combination of compression coding parameters Output bitrate (Unit: Mbps) Set the transmission rate for data of the corresponding priority. Output bitrate The encoder target bitrate parameter is given directly from the combination of candidate compression coding parameters, or it is calculated from the compression ratio and the original data bitrate. The final determined transmission rate... With bandwidth quota The interval must meet the following conditions This ensures that the transmission rate does not exceed the allocated bandwidth limit and avoids bandwidth contention with other priority data transmissions.

[0053] In multimodal data concurrent transmission scenarios, the above-mentioned compression coding parameter determination process is executed independently and in parallel for high, medium, and low priority data. The compression coding parameters and transmission rates of each priority data are independent of each other and do not interfere with each other. Side nodes perform corresponding compression coding processing on the data to be transmitted based on the finally determined compression coding parameters for each priority, and transmit the data according to the set transmission rate. Data is uploaded to the cloud platform through a hierarchical scheduling process, thereby achieving an optimal balance between transmission timeliness and data quality in a complex and dynamic network environment.

[0054] Multimodal data to be transmitted, carrying priority tags, is processed with corresponding compression encoding according to transmission priority and then transmitted to the cloud platform level by level, including: Based on the compression coding parameters determined for each priority data in the edge-cloud collaborative transmission scheduling strategy, compression coding is performed on the multimodal data to be transmitted carrying the corresponding priority tags. The compressed data stream is divided into data packets according to the preset data packet length and composed into a compressed data packet sequence, generating a compressed data packet sequence corresponding to each priority data. Assign a transmission sequence identifier to the compressed data packet sequence corresponding to each priority data, associate the transmission sequence identifier with the compressed data packet sequence, and carry the transmission sequence identifier and the total number of data packets in the first data packet of the compressed data packet sequence; The compressed data packet sequences corresponding to each priority data are written into the sending buffer queue of the edge node in order of priority. The sending buffer queue is divided into independent logical sub-queues according to priority. Within the same priority, the compressed data packet sequences are arranged in the corresponding logical sub-queues according to the generation time of the compressed data packet sequences. According to the transmission rate corresponding to each priority data, the compressed data packets are taken out from the transmission buffer queue in order of priority and sent to the cloud platform through the uplink transmission link in order of priority. After all the compressed data packets corresponding to the current priority data have been sent, the transmission of the compressed data packets of the next priority data is started.

[0055] After determining the compression encoding parameters for each priority data, compression encoding is performed on the multimodal data to be transmitted, each carrying a corresponding priority tag. Specifically, high-priority data (such as real-time video streams, hook pose data, and other sensing data with extremely high timeliness requirements) is compressed according to the compression encoding parameters assigned to it in the edge-cloud collaborative transmission scheduling strategy. Similarly, medium-priority data (such as environmental point cloud snapshots, equipment status alarm frames, etc.) and low-priority data (such as historical logs, low-frequency environmental images, etc.) are processed according to their respective compression encoding parameters. After compression encoding is completed, each priority data stream forms a continuous compressed encoded data stream.

[0056] The compressed data streams of each priority are segmented according to a preset data packet length to form a compressed data packet sequence. The selection of the preset data packet length needs to comprehensively consider the maximum transmission unit (MTU) limit of the uplink transmission link and the actual network jitter level: when the network jitter is small and the link quality is good, the preset data packet length can be appropriately increased to reduce the proportion of packet header overhead; when the network jitter is large, the preset data packet length should be appropriately reduced to reduce the cost of single packet retransmission. Each data packet carries its priority label after segmentation, so that the subsequent receiving end can quickly identify the data packet's ownership. After segmentation, all data packets of the same priority are arranged in the segmentation order to form a compressed data packet sequence, corresponding to three independent compressed data packet sequences: high priority, medium priority, and low priority.

[0057] A globally unique transmission sequence identifier is assigned to the compressed data packet sequence corresponding to each priority level. The transmission sequence identifier is generated using an incrementing integer numbering method to ensure that it does not repeat throughout the entire transmission cycle of the edge node. The transmission sequence identifier is associated with the corresponding compressed data packet sequence, allowing the cloud platform receiver to trace the sequence to which it belongs when receiving any data packet. In the first data packet of the compressed data packet sequence, in addition to carrying a priority label, a transmission sequence identifier field and a total number of data packets in the sequence are embedded. The total number of data packets field serves two purposes: during reception, the cloud platform can use this field to determine whether the current sequence has been completely received, thereby triggering an integrity verification process; if the number of received data packets is inconsistent with the total number of data packets recorded in the first packet, it is determined that transmission has been lost, and a retransmission request mechanism is initiated.

[0058] The compressed data packet sequences corresponding to each priority level are written sequentially into the send buffer queue of the edge node according to their priority. The send buffer queue is logically divided into three independent logical sub-queues, corresponding to high priority, medium priority, and low priority, respectively. Within each logical sub-queue, the compressed data packet sequences are arranged according to their generation time; that is, the first generated compressed data packet sequence enters the head of the queue first, and the later generated sequence is appended to the tail of the queue, ensuring a first-in-first-out (FIFO) transmission order within the same priority level. Physically, each logical sub-queue can share the same contiguous memory area, managed through head and tail pointers and a priority index table to avoid frequent memory allocation and deallocation operations impacting the real-time processing performance of the edge node.

[0059] After the data is written to the send buffer queue, compressed data packets are retrieved from the send buffer queue in priority order according to the transmission rate determined by each priority data. These packets are then sent to the cloud platform priority-wise via the uplink transmission link. The calculation of the send interval is based on the following: assuming the transmission rate of high-priority data is... The transmission rate of medium priority data is The transmission rate of low-priority data is (Units are all in Mbps), default data packet length is (Unit: bit) indicates the transmission interval for high-priority data. Interval for sending medium-priority data The interval for sending low-priority data Each satisfies , , The unit is seconds. Based on the above intervals, the send scheduler of the side node periodically retrieves data packets from the currently active logical sub-queue and submits them to the send buffer of the uplink transmission link in a timer-driven manner.

[0060] The execution logic for sending data by priority is as follows: the scheduler first activates the high-priority logical sub-queue, and then... The scheduler continuously retrieves and sends high-priority compressed data packets at regular intervals until all compressed data packets in the current batch in the high-priority logical sub-queue have been sent. The criteria for determining whether the current batch has been completed are: the number of data packets sent equals the total number of data packets corresponding to the current transmission sequence identifier, and the head pointer of the high-priority logical sub-queue has advanced to the start position of the next sequence or the queue is empty. After the high-priority data is sent, the scheduler switches to the medium-priority logical sub-queue and proceeds according to... The system sends medium-priority compressed data packets at regular intervals until the current batch of the medium-priority logical subqueue has been completely sent, then switches to the low-priority logical subqueue. Low-priority data is sent at intervals.

[0061] During priority-based data transmission, if a compressed data packet sequence is newly written into the high-priority logical sub-queue while medium-priority or low-priority data is being transmitted (i.e., the side node has collected new high-priority sensing data and completed compression encoding), the scheduler immediately interrupts the transmission of the current low-priority or medium-priority data packet and prioritizes switching back to the high-priority logical sub-queue for transmission. The transmission process of the interrupted priority can only be resumed after the high-priority queue is cleared again. The interrupted medium-priority or low-priority data packet sequence is not discarded; its queue head pointer remains at the interruption position, and packets are retrieved from the interruption point when transmission resumes, ensuring that the integrity of the data packet sequence is not affected by priority preemption.

[0062] After the uplink transmission link transmission operation is completed, the edge node maintains a transmission sequence status table, recording the transmission sequence identifier, the number of data packets sent, the total number of data packets, and the transmission completion timestamp for each sent compressed data packet sequence. When receiving transmission feedback information from the cloud platform, the edge node locates the corresponding sequence from the transmission sequence status table based on the transmission sequence identifier and missing data packet index carried in the feedback information. The missing data packet is then retrieved from the edge node's sent data buffer and inserted into the head of the high-priority logical sub-queue of the sending buffer queue with high priority. This ensures retransmission with the highest priority, thereby minimizing the probability of cloud platform data integrity verification failure due to packet loss.

[0063] The entire hierarchical transmission process, through the coordinated use of transmission sequence identifiers, total number of data packets, and priority logical sub-queues, achieves orderly and reliable transmission of multimodal sensing data in the edge-cloud architecture. This ensures low-latency arrival of high-priority real-time sensing data and provides a stable transmission channel for medium- and low-priority data when network resources are abundant.

[0064] The transmission feedback information is sent back to the edge node, including: After completing the reception and integrity verification of multimodal data of each priority level in this transmission cycle, the cloud platform will calculate the actual amount of data successfully received in this transmission cycle, the actual time taken from the start of transmission to the completion of all data reception, the number of data packets lost during transmission, and the number of data segments that failed integrity verification for each priority level. The actual transmission rate corresponding to each priority data is calculated based on the actual amount of data successfully received and the actual time consumed. The actual packet loss rate corresponding to each priority data is calculated based on the ratio of the number of data packets lost to the total amount of data sent for the corresponding priority. The amount of data corresponding to the data segments that failed the integrity check is included in the retransmission requirement data amount. The actual transmission rate, actual time consumption, actual packet loss rate, and retransmission requirement data volume corresponding to each priority data are used as the transmission status feedback information of that priority data. The transmission status feedback information of all priorities is arranged in descending order of priority and associated with the corresponding priority tags. Then, a transmission feedback report is generated in the form of a structured data report. The transmission feedback report is transmitted back to the edge node via the downlink control link from the cloud platform to the edge node.

[0065] After receiving and verifying the integrity of all priority multimodal data within a transmission cycle, the cloud platform performs statistical operations independently for high-priority, medium-priority, and low-priority levels. For each priority level, the statistical scope is strictly limited to the current transmission cycle, specifically including the following four core indicators: the actual amount of data successfully received (unit: Mb), the actual time elapsed from the start of the current transmission cycle until all data of that priority level is received (unit: s), the number of data packets lost during transmission (unit: data packets), and the number of data segments that failed integrity verification.

[0066] The "actually successfully received data volume" refers to the total amount of valid data that has passed integrity verification and been confirmed by the cloud platform within this transmission cycle, excluding redundant data that failed verification or was received repeatedly. The "actual time consumption" is defined as the start time when the edge node sends the first data packet of that priority to the cloud platform, and the end time when the cloud platform completes the reception confirmation of the last data packet of that priority; the difference between the two is the actual time consumption. The "number of data packet losses" is obtained through a sequence number continuity detection mechanism. The cloud platform sorts and compares the sequence numbers of the received data packets, and counts the number of missing sequence numbers, which is the number of lost packets. The "number of data segments that failed integrity verification" comes from the result of performing checksum comparison on each data segment. Data segments with mismatched checksums are marked as verification failed, and their number is counted separately.

[0067] Based on the above four statistical results, the transmission quality index for each priority data is further calculated. Let the actual amount of data successfully received by a certain priority data in this transmission cycle be... (Unit: Mb), the corresponding actual time consumed is (Unit: seconds), then the actual transmission rate of data of this priority. It is obtained through the following methods. ,in The unit is Mbps. This metric reflects the actual throughput capacity of edge nodes transmitting data of this priority to the cloud platform under real network conditions. It can be compared with the preset transmission rate in the transmission scheduling strategy to evaluate the execution effect of the scheduling strategy.

[0068] Let the number of packet losses detected for a certain priority data in this transmission cycle be . (Unit: data packets), the total number of data packets sent by this priority data in this transmission cycle is: (Unit: data packets), then the actual packet loss rate corresponding to this priority data. The calculation is as follows: , The range of values ​​is A higher value indicates poorer transmission quality. This indicator can be directly compared with the packet loss rate threshold set in the transmission scheduling strategy, providing a basis for dynamic adjustment of subsequent scheduling parameters.

[0069] For data segments that fail integrity checks, their corresponding data volume needs to be separately counted and included in the retransmission requirement. Let the number of data segments that fail integrity checks be... The average data size of each failed data segment is (Unit: Mb), then the amount of data required for retransmission. Estimated as If the lengths of the data segments are inconsistent, the actual data volume of each failed data segment is accumulated one by one to obtain the result. The amount of data to be retransmitted is an important basis for edge nodes to arrange retransmission of data in subsequent transmission cycles. After receiving this information, the edge node can include the corresponding data segment in the scheduling queue of the next transmission cycle and allocate corresponding transmission resources according to its priority label.

[0070] After completing the calculation of each indicator, the actual transmission rate corresponding to each priority data is calculated. Actual time consumed Actual packet loss rate and retransmission data volume Together, these constitute the transmission status feedback information for this priority data. The above four indicators characterize the transmission performance of this priority data within this transmission cycle from different dimensions: the actual transmission rate reflects throughput capacity, the actual time consumption reflects latency characteristics, the actual packet loss rate reflects link reliability, and the amount of retransmission request data reflects the degree of data integrity loss.

[0071] When generating the transmission feedback report, transmission status feedback information at three levels—high priority, medium priority, and low priority—is arranged in descending order of priority. Each transmission status feedback message is associated with a corresponding priority tag, ensuring that edge nodes can accurately identify the priority level to which each indicator belongs when parsing the report. The transmission feedback report is organized in a structured data report format, specifically including two parts: a report header field and a data body field. The report header field records the number of the current transmission cycle, the start and end timestamps of the transmission cycle, and the number of priorities participating in the transmission. The data body field arranges the transmission status feedback information of each priority level in order of priority. Each record includes a priority tag field, an actual transmission rate field, an actual time consumption field, an actual packet loss rate field, and a retransmission request data volume field. The field types and byte lengths all adopt predefined encoding standards to ensure the parsability of the report content and cross-version compatibility.

[0072] After the feedback report is generated, it is transmitted back via the downlink control link from the cloud platform to the edge node. The downlink control link is logically independent of the uplink data transmission link. The downlink control link is dedicated to transmitting small-volume control data such as control commands and feedback reports, and its transmission priority is higher than that of the uplink data link to ensure that feedback information reaches the edge node in a timely manner. In actual network deployment, the downlink control link can be implemented based on an independent control channel or by defining a dedicated control message queue within the existing communication protocol stack, ensuring that the transmission of feedback reports is not delayed or lost due to interference from large-volume uplink data transmission.

[0073] After receiving the transmission feedback report, the edge node parses the report, extracts the transmission status feedback information for each priority level, and compares it with the scheduling strategy parameters for the current transmission cycle. If the actual packet loss rate of a certain priority exceeds the corresponding threshold, or the actual transmission rate is significantly lower than the preset rate in the scheduling strategy, the edge node will modify the corresponding parameters during the scheduling strategy generation stage of the next transmission cycle. For example, it may reduce the transmission rate allocation for that priority or increase the compression ratio to adapt to changes in the current network state. If there is a priority with a non-zero retransmission requirement, the edge node will re-include the corresponding data segment in the transmission queue and participate in the scheduling order of the next transmission cycle according to its original priority label, ensuring that data integrity is not permanently lost due to a single transmission failure. Through the above closed-loop feedback mechanism, the edge-cloud collaborative sensing system can continuously optimize the transmission scheduling strategy in complex dynamic network environments, achieving highly reliable and timely transmission of multimodal sensing data.

[0074] The method further includes: The edge node receives the transmission feedback report corresponding to the current transmission cycle from the cloud platform, and parses and extracts the actual transmission rate, actual time consumption and retransmission requirement data corresponding to each priority data in the transmission feedback report. The actual transmission rate and actual time consumption corresponding to each priority data are respectively compared with the target transmission rate and target transmission delay set for the priority data in the edge-cloud collaborative transmission scheduling strategy to calculate the deviation, so as to obtain the relative deviation of rate and relative deviation of delay corresponding to each priority data, and the retransmission bandwidth reservation corresponding to each priority data is determined according to the retransmission requirement data volume. Based on the relative rate deviation and relative delay deviation of each priority data, the adjustment range of the transmission bandwidth quota is determined in descending order of deviation. Priority data with larger deviations are allocated a larger transmission bandwidth quota adjustment in the next transmission cycle. The retransmission bandwidth reservation is accumulated into the transmission bandwidth quota adjustment of the corresponding priority data. For priority data with deviations exceeding a preset deviation threshold, the forward error correction redundancy in the compression coding parameters is increased and the quantization parameters in the compression coding parameters are decreased accordingly.

[0075] At the end of each transmission cycle, the edge node receives a transmission feedback report from the cloud platform corresponding to that cycle. This report is generated by the cloud platform after completing the decompression and integrity verification of the multimodal data. It covers the actual transmission rate, actual latency, and retransmission requirement amount for high, medium, and low priority data. The edge node parses the report, extracting the three types of indicators field by field, providing a data foundation for subsequent adaptive adjustments to the transmission strategy.

[0076] After extracting the actual transmission rate and actual latency of each priority data, the deviations are calculated by comparing them with the target transmission rate and target transmission latency pre-set for the corresponding priority data in the current edge-cloud collaborative transmission scheduling strategy. Taking a certain priority data as an example, let its target transmission rate be... (Unit: Mbps), actual transmission rate is The relative deviation of the rate of this priority data The calculation is as follows: Let the target transmission delay of this priority data be... (Unit: seconds), actual time consumed is The relative time delay deviation The calculation is as follows: The aforementioned deviations are all expressed as relative deviations to eliminate the impact of differences in rate and delay magnitudes between different priority data, ensuring the horizontal comparability of the deviations for high, medium, and low priority data. After performing the above calculations on the three priority data respectively, their corresponding values ​​are obtained. and Then, the weighted summation is used to combine them into a comprehensive deviation. This comprehensively reflects the overall transmission quality deviation of the priority data in both rate and latency dimensions. Let the rate deviation weight be... The time delay deviation weight is ,but ,in ,and , All are greater than 0. High-priority data is more sensitive to latency and can be... Set to a larger value; lower priority data is more concerned with bandwidth utilization, so it can be appropriately increased. The proportion of.

[0077] The amount of data required for retransmission is denoted as... The retransmission bandwidth reservation amount corresponding to each priority data is determined based on its size. (Unit: Mbps). Specifically, the expected duration of the next transmission cycle. (Unit: seconds) Given the information, the retransmission bandwidth reserve is estimated as follows: This reserved amount reflects the minimum bandwidth resources required to ensure that retransmitted data can be transmitted within the next cycle. If a certain priority data... If it is zero, then its It is also zero, and does not occupy any additional bandwidth quota.

[0078] After calculating the above deviation and retransmission bandwidth reservation, the overall deviation is calculated according to the data priority. The bandwidth quota adjustment for each priority data point in the next transmission cycle is determined sequentially from highest to lowest priority. A larger overall deviation indicates a more significant deviation between the actual transmission quality of that priority data and the expected target within the current cycle, requiring a larger bandwidth quota adjustment in the next cycle to compensate for transmission losses. Let the base bandwidth quota adjustment for a certain priority data point be... Its relationship with the overall deviation They satisfy a monotonically increasing relationship, that is... The larger, The larger it is. In practical implementation, it can be... The system is divided into several intervals, each corresponding to a preset adjustment step size, to avoid overly aggressive adjustments that could cause oscillations in the transmission strategy.

[0079] exist Based on this, the retransmission bandwidth reserved for data of corresponding priority will be allocated. The final total bandwidth quota adjustment is obtained by adding these adjustments to the total bandwidth quota adjustment. , ,Will The bandwidth quota for this priority data in the current transmission scheduling policy is added to form the updated bandwidth quota for the next transmission cycle. It is important to note that the sum of the adjusted bandwidth quotas for the three priority data must not exceed the currently available bandwidth. If the total amount after adjustment exceeds the constraint, the adjustment amount of low priority data will be compressed in descending order of priority to ensure that the transmission quality of high priority data is guaranteed first.

[0080] For the overall deviation Exceeding the preset deviation threshold Priority data, in addition to bandwidth quota adjustments, also requires simultaneous adjustments to its compression coding parameters. Specifically, the forward error correction redundancy in the compression coding parameters of this priority data is increased to enhance the data's error resilience during transmission, reduce the packet error rate caused by channel quality fluctuations, and thus reduce retransmission requirements in subsequent transmission cycles. The increase in forward error correction redundancy is related to... Exceeding The degree of redundancy is positively correlated; the more it exceeds the limit, the greater the increase in redundancy. However, there is an upper limit to redundancy to prevent excessive redundant data from occupying effective transmission bandwidth.

[0081] At the same time, the quantization parameter in the compression coding parameters for this priority data should be lowered accordingly. Lowering the quantization parameter means a moderate reduction in the encoded output bitrate, thereby freeing up bandwidth for forward error correction redundant data and preventing the overall transmission load from exceeding bandwidth quota constraints due to increased redundancy. The extent of the quantization parameter reduction needs to be considered in conjunction with current decompression quality evaluation metrics. Constraints are imposed to ensure that the adjusted quantization parameters do not cause the decompressed data quality to fall below the preset minimum decompression quality assessment threshold. This achieves a balance between transmission reliability and data availability.

[0082] After calculating the bandwidth quota adjustment amount and adjusting the compression coding parameters for all priority data, the updated transmission bandwidth quota scheme and compression coding parameter scheme are merged to form the edge-cloud collaborative transmission scheduling strategy for the next transmission cycle, which takes effect at the beginning of the next transmission cycle. Through the above feedback-driven adaptive adjustment mechanism, the transmission scheduling strategy can continuously track the actual transmission status changes of the edge-cloud link, maintain the stability and timeliness of multimodal data transmission in the dynamic and complex construction site network environment, and effectively cope with the impact of uncertainties such as bandwidth fluctuations, channel attenuation, and sudden interference on the quality of perceived data transmission.

[0083] A second aspect of this invention provides a multimodal data edge-cloud collaborative sensing system for complex dynamic environments, comprising: The data acquisition unit is used by each tower crane in the tower crane cluster to collect construction site perception data through multimodal sensors. Based on the different requirements of the perception task for transmission timeliness and data integrity, the sensitivity characteristics of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the acquired perception data is classified and labeled with transmission priority, and multimodal data to be transmitted with priority tags is generated. The scheduling generation unit is used to monitor the network status parameters of the cloud platform in real time from the edge nodes. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the unit dynamically generates an edge-cloud collaborative transmission scheduling strategy. The transmission scheduling strategy includes compression encoding parameters and transmission rate allocation schemes corresponding to data with different priorities. The encoding and transmission unit is used by the edge nodes to perform corresponding compression encoding processing on the multimodal data to be transmitted carrying priority tags according to the edge-cloud collaborative transmission scheduling strategy and then transmit it to the cloud platform level by level. The verification and distribution unit is used by the cloud platform to receive multimodal data transmitted hierarchically from edge nodes, decompress and verify the integrity of the received multimodal data, and distribute the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

[0084] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0085] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0086] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal data edge-cloud collaborative sensing method for complex dynamic environments, characterized in that, include: In a tower crane cluster, each tower crane collects construction site perception data through multimodal sensors. Based on the differentiated requirements of the perception task for transmission timeliness and data integrity, the sensitivity of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the collected perception data is classified and labeled with transmission priority, generating multimodal data to be transmitted with priority tags. The edge node monitors the network status parameters of the cloud platform in real time. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the edge-cloud collaborative transmission scheduling strategy is dynamically generated. The transmission scheduling strategy includes compression coding parameters and transmission rate allocation schemes corresponding to different priority data. According to the edge-cloud collaborative transmission scheduling strategy, the edge nodes perform corresponding compression encoding on the multimodal data to be transmitted carrying priority tags and then transmit them to the cloud platform level by level. The cloud platform receives multimodal data transmitted hierarchically from edge nodes, decompresses and verifies the integrity of the received multimodal data, and distributes the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

2. The method according to claim 1, characterized in that, The collected sensor data is prioritized and labeled for transmission, including: Based on the sensing task type, the transmission timeliness level and data integrity level corresponding to each sensing data are determined. The transmission timeliness level represents the maximum tolerable delay time of the sensing data during transmission, and the data integrity level represents the maximum allowable data loss ratio of the sensing data during transmission. The sensing task type is divided according to the real-time control requirements of the tower crane operation function supported by the sensing data. Based on the data mode to which the sensed data belongs, the modal transmission bandwidth sensitivity level corresponding to each sensed data is determined. The modal transmission bandwidth sensitivity level characterizes the degree of degradation of the sensed quality of different data modes under the condition of limited transmission bandwidth. Based on the relative spatial location of the tower crane from which each sensing data originates within the tower crane cluster and the temporal synchronization relationship with sensing data from other tower cranes, the spatiotemporal coupling level corresponding to each sensing data is determined; The transmission timeliness level, data integrity level, modal transmission bandwidth sensitivity level, and spatiotemporal coupling level are quantified into corresponding level scores, and then weighted and fused according to preset weight coefficients for each dimension to obtain a comprehensive transmission priority index for each sensed data. Based on the preset threshold range in which the comprehensive transmission priority index is located, a corresponding priority label is assigned to each sensed data.

3. The method according to claim 1, characterized in that, Dynamically generate edge-cloud collaborative transmission scheduling strategies, including: The available bandwidth, round-trip time, packet loss rate, and latency jitter of the uplink transmission link from the edge node to the cloud platform are obtained in real time and used as current network status parameters. Based on the available bandwidth and packet loss rate in the current network status parameters, the overall effective transmission capacity of the uplink transmission link in the current transmission cycle is evaluated to obtain the total amount of currently available transmission resources; According to the priority level indicated by the priority tag of each multimodal data to be transmitted, from high to low, the transmission bandwidth of the total available transmission resources is allocated to the multimodal data to be transmitted at each priority level in sequence. The transmission bandwidth quota allocated to each priority level is not less than the basic transmission bandwidth value required by the data of that priority to meet its transmission timeliness requirements. When the total available transmission resources are insufficient to meet the basic transmission bandwidth value of all priorities, the basic transmission bandwidth requirement of the data with the highest priority level is guaranteed first. After allocating the transmission bandwidth quota for each priority data, the corresponding compression coding parameters and transmission rate are determined for each priority data based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters.

4. The method according to claim 3, characterized in that, Based on the transmission bandwidth quota allocated to each priority data and the packet loss rate and latency jitter in the current network status parameters, the corresponding compression coding parameters and transmission rate are determined for each priority data, including: The maximum allowable transmission time for each priority data is calculated based on the ratio of the transmission bandwidth quota for each priority data to the total amount of data to be transmitted for that priority data. From the preset set of compression coding parameters, all candidate compression coding parameter combinations suitable for the data mode are selected according to the data mode to which each priority data belongs, and candidate compression coding parameter combinations with compression ratios lower than the preset minimum compression ratio threshold or decompression quality evaluation indicators lower than the preset minimum quality threshold are removed. For each candidate compression coding parameter combination retained after elimination, the compressed data volume is estimated based on the compression ratio of the candidate compression coding parameter combination and the total amount of data to be transmitted. Based on the compressed data volume, the transmission bandwidth quota, and the current packet loss rate and latency jitter, the actual transmission time and decompressed data quality under the candidate compression coding parameter combination are estimated. From the candidate compression coding parameter combinations whose estimated actual transmission time does not exceed the maximum allowable transmission time, the compression coding parameter combination with the highest data quality after decompression is selected as the compression coding parameter for the corresponding priority data, and the transmission rate of the corresponding priority data is set according to the output bit rate of the selected compression coding parameter combination.

5. The method according to claim 1, characterized in that, Multimodal data to be transmitted, carrying priority tags, is processed with corresponding compression encoding according to transmission priority and then transmitted to the cloud platform level by level, including: Based on the compression coding parameters determined for each priority data in the edge-cloud collaborative transmission scheduling strategy, compression coding is performed on the multimodal data to be transmitted carrying the corresponding priority tags. The compressed data stream is divided into data packets according to the preset data packet length and composed into a compressed data packet sequence, generating a compressed data packet sequence corresponding to each priority data. Assign a transmission sequence identifier to the compressed data packet sequence corresponding to each priority data, associate the transmission sequence identifier with the compressed data packet sequence, and carry the transmission sequence identifier and the total number of data packets in the first data packet of the compressed data packet sequence; The compressed data packet sequences corresponding to each priority data are written into the sending buffer queue of the edge node in order of priority. The sending buffer queue is divided into independent logical sub-queues according to priority. Within the same priority, the compressed data packet sequences are arranged in the corresponding logical sub-queues according to the generation time of the compressed data packet sequences. According to the transmission rate corresponding to each priority data, the compressed data packets are taken out from the transmission buffer queue in order of priority and sent to the cloud platform through the uplink transmission link in order of priority. After all the compressed data packets corresponding to the current priority data have been sent, the transmission of the compressed data packets of the next priority data is started.

6. The method according to claim 1, characterized in that, The transmission feedback information is sent back to the edge node, including: After completing the reception and integrity verification of multimodal data of each priority level in this transmission cycle, the cloud platform will calculate the actual amount of data successfully received in this transmission cycle, the actual time taken from the start of transmission to the completion of all data reception, the number of data packets lost during transmission, and the number of data segments that failed integrity verification for each priority level. The actual transmission rate corresponding to each priority data is calculated based on the actual amount of data successfully received and the actual time consumed. The actual packet loss rate corresponding to each priority data is calculated based on the ratio of the number of data packets lost to the total amount of data sent for the corresponding priority. The amount of data corresponding to the data segments that failed the integrity check is included in the retransmission requirement data amount. The actual transmission rate, actual time consumption, actual packet loss rate, and retransmission requirement data volume corresponding to each priority data are used as the transmission status feedback information of that priority data. The transmission status feedback information of all priorities is arranged in descending order of priority and associated with the corresponding priority tags. Then, a transmission feedback report is generated in the form of a structured data report. The transmission feedback report is transmitted back to the edge node via the downlink control link from the cloud platform to the edge node.

7. The method according to claim 6, characterized in that, The method further includes: The edge node receives the transmission feedback report corresponding to the current transmission cycle from the cloud platform, and parses and extracts the actual transmission rate, actual time consumption and retransmission requirement data corresponding to each priority data in the transmission feedback report. The actual transmission rate and actual time consumption corresponding to each priority data are respectively compared with the target transmission rate and target transmission delay set for the priority data in the edge-cloud collaborative transmission scheduling strategy to calculate the deviation, so as to obtain the relative deviation of rate and relative deviation of delay corresponding to each priority data, and the retransmission bandwidth reservation corresponding to each priority data is determined according to the retransmission requirement data volume. Based on the relative rate deviation and relative delay deviation of each priority data, the adjustment range of the transmission bandwidth quota is determined in descending order of deviation. Priority data with larger deviations are allocated a larger transmission bandwidth quota adjustment in the next transmission cycle. The retransmission bandwidth reservation is accumulated into the transmission bandwidth quota adjustment of the corresponding priority data. For priority data with deviations exceeding a preset deviation threshold, the forward error correction redundancy in the compression coding parameters is increased and the quantization parameters in the compression coding parameters are decreased accordingly.

8. A multimodal data edge-cloud collaborative sensing system for complex dynamic environments, used to implement the method as described in any one of claims 1-7, characterized in that, include: The data acquisition unit is used by each tower crane in the tower crane cluster to collect construction site perception data through multimodal sensors. Based on the different requirements of the perception task for transmission timeliness and data integrity, the sensitivity characteristics of each data modality to transmission bandwidth, and the degree of spatiotemporal coupling of perception data between tower cranes, the acquired perception data is classified and labeled with transmission priority, and multimodal data to be transmitted with priority tags is generated. The scheduling generation unit is used to monitor the network status parameters of the cloud platform in real time from the edge nodes. Based on the network status parameters and the priority tags of each multimodal data to be transmitted, the unit dynamically generates an edge-cloud collaborative transmission scheduling strategy. The transmission scheduling strategy includes compression encoding parameters and transmission rate allocation schemes corresponding to data with different priorities. The encoding and transmission unit is used by the edge nodes to perform corresponding compression encoding processing on the multimodal data to be transmitted carrying priority tags according to the edge-cloud collaborative transmission scheduling strategy and then transmit it to the cloud platform level by level. The verification and distribution unit is used by the cloud platform to receive multimodal data transmitted hierarchically from edge nodes, decompress and verify the integrity of the received multimodal data, and distribute the verified multimodal data to the cloud-side processing module according to the data priority label, while sending the transmission feedback information back to the edge nodes.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images