Data transmission method for tower group unmanned operation system based on cloud-edge collaboration

Abstract

An embodiment of the present application provides a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, the method comprising: performing a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into a shared feature extraction layer of a preset initial model, determining the corresponding sensor features and video features, inputting the sensor features into a preset priority branch layer, determining the corresponding high-priority sensor data, inputting the video features into a preset image processing branch layer, determining the corresponding key video image information, inputting the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determining the corresponding tower crane data transmission model; receiving real-time sensor data and video data of the tower crane, inputting the real-time sensor data and video data into the tower crane data transmission model, determining the optimal data transmission method, performing data transmission according to the optimal data transmission method, and realizing real-time tower crane control. The present application can improve the control efficiency of the tower crane.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration. Background Art

[0002] As special operation equipment, hoisting machinery has frequent accidents. If it is not managed and operated properly, it is easy to cause casualties and property losses. According to the statistics of the notice of the State Administration for Market Regulation on the safety status of special equipment nationwide in 2023, the causes of accidents are mainly divided into two categories: one is accidents caused by improper use and management, accounting for about 84.44%. Illegal operation is still the main cause of accidents, which is specifically manifested in illegal operation by operators, improper operation or even unlicensed operation, lack of maintenance, poor management, etc. Second, accidents caused by failure of safety components and failure of protective devices due to equipment defects and inadequate maintenance account for about 15.56%.

[0003] During the operation of lifting machinery, especially tower cranes, the driver in the cab needs to pay attention to the relative positions of the hook, the load, the tower arm, other tower cranes, buildings, and obstacles to avoid collisions. However, when the driver is sitting in the cab, his vision will be blocked, and the angles of view he needs to focus on are not in the same direction. Therefore, other ground staff are often required to cooperate and give specific operating instructions to complete the delicate tower crane movements. The existing solution is to transmit data from the tower crane to the ground end, using a direct cable connection solution. However, since the tower crane has multiple degrees of freedom, there is a risk of cable winding. When controlling the tower crane, the safety of the cable must be considered, resulting in low efficiency in tower crane control.

[0004] Due to the high risk of tower crane accidents and the inefficiency of the measures taken to avoid risks, this application proposes a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, which can effectively improve the control efficiency of tower cranes and reduce the probability of tower crane accidents. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, which can improve the control efficiency of tower cranes.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, comprising:

[0008] Receive original sensor data and video data of the tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback the three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0009] Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0010] Receive real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission mode, perform data transmission according to the optimal data transmission mode, and realize real-time tower crane control.

[0011] Furthermore, before receiving the original sensor data and video data of the tower crane, the method further comprises:

[0012] Install ultrasonic distance measuring sensors on the front end and rear end of the tower crane boom and the hook to determine the corresponding ultrasonic sensor data;

[0013] Install position / angle sensors on the tower crane boom and hook to determine the corresponding position / angle sensor data;

[0014] Install zoom multi-channel cameras on the tower crane cab and determine the corresponding video data.

[0015] Furthermore, the step of inputting the raw sensor data and the video data into a shared feature extraction layer of a preset initial model to perform a shared feature extraction operation to determine corresponding sensor features and video features includes:

[0016] Performing preliminary feature fusion on the raw sensor data and the video data to determine corresponding fusion features;

[0017] The fused features are input into a preset convolutional neural network to perform a high-dimensional feature extraction operation to determine the corresponding sensor features and video features.

[0018] Furthermore, the inputting of the sensor features into a preset priority branch layer, performing weighted processing operations on the key features according to an attention algorithm, and determining corresponding high-priority sensor data includes:

[0019] Input the sensor features into a preset recursive neural network to determine corresponding priority classification features, wherein the sensor features are obtained by extracting features from the original sensor data of the tower crane after scene labeling, priority labeling, timeliness labeling, and importance labeling;

[0020] A weighted processing operation is performed on key features in the priority classification features according to an attention algorithm to determine corresponding high-priority sensor data.

[0021] Furthermore, the step of inputting the video features into a preset image processing branch layer, performing compression processing operations on the video features according to a codec network, and determining corresponding key video image information includes:

[0022] The video features are input into a preset image processing branch layer to determine the corresponding video content importance features, wherein the video features are obtained by feature extraction after compression requirement annotation, optimal compression ratio annotation, content complexity annotation and transmission priority annotation are performed on the original video data of the tower crane;

[0023] The compression ratio operation is dynamically adjusted according to the variable compression ratio algorithm and the importance characteristics of the video content to determine the corresponding key video image information.

[0024] Further, the high priority sensor data and the key video image information are input into the output layer of the preset initial model, and the corresponding loss function is determined according to the comprehensive data after passing through the output layer, including;

[0025] Inputting the high priority sensor data into an output layer of a preset initial model to determine a corresponding priority loss;

[0026] Inputting the key video image information into an output layer of a preset initial model to determine a corresponding image compression loss;

[0027] A comprehensive loss function calculation operation is performed according to the priority loss, the image compression loss and a preset weight coefficient to determine a corresponding loss function.

[0028] Further, the parameter updating operation is performed on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding tower crane data transmission model, including:

[0029] Performing a gradient calculation operation on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding gradient value;

[0030] According to the gradient value and the preset optimization algorithm, a parameter update operation is performed on the preset initial model to determine a corresponding tower crane data transmission model.

[0031] In a second aspect, the present application provides a data transmission device for a tower group unmanned operation system based on cloud-edge collaboration, comprising:

[0032] A wireless transmission data receiving module is used to receive original sensor data and video data of the tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback the three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0033] A wireless transmission model optimization module is used to input the original sensor data and video data into the shared feature extraction layer of a preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into a preset priority branch layer, perform a weighted processing operation on the key features according to an attention algorithm, determine the corresponding high-priority sensor data, input the video features into a preset image processing branch layer, perform a compression processing operation on the video features according to a codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of a preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and a back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0034] The wireless data transmission real-time transmission module is used to receive the real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission method, perform data transmission according to the optimal data transmission method, and realize real-time tower crane control.

[0035] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration are implemented.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration.

[0038] It can be seen from the above technical scheme that the present application provides a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, which performs a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of a preset initial model, determines the corresponding sensor features and video features, inputs the sensor features into the preset priority branch layer, determines the corresponding high-priority sensor data, inputs the video features into the preset image processing branch layer, determines the corresponding key video image information, inputs the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determines the corresponding tower crane data transmission model; receives real-time sensor data and video data of the tower crane, inputs the real-time sensor data and video data into the tower crane data transmission model, determines the optimal data transmission method, performs data transmission according to the optimal data transmission method, and realizes real-time tower crane control, thereby improving the tower crane control efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is one of the flow charts of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0041] Figure 2 This is a second flow chart of a method for transmitting data of a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0042] Figure 3 This is a flowchart of the third method for transmitting data of a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0043] Figure 4 This is a fourth flow chart of a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0044] Figure 5 This is a fifth flow chart of a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0045] Figure 6 This is a sixth flow chart of a method for transmitting data of a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0046] Figure 7 This is the seventh flow chart of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration in the embodiment of the present application;

[0047] Figure 8 This is a structural diagram of a data transmission device for a tower group unmanned operation system based on cloud-edge collaboration in an embodiment of the present application;

[0048] Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0049] Reference numerals:

[0050] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0053] Considering the high incidence and high danger of tower crane accidents and the inefficiency of the means taken to avoid risks. The present application provides a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, which performs a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determines the corresponding sensor features and video features, inputs the sensor features into the preset priority branch layer, determines the corresponding high-priority sensor data, inputs the video features into the preset image processing branch layer, determines the corresponding key video image information, inputs the high-priority sensor data and key video image information into the output layer of the preset initial model, and determines the corresponding tower crane data transmission model; receives the real-time sensor data and video data of the tower crane, inputs the real-time sensor data and video data into the tower crane data transmission model, determines the optimal data transmission mode, performs data transmission according to the optimal data transmission mode, and realizes real-time tower crane control, thereby improving the tower crane control efficiency.

[0054] In order to improve the control efficiency of tower cranes, this application provides an embodiment of a data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, see Figure 1 The data transmission method of the tower group unmanned operation system based on cloud-edge collaboration specifically includes the following contents:

[0055] Step S101: receiving original sensor data and video data of a tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0056] Optionally, in this embodiment, this step is applied to the wireless transmission module, which receives the original sensor data and video data of the tower crane. The purpose is to build an intelligent wireless transmission channel so that the connection between the tower crane and the ground control station is not affected by cables, and at the same time has the characteristics of low data transmission delay and high efficiency.

[0057] It is understandable that since the tower crane has multiple degrees of freedom, if a direct cable connection is used to transmit data from the tower crane to the ground, there may be a risk of cable entanglement, so a wireless communication channel is required.

[0058] Optionally, in this embodiment, the wireless transmission module is connected to the data acquisition module and the data processing module. The data acquisition module is placed on the tower crane and is responsible for collecting tower crane sensor data and video data, and transmitting the collected data to the wireless transmission module on the tower crane for interaction with the ground wireless transmission module. Through intelligent wireless interaction means, the ground end can quickly obtain data information on the tower crane and process it in the data processing module, thereby improving the tower crane control efficiency and reducing the probability of tower crane accidents.

[0059] Optionally, in this embodiment, the data acquisition module and the data acquisition card module are designed based on FPGA, and their functions are: to collect and package the data of each sensor and send it to the back end. Since the video data is a raw bit stream, an AD conversion module is required; the sensor data is also transmitted to the acquisition card. The acquisition card collects and packages various types of data and transmits them to the back end through the optical port.

[0060] Optionally, in this embodiment, the original sensor data collected by the data acquisition module includes ultrasonic sensor data and position / angle sensor data, wherein the ultrasonic sensor data is measured by ultrasonic ranging sensors installed at the front and rear ends of the tower arm and the hook; the position / angle sensor data is measured by position / angle sensors installed on the boom and the hook respectively; the video data collected by the data acquisition module is recorded by multiple zoom cameras with a pan-tilt head installed in the cab (simulating various angles that the human eye needs to pay attention to).

[0061] Specifically, ultrasonic sensors are used to detect obstacles that may appear at both ends of the boom during movement; cameras are used for refined auxiliary control and monitoring; and position / angle sensors are used to provide real-time feedback on the three-dimensional spatial information of the current tower crane boom and hook.

[0062] It can be understood that the tower crane can be sensed in all directions through the above sensors arranged in various positions of the tower crane:

[0063] The position / angle sensor can be used to feedback the status of the tower crane itself, including the position and angle of the tower crane itself, to prevent the tower crane from overturning due to excessive load, excessive wind pressure or other factors, causing serious accidents;

[0064] Ultrasonic sensors are placed at the front and rear ends of the tower crane's boom and at the hook to protect the crane's range of motion in all directions, including the distance between the crane and other obstacles. Because tower cranes usually work in groups, how to avoid collisions between cranes and accidents is also an issue that must be paid attention to.

[0065] Optionally, in this embodiment, due to its large bandwidth attribute, video data transmission is often encoded at the data generation end and decoded at the playback end. Although this method effectively reduces the amount of data transmission, the encoding and decoding operations often involve data caching and encoding and decoding calculations, resulting in end-to-end video delays. This solution adopts a non-encoding and decoding method, collects the original raw stream data through the FPGA data acquisition card, transmits it to the microwave data transmission upper module (installed on the tower crane) through the optical port, transmits it to the microwave lower module (installed under the tower crane) through point-to-point high-throughput transmission between microwave modules, and then transmits it to the optical switch, and then enters the control center for data processing. Compared with video data, various types of sensor data are very small in size, and are also transmitted to the control center for data processing via this link. The motion control instructions of the tower crane, including remote control by personnel through the remote control and control instructions output by the control center, are transmitted back to the controller at the tower crane end via the secondary link.

[0066] Optionally, after obtaining the sensor and video data collected by the above data acquisition module, it is necessary to preprocess these data to lay a solid data foundation for subsequent use of the original data for model training.

[0067] Specifically, first, the sensor data is standardized and the video data is subjected to image preprocessing (such as scaling and normalization).

[0068] Secondly, the sensor data is labeled. The types of annotations include:

[0069] Raw sensor data: includes real-time data from various sensors, such as distance, speed, direction, temperature, etc.

[0070] Scenario annotation: The application scenario type corresponding to each sensor data, such as normal, fault, obstacle detection, etc., is used to distinguish the context of different data.

[0071] Priority marking: Mark the data priority (high, medium, low) according to different scenarios and data types to ensure that key data can be transmitted and processed first.

[0072] Timeliness requirements: Mark the effective time limit or real-time requirements of various types of data to optimize the transmission strategy for real-time processing.

[0073] Data importance classification: Label the importance of data and classify it into necessary, secondary, low priority, etc., to further optimize the resource allocation for transmission and processing.

[0074] Finally, the video data is labeled. The types of annotations include:

[0075] Video data samples: including video frame sequences of different scenes and complexities, such as video data of different weather, lighting and object motion states from the perspective of a tower crane.

[0076] Compression requirement labeling: Each frame is labeled whether it needs compression (labeling value: 0 means no compression, 1 means compression), which helps the model identify the compression decision requirements under different bandwidth conditions.

[0077] Optimal compression ratio annotation: For frames that need to be compressed, the optimal compression ratio of the frame is annotated (such as 20%, 50%, 80%) to support the model to automatically adjust the compression ratio under different bandwidth conditions.

[0078] Content complexity labeling: Label the content complexity of the video scene (low, medium, high). For example, static background is low, tower crane movement or high wind speed is medium, fast motion or dynamic background is high.

[0079] Transmission priority marking: Mark the transmission priority (high, medium, low) of each frame to ensure that high-priority key frames are given priority transmission when the network is congested.

[0080] It can be understood that through the above-mentioned reception and preprocessing of tower crane sensor data and video data, we have obtained standardized tower crane sensor data sets and tower crane video data sets that can be used for machine learning, so that through subsequent machine learning, the intelligent model can learn the relationship between sensor data and video data and annotations, and make intelligent decisions in future applications.

[0081] Step S102: Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0082] Optionally, in this embodiment, the purpose of this step is to input the sensor data and video data obtained after the above preprocessing into the initial model architecture for model training, and iteratively tune the initial model so that the trained model can identify the optimal wireless transmission method for the data.

[0083] Optionally, in this embodiment, the technical means adopted in this step is to construct a multi-task learning model framework, adopt a dual-branch neural network architecture, that is, share the underlying feature extraction layer, and then design independent branches for the two tasks of sensor data and video data respectively.

[0084] Optionally, in this embodiment, the original sensor data and video data are input into a shared feature extraction layer of a preset initial model to perform a shared feature extraction operation to determine corresponding sensor features and video features.

[0085] Specifically, the shared feature extraction layer, which is used to extract basic features from the input sensor data and video data, is the basic part of the entire model.

[0086] The sensor data and video data are initially fused to obtain fusion features, and a convolutional neural network (CNN) is used to extract high-dimensional features from the fusion features. Multi-layer convolution and pooling operations are used to capture local features and patterns in the data. In the feature extraction process, regularization and normalization techniques are used to ensure the generalization ability and stability of the model. Finally, the output feature vector will be used for both the sensor data screening branch and the video data compression branch to support multi-task learning.

[0087] It can be understood that the shared feature extraction layer supports the needs of data screening and image compression simultaneously through joint feature representation, thereby improving the overall efficiency of the model.

[0088] Optionally, in this embodiment, the sensor features are input into a preset priority branch layer, and a weighted processing operation is performed on the key features according to an attention algorithm to determine corresponding high-priority sensor data.

[0089] Specifically, the priority branch is responsible for analyzing the sensor data and classifying the data by priority.

[0090] Recurrent neural networks (RNNs) are used to process time series data, and data is prioritized based on different features of sensor data (such as temperature, angle, vibration, etc.). Attention mechanisms are applied to weight key features, identify high-priority data, and multi-classification cross entropy loss functions are used for priority classification training. High-priority data will be transmitted first through the link to ensure real-time and reliability.

[0091] Among them, different features of sensor data are obtained by extracting the labeled labels, and the label annotation specifically includes:

[0092] Scene annotation: Annotate the scene corresponding to the sensor data. For example, the "obstacle detection" scene is used to indicate the situation where the ultrasonic sensors at both ends of the boom detect obstacles, and the "positioning feedback" scene is used to indicate the position feedback data of the tower arm or hook.

[0093] Priority labeling: Categorize data according to scenarios and importance. For example, obstacle detection is marked as "high priority" and temperature data is marked as "low priority" to ensure that critical data is transmitted and processed first.

[0094] Timeliness requirements: Mark the validity period of each type of data. For example, obstacle detection data needs to be transmitted within 1 second, and temperature monitoring data can be transmitted within 5 minutes to meet real-time requirements.

[0095] Data importance: The importance category of the data is marked. For example, "necessary" data is used for safety detection and must be transmitted in real time; "secondary" data, such as temperature information, is only used for environmental monitoring and does not affect the core control of the tower crane.

[0096] It is understandable that sensor data is annotated with raw sensor data, scene annotation, priority annotation, timeliness requirement annotation, and data importance classification before being trained by the model. After multiple annotated sensor data, the model can prioritize new data by learning the characteristic patterns in the annotation information. Annotation information such as scene type, priority, timeliness, and importance are input into the model as labels to help the model identify the characteristics of high-priority and low-priority data.

[0097] Specifically, first, the model extracts features from the raw sensor data. Different annotation information (such as priority and importance) will extract different feature patterns to help the model identify the content, timeliness, and importance of the sensor data. Through the multi-label learning model, the model learns the correspondence between the annotation information and the sensor data, for example:

[0098] In high-priority scenarios, the model recognizes the association of specific sensor data (e.g., obstacle detection signals) with high-priority labels;

[0099] In the low-priority scenario, the model learns that the signal has a greater tolerance for delays and therefore assigns it a lower priority.

[0100] Based on the learned features and priority rules, the model can prioritize input data in a real-time environment. For high-priority data, the model will mark it as critical data for priority processing or transmission.

[0101] For example, suppose that during the operation of a tower crane, the sensor data is extracted through shared features to obtain the following features:

[0102] Ultrasonic sensor features: Detects obstacles 2 meters ahead.

[0103] Position sensor characteristics: The current height of the hook is 20 meters and the angle deflection is 5 degrees.

[0104] Marking information:

[0105] Ultrasonic features are marked as: high priority (safety-related), and the timeliness requirement is to be transmitted within 1 second.

[0106] The position sensor feature is marked as: Medium priority (operation assistance), allowing updates within 3 seconds.

[0107] Fusion results:

[0108] Ultrasonic data will be transmitted first and trigger an alarm.

[0109] The position sensor data is used for system recording and has no impact due to delayed transmission.

[0110] Through the above process, the system realizes the priority classification and real-time management of multi-dimensional sensor data, determines the sensor data for high-priority transmission, and ensures rapid response and efficient transmission of key data.

[0111] Optionally, in this embodiment, the video features are input into a preset image processing branch layer, and compression processing operations are performed on the video features according to a codec network to determine corresponding key video image information.

[0112] Specifically, the image processing branch layer performs compression processing on video data to reduce the amount of data transmission.

[0113] First, this solution gives priority to non-encoding video transmission, which is more suitable for scenarios that need to maintain data originality and low latency, such as real-time control of tower cranes. However, due to the large bandwidth of video data, this solution combines artificial intelligence with non-encoding video transmission to improve it, intelligently compress the non-critical part of the video, and ensure that the critical video can be transmitted in a timely and fast manner.

[0114] Specifically, a convolutional coding-decoding network (CNN Autoencoder) is used to compress video images and filter backgrounds.

[0115] More specifically, a convolutional coding-decoding network (CNN Autoencoder) is used to learn important video content features using the label annotation of video data. During the compression process, variable compression ratio technology is used to dynamically adjust the compression ratio according to the importance of the image content, and the compression effect is optimized through the MSE (mean square error) loss function to retain the key information of the image as much as possible. In this way, video images can be intelligently compressed, background information and irrelevant content can be removed, and key information can be retained during transmission, while reducing bandwidth requirements.

[0116] The labeling of video data is performed before feature extraction, that is, in the data preprocessing stage, which includes:

[0117] Video data samples: including video frame sequences of different scenes and complexities, such as video data of different weather, lighting and object motion states from the perspective of a tower crane.

[0118] Compression requirement labeling: Each frame is labeled whether it needs compression (labeling value: 0 means no compression, 1 means compression), which helps the model identify the compression decision requirements under different bandwidth conditions.

[0119] Optimal compression ratio annotation: For frames that need to be compressed, the optimal compression ratio of the frame is annotated (such as 20%, 50%, 80%) to support the model to automatically adjust the compression ratio under different bandwidth conditions.

[0120] Content complexity labeling: Label the content complexity of the video scene (low, medium, high). For example, static background is low, tower crane movement or high wind speed is medium, fast motion or dynamic background is high.

[0121] Transmission priority marking: Mark the transmission priority (high, medium, low) of each frame to ensure that high-priority key frames are given priority transmission when the network is congested.

[0122] It can be understood that the priority model recognition process of video data is similar to that of sensors, and both determine whether to compress and the optimal compression ratio through the relationship between model recognition label annotation and scene.

[0123] The high-dimensional video features obtained after feature extraction are weighted and fused, and the weights of key features are increased by the feature weights in the label information. The weighted feature vector is input into the preset image processing branch layer, which classifies and processes video data of different priorities through different processing modules, such as giving priority to high-priority emergency scene data.

[0124] In the image processing branch layer, data of different priorities will be classified and processed, for example:

[0125] High-priority video segment: Improves frame rate and clarity to ensure that details of emergency events are clearly presented.

[0126] Low priority video segments: Reduce resolution or compress frame rate to optimize system resource utilization.

[0127] For example, tower crane cameras continuously collect video images, where different scenarios (such as "emergency events", "routine monitoring", and "low-priority areas") require different image processing methods.

[0128] High priority images: trigger an alarm immediately and are processed in high definition.

[0129] Medium priority images: Remain processed and stored normally.

[0130] Low priority images: stored at lower resolution and transmitted with a delay.

[0131] It can be understood that through the above-mentioned intelligent video compression method, the model can dynamically adjust the processing priority of the image according to the annotation features, ensuring the rational allocation of resources and rapid response to emergencies.

[0132] Optionally, in this embodiment, the high-priority sensor data and the key video image information are input into the output layer of a preset initial model, and the corresponding loss function is determined based on the comprehensive data after passing through the output layer.

[0133] Specifically, the output layer is used to output the filtered and compressed data results.

[0134] More specifically, the sensor data screening branch outputs the priority classification result, marks the high priority data, and calculates the priority loss; the image compression branch outputs the compressed image data and calculates the image compression loss; the output layer combines the results of the two branches, integrates the data of different priorities and the compressed image, and calculates the comprehensive loss, where the comprehensive loss is the weighted sum of the priority loss and the image compression loss.

[0135] Optionally, a parameter update operation is performed on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding tower crane data transmission model.

[0136] Specifically, after obtaining the comprehensive loss, the minimum loss function is determined through iterative training, the gradient is calculated using the back propagation algorithm based on the minimum comprehensive loss, and the model parameters are updated using an optimization algorithm (such as Adam or SGD).

[0137] After updating the model parameters, the validation set is used to evaluate the performance, including data screening accuracy and image compression quality (such as PSNR). The hyperparameters are adjusted to balance the performance of the two tasks. Multiple rounds of iterative training are performed until the performance of the model on the validation set is stable, and the tower crane data transmission model is obtained.

[0138] It can be understood that through the above training process, the obtained model can not only realize efficient data screening and priority management, but also intelligently compress video data to improve the efficiency and reliability of the overall transmission link.

[0139] Step S103: receiving real-time sensor data and video data of the tower crane, inputting the real-time sensor data and video data into the tower crane data transmission model, determining an optimal data transmission mode, performing data transmission according to the optimal data transmission mode, and realizing real-time tower crane control.

[0140] Optionally, after obtaining the above model, the model is deployed on the tower crane end of the wireless transmission module. Various sensors of the tower crane continuously collect data, and the video camera records the real-time image of the operation site. These data are input into the shared feature extraction layer of the tower crane data transmission model to extract different types of data features to provide a basis for subsequent decision-making; sensor data is prioritized according to scene, real-time, security, and importance requirements; to reduce bandwidth occupancy, video data is intelligently compressed through the codec network of the image processing branch layer, retaining important details, improving image compression quality, and enabling key video frames to be stably transmitted in a low-bandwidth environment. The optimal data transmission method is determined by the above method, wherein high-priority sensor data and compressed video data are transmitted to the control center via wireless transmission for operators to monitor and adjust the operation of the tower crane in real time. Through the optimization of the loss function of the model, the data transmission model is guaranteed to be updated in real time under different transmission conditions, realizing efficient monitoring and intelligent control of tower crane operations.

[0141] This example shows how to construct a tower crane data transmission model in this embodiment, realize intelligent processing of real-time data transmission of tower crane, ensure fast real-time transmission of important data, improve tower crane control efficiency, and reduce tower crane accidents.

[0142] From the above description, it can be seen that the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration provided in the embodiment of the present application can perform a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determine the corresponding tower crane data transmission model; receive real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission method, perform data transmission according to the optimal data transmission method, and realize real-time tower crane control, thereby improving the tower crane control efficiency.

[0143] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 2 , and can also include the following:

[0144] Step S201: installing ultrasonic distance measuring sensors on the front end and rear end of the boom and the hook of the tower crane to determine corresponding ultrasonic sensor data;

[0145] Step S202: installing position / angle sensors on the tower crane boom and hook to determine corresponding position / angle sensor data;

[0146] Step S203: Perform zoom multi-channel camera installation operation on the tower crane cab to determine corresponding video data.

[0147] Optionally, in this embodiment, the raw sensor data collected by the data acquisition module includes ultrasonic sensor data and position / angle sensor data, and the video data collected by the data acquisition module is recorded by multiple zoom cameras with a pan / tilt mount installed in the cab (simulating various angles that the human eye needs to pay attention to).

[0148] Specifically, ultrasonic sensors are used to detect obstacles that may appear at both ends of the boom during movement; cameras are used for refined auxiliary control and monitoring; and position / angle sensors are used to provide real-time feedback on the three-dimensional spatial information of the current tower crane boom and hook.

[0149] It can be understood that the tower crane can be sensed in all directions through the above sensors arranged in various positions of the tower crane:

[0150] The position / angle sensor can be used to feedback the status of the tower crane itself, including the position and angle of the tower crane itself, to prevent the tower crane from overturning due to excessive load, excessive wind pressure or other factors, causing serious accidents;

[0151] Ultrasonic sensors are placed at the front and rear ends of the tower crane's boom and at the hook to protect the crane's range of motion in all directions, including the distance between the crane and other obstacles. Because tower cranes usually work in groups, how to avoid collisions between cranes and accidents is also an issue that must be paid attention to.

[0152] Due to its large bandwidth, video data transmission is often encoded at the data generation end and decoded at the playback end. Although this method effectively reduces the amount of data transmission, the encoding and decoding operations often involve data caching and encoding and decoding calculations, resulting in end-to-end video delays. This solution adopts a non-encoding and decoding method. The original raw stream data is collected through the FPGA data acquisition card, transmitted to the microwave data transmission upper module (installed on the tower crane) through the optical port, and transmitted to the microwave lower module (installed under the tower crane) through point-to-point high-throughput transmission between microwave modules, and then transmitted to the optical switch, and then enters the control center for data processing. Compared with video data, various types of sensor data are very small in size and are also transmitted to the control center for data processing through this link. The motion control instructions of the tower crane, including remote control by personnel through the remote control and control instructions output by the control center, are transmitted back to the controller at the tower crane end through the secondary link.

[0153] Through step S203, this embodiment realizes the all-round collection of tower crane data by arranging sensors and cameras at various parts of the tower crane, which provides a solid data foundation for subsequent training models through tower crane data and reducing tower crane accidents.

[0154] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 3 , and can also include the following:

[0155] Step S301: performing preliminary feature fusion on the original sensor data and the video data to determine corresponding fusion features;

[0156] Step S302: Input the fused features into a preset convolutional neural network to perform a high-dimensional feature extraction operation to determine corresponding sensor features and video features.

[0157] Optionally, in this embodiment, the sensor data and the video data are initially fused to obtain fused features, and a convolutional neural network (CNN) is used to extract high-dimensional features from the fused features. Multi-layer convolution and pooling operations are used to capture local features and patterns in the data. In the feature extraction process, regularization and normalization techniques are used to ensure the generalization ability and stability of the model. Finally, the output feature vector will be used for both the sensor data screening branch and the video data compression branch to support multi-task learning.

[0158] It can be understood that the shared feature extraction layer supports the needs of data screening and image compression simultaneously through joint feature representation, thereby improving the overall efficiency of the model.

[0159] Through step S302, this embodiment realizes the fusion of sensor data and video data and the extraction of high-dimensional features, laying a data foundation for subsequent two-branch training.

[0160] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 4 , and can also include the following:

[0161] Step S401: inputting the sensor features into a preset recursive neural network to determine corresponding priority classification features, wherein the sensor features are obtained by extracting the original sensor data of the tower crane after scene labeling, priority labeling, timeliness labeling and importance labeling;

[0162] Step S402: performing weighted processing operations on key features in the priority classification features according to an attention algorithm to determine corresponding high-priority sensor data.

[0163] Optionally, in this embodiment, a recurrent neural network (RNN) is used to process time series data, and the data is prioritized according to different features of the sensor data (such as temperature, angle, vibration, etc.). The attention mechanism is applied to weight key features, identify high-priority data, and use a multi-classification cross entropy loss function for priority classification training. High-priority data will be transmitted preferentially through the link to ensure real-time and reliability.

[0164] Among them, different features of sensor data are obtained by extracting the labeled labels, and the label annotation specifically includes:

[0165] Scene annotation: Annotate the scene corresponding to the sensor data. For example, the "obstacle detection" scene is used to indicate the situation where the ultrasonic sensors at both ends of the boom detect obstacles, and the "positioning feedback" scene is used to indicate the position feedback data of the tower arm or hook.

[0166] Priority labeling: Categorize data according to scenarios and importance. For example, obstacle detection is marked as "high priority" and temperature data is marked as "low priority" to ensure that critical data is transmitted and processed first.

[0167] Timeliness requirements: Mark the validity period of each type of data. For example, obstacle detection data needs to be transmitted within 1 second, and temperature monitoring data can be transmitted within 5 minutes to meet real-time requirements.

[0168] Data importance: The importance category of the data is marked. For example, "necessary" data is used for safety detection and must be transmitted in real time; "secondary" data, such as temperature information, is only used for environmental monitoring and does not affect the core control of the tower crane.

[0169] It is understandable that sensor data is annotated with raw sensor data, scene annotation, priority annotation, timeliness requirement annotation, and data importance classification before being trained by the model. After multiple annotated sensor data, the model can prioritize new data by learning the characteristic patterns in the annotation information. Annotation information such as scene type, priority, timeliness, and importance are input into the model as labels to help the model identify the characteristics of high-priority and low-priority data.

[0170] Specifically, first, the model extracts features from the raw sensor data. Different annotation information (such as priority and importance) will extract different feature patterns to help the model identify the content, timeliness, and importance of the sensor data. Through the multi-label learning model, the model learns the correspondence between the annotation information and the sensor data, for example:

[0171] In high-priority scenarios, the model recognizes the association of specific sensor data (e.g., obstacle detection signals) with high-priority labels;

[0172] In the low-priority scenario, the model learns that the signal has a greater tolerance for delays and therefore assigns it a lower priority.

[0173] Based on the learned features and priority rules, the model can prioritize input data in a real-time environment. For high-priority data, the model will mark it as critical data for priority processing or transmission.

[0174] For example, suppose that during the operation of a tower crane, the sensor data is extracted through shared features to obtain the following features:

[0175] Ultrasonic sensor features: Detects obstacles 2 meters ahead.

[0176] Position sensor characteristics: The current height of the hook is 20 meters and the angle deflection is 5 degrees.

[0177] Marking information:

[0178] Ultrasonic features are marked as: high priority (safety-related), and the timeliness requirement is to be transmitted within 1 second.

[0179] The position sensor feature is marked as: Medium priority (operation assistance), allowing updates within 3 seconds.

[0180] Fusion results:

[0181] Ultrasonic data will be transmitted first and trigger an alarm.

[0182] The position sensor data is used for system recording and has no impact due to delayed transmission.

[0183] Through step S402, this embodiment successfully implements the priority classification and real-time management of multi-dimensional sensor data, determines the sensor data for high priority transmission, and ensures rapid response and efficient transmission of key data.

[0184] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 5 , and can also include the following:

[0185] Step S501: inputting the video features into a preset image processing branch layer to determine the corresponding video content importance features, wherein the video features are obtained by extracting features from the original video data of the tower crane after marking compression requirements, optimal compression ratios, content complexity, and transmission priority;

[0186] Step S502: dynamically adjusting the compression ratio according to the variable compression ratio algorithm and the importance characteristics of the video content to determine the corresponding key video image information.

[0187] Optionally, in this embodiment, the image processing branch layer performs compression processing on the video data to reduce the amount of data transmission.

[0188] First, this solution gives priority to non-encoding video transmission, which is more suitable for scenarios that need to maintain data originality and low latency, such as real-time control of tower cranes. However, due to the large bandwidth of video data, this solution combines artificial intelligence with non-encoding video transmission to improve it, intelligently compress the non-critical part of the video, and ensure that the critical video can be transmitted in a timely and fast manner.

[0189] Specifically, a convolutional coding-decoding network (CNN Autoencoder) is used to compress video images and filter backgrounds.

[0190] More specifically, a convolutional coding-decoding network (CNN Autoencoder) is used to learn important video content features using the label annotation of video data. During the compression process, variable compression ratio technology is used to dynamically adjust the compression ratio according to the importance of the image content, and the compression effect is optimized through the MSE (mean square error) loss function to retain the key information of the image as much as possible. In this way, video images can be intelligently compressed, background information and irrelevant content can be removed, and key information can be retained during transmission, while reducing bandwidth requirements.

[0191] The labeling of video data is performed before feature extraction, that is, in the data preprocessing stage, which includes:

[0192] Video data samples: including video frame sequences of different scenes and complexities, such as video data of different weather, lighting and object motion states from the perspective of a tower crane.

[0193] Compression requirement labeling: Each frame is labeled whether it needs compression (labeling value: 0 means no compression, 1 means compression), which helps the model identify the compression decision requirements under different bandwidth conditions.

[0194] Optimal compression ratio annotation: For frames that need to be compressed, the optimal compression ratio of the frame is annotated (such as 20%, 50%, 80%) to support the model to automatically adjust the compression ratio under different bandwidth conditions.

[0195] Content complexity labeling: Label the content complexity of the video scene (low, medium, high). For example, static background is low, tower crane movement or high wind speed is medium, fast motion or dynamic background is high.

[0196] Transmission priority marking: Mark the transmission priority (high, medium, low) of each frame to ensure that high-priority key frames are given priority transmission when the network is congested.

[0197] It can be understood that the priority model recognition process of video data is similar to that of sensors, and both determine whether to compress and the optimal compression ratio through the relationship between model recognition label annotation and scene.

[0198] The high-dimensional video features obtained after feature extraction are weighted and fused, and the weights of key features are increased by the feature weights in the label information. The weighted feature vector is input into the preset image processing branch layer, which classifies and processes video data of different priorities through different processing modules, such as giving priority to high-priority emergency scene data.

[0199] In the image processing branch layer, data of different priorities will be classified and processed, for example:

[0200] High-priority video segment: Improves frame rate and clarity to ensure that details of emergency events are clearly presented.

[0201] Low priority video segments: Reduce resolution or compress frame rate to optimize system resource utilization.

[0202] For example, tower crane cameras continuously collect video images, where different scenarios (such as "emergency events", "routine monitoring", and "low-priority areas") require different image processing methods.

[0203] High priority images: trigger an alarm immediately and are processed in high definition.

[0204] Medium priority images: Remain processed and stored normally.

[0205] Low priority images: stored at lower resolution and transmitted with a delay.

[0206] Through step S502, this embodiment successfully enables the model to dynamically adjust the processing priority of the image according to the annotation features, thereby ensuring reasonable allocation of resources and rapid response to emergency events.

[0207] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 6 , and can also include the following:

[0208] Step S601: inputting the high priority sensor data into the output layer of a preset initial model to determine the corresponding priority loss;

[0209] Step S602: inputting the key video image information into the output layer of a preset initial model to determine the corresponding image compression loss;

[0210] Step S603: Perform a comprehensive loss function calculation operation according to the priority loss, the image compression loss and a preset weight coefficient to determine a corresponding loss function.

[0211] Optionally, in this embodiment, an output layer is used to output the filtered and compressed data results.

[0212] Optionally, in this embodiment, the sensor data screening branch outputs the priority classification result, marks the high priority data, and calculates the priority loss; the image compression branch outputs the compressed image data and calculates the image compression loss; the output layer combines the results of the two branches, integrates the data of different priorities and the compressed image, and calculates the comprehensive loss, wherein the comprehensive loss is obtained by weighted summing of the priority loss and the image compression loss.

[0213] Through step S603, this embodiment successfully calculates the comprehensive loss function of the two branches, laying the foundation for obtaining the optimal model later.

[0214] In one embodiment of the tower group unmanned operation system data transmission method based on cloud-edge collaboration of the present application, see Figure 7 , and can also include the following:

[0215] Step S701: performing a gradient calculation operation on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding gradient value;

[0216] Step S702: performing a parameter update operation on the preset initial model according to the gradient value and the preset optimization algorithm to determine a corresponding tower crane data transmission model.

[0217] Optionally, in this embodiment, after obtaining the comprehensive loss, the minimum loss function is determined through iterative training, the gradient is calculated using the back propagation algorithm based on the minimum comprehensive loss, and the model parameters are updated using an optimization algorithm (such as Adam or SGD).

[0218] After updating the model parameters, the validation set is used to evaluate the performance, including data screening accuracy and image compression quality (such as PSNR). The hyperparameters are adjusted to balance the performance of the two tasks. Multiple rounds of iterative training are performed until the performance of the model on the validation set is stable, and the tower crane data transmission model is obtained.

[0219] It can be understood that through the above training process, the obtained model can not only realize efficient data screening and priority management, but also intelligently compress video data to improve the efficiency and reliability of the overall transmission link.

[0220] Through step S702, this embodiment successfully obtains the tower crane data transmission model, laying the foundation for subsequently calculating the optimal data transmission method of the tower group unmanned operation system based on cloud-edge collaboration through the model, and improving the tower crane control efficiency through this data transmission method.

[0221] In order to improve the control efficiency of tower cranes, the present application provides an embodiment of a tower group unmanned operation system data transmission device based on cloud-edge collaboration for realizing all or part of the content of the tower group unmanned operation system data transmission method based on cloud-edge collaboration, see Figure 8 The data transmission device of the tower group unmanned operation system based on cloud-edge collaboration specifically includes the following contents:

[0222] The wireless transmission data receiving module 10 is used to receive the original sensor data and video data of the tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during the movement, the position / angle sensor data is used to feedback the three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0223] The wireless transmission model optimization module 20 is used to input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0224] The wireless data real-time transmission module 30 is used to receive the real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission method, perform data transmission according to the optimal data transmission method, and realize real-time tower crane control.

[0225] From the above description, it can be seen that the data transmission device of the tower group unmanned operation system based on cloud-edge collaboration provided in the embodiment of the present application can perform a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determine the corresponding tower crane data transmission model; receive real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission method, perform data transmission according to the optimal data transmission method, and realize real-time tower crane control, thereby improving the tower crane control efficiency.

[0226] From the hardware level, in order to improve the control efficiency of the tower crane, the present application provides an embodiment of an electronic device for implementing all or part of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration, and the electronic device specifically includes the following contents:

[0227] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize the information transmission between the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration and the core business system, user terminal and related databases and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration and the embodiment of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0228] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0229] In practical applications, part of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0230] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0231] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0232] In one embodiment, the data transmission method function of the tower group unmanned operation system based on cloud-edge collaboration can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control:

[0233] Step S101: receiving original sensor data and video data of a tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0234] Step S102: Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0235] Step S103: receiving real-time sensor data and video data of the tower crane, inputting the real-time sensor data and video data into the tower crane data transmission model, determining an optimal data transmission mode, performing data transmission according to the optimal data transmission mode, and realizing real-time tower crane control.

[0236] From the above description, it can be seen that the electronic device provided in the embodiment of the present application performs a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determines the corresponding sensor features and video features, inputs the sensor features into the preset priority branch layer, determines the corresponding high-priority sensor data, inputs the video features into the preset image processing branch layer, determines the corresponding key video image information, inputs the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determines the corresponding tower crane data transmission model; receives real-time sensor data and video data of the tower crane, inputs the real-time sensor data and video data into the tower crane data transmission model, determines the optimal data transmission method, performs data transmission according to the optimal data transmission method, and realizes real-time tower crane control, thereby improving the tower crane control efficiency.

[0237] In another embodiment, the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration can be configured separately from the central processor 9100. For example, the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration can be configured as a chip connected to the central processor 9100, and the function of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration can be realized through the control of the central processor.

[0238] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0239] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0240] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0241] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0242] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0243] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0244] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0245] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the machine through the microphone 9132, and the sound stored on the machine can be played through the speaker 9131.

[0246] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration in the above-mentioned embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration in the above-mentioned embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0247] Step S101: receiving original sensor data and video data of a tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0248] Step S102: Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0249] Step S103: receiving real-time sensor data and video data of the tower crane, inputting the real-time sensor data and video data into the tower crane data transmission model, determining an optimal data transmission mode, performing data transmission according to the optimal data transmission mode, and realizing real-time tower crane control.

[0250] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application performs a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determines the corresponding sensor features and video features, inputs the sensor features into the preset priority branch layer, determines the corresponding high-priority sensor data, inputs the video features into the preset image processing branch layer, determines the corresponding key video image information, inputs the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determines the corresponding tower crane data transmission model; receives real-time sensor data and video data of the tower crane, inputs the real-time sensor data and video data into the tower crane data transmission model, determines the optimal data transmission method, performs data transmission according to the optimal data transmission method, and realizes real-time tower crane control, thereby improving the control efficiency of the tower crane.

[0251] The embodiments of the present application also provide a computer program product capable of implementing all steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the data transmission method for a tower group unmanned operation system based on cloud-edge collaboration are implemented. For example, the computer program / instruction implements the following steps:

[0252] Step S101: receiving original sensor data and video data of a tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring;

[0253] Step S102: Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into the preset priority branch layer, perform a weighted processing operation on the key features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model;

[0254] Step S103: receiving real-time sensor data and video data of the tower crane, inputting the real-time sensor data and video data into the tower crane data transmission model, determining an optimal data transmission mode, performing data transmission according to the optimal data transmission mode, and realizing real-time tower crane control.

[0255] From the above description, it can be seen that the computer program product provided by the embodiment of the present application performs a shared feature extraction operation by inputting the original sensor data and video data of the tower crane into the shared feature extraction layer of the preset initial model, determines the corresponding sensor features and video features, inputs the sensor features into the preset priority branch layer, determines the corresponding high-priority sensor data, inputs the video features into the preset image processing branch layer, determines the corresponding key video image information, inputs the high-priority sensor data and the key video image information into the output layer of the preset initial model, and determines the corresponding tower crane data transmission model; receives real-time sensor data and video data of the tower crane, inputs the real-time sensor data and video data into the tower crane data transmission model, determines the optimal data transmission method, performs data transmission according to the optimal data transmission method, and realizes real-time tower crane control, thereby improving the tower crane control efficiency.

[0256] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0257] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0258] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0260] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data transmission method for a tower group unmanned operation system based on cloud-edge collaboration, characterized in that: Applied to a wireless transmission module, the wireless transmission module is connected to a data acquisition module and a data processing module respectively, and the method comprises: Receive original sensor data and video data of the tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback the three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring; Input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into a preset recursive neural network, determine the corresponding priority classification features, wherein the sensor features are obtained by performing scene labeling, priority labeling, timeliness labeling and importance labeling on the original sensor data of the tower crane, and then extracting the features, perform a weighted processing operation on the key features in the priority classification features according to the attention algorithm, determine the corresponding high-priority sensor data, input the video features into the preset image processing branch layer, perform a compression processing operation on the video features according to the codec network, determine the corresponding key video image information, input the high-priority sensor data and the key video image information into the output layer of the preset initial model, determine the corresponding loss function according to the comprehensive data after passing through the output layer, perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm, and determine the corresponding tower crane data transmission model; Receive real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission mode, perform data transmission according to the optimal data transmission mode, and realize real-time tower crane control.

2. The data transmission method of the tower group unmanned operation system based on cloud-edge collaboration according to claim 1 is characterized in that: Before receiving the original sensor data and video data of the tower crane, the method includes: Install ultrasonic distance measuring sensors on the front end and rear end of the tower crane boom and the hook to determine the corresponding ultrasonic sensor data; Install position / angle sensors on the tower crane boom and hook to determine the corresponding position / angle sensor data; Install zoom multi-channel cameras on the tower crane cab and determine the corresponding video data.

3. The data transmission method for a tower group unmanned operation system based on cloud-edge collaboration according to claim 1 is characterized in that: The step of inputting the raw sensor data and the video data into a shared feature extraction layer of a preset initial model to perform a shared feature extraction operation and determine corresponding sensor features and video features includes: Performing preliminary feature fusion on the raw sensor data and the video data to determine corresponding fusion features; The fused features are input into a preset convolutional neural network to perform a high-dimensional feature extraction operation to determine the corresponding sensor features and video features.

4. The data transmission method for a tower group unmanned operation system based on cloud-edge collaboration according to claim 1 is characterized in that: The step of inputting the video features into a preset image processing branch layer, performing a compression processing operation on the video features according to a codec network, and determining corresponding key video image information includes: The video features are input into a preset image processing branch layer to determine the corresponding video content importance features, wherein the video features are obtained by feature extraction after compression requirement annotation, optimal compression ratio annotation, content complexity annotation and transmission priority annotation are performed on the original video data of the tower crane; The compression ratio operation is dynamically adjusted according to the variable compression ratio algorithm and the importance characteristics of the video content to determine the corresponding key video image information.

5. The data transmission method for tower group unmanned operation system based on cloud-edge collaboration according to claim 1 is characterized in that: The inputting the high priority sensor data and the key video image information into the output layer of the preset initial model, and determining the corresponding loss function according to the comprehensive data after passing through the output layer, comprises: Inputting the high priority sensor data into an output layer of a preset initial model to determine a corresponding priority loss; Inputting the key video image information into an output layer of a preset initial model to determine a corresponding image compression loss; A comprehensive loss function calculation operation is performed according to the priority loss, the image compression loss and a preset weight coefficient to determine a corresponding loss function.

6. The data transmission method for tower group unmanned operation system based on cloud-edge collaboration according to claim 1 is characterized in that: The step of performing a parameter updating operation on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding tower crane data transmission model includes: Performing a gradient calculation operation on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding gradient value; According to the gradient value and the preset optimization algorithm, a parameter update operation is performed on the preset initial model to determine a corresponding tower crane data transmission model.

7. A data transmission device for a tower group unmanned operation system based on cloud-edge collaboration, characterized in that: The device comprises: A wireless transmission data receiving module is used to receive original sensor data and video data of the tower crane, wherein the original sensor data includes ultrasonic sensor data and position / angle sensor data, the ultrasonic sensor data is used to indicate obstacles that may appear at both ends of the boom during movement, the position / angle sensor data is used to feedback the three-dimensional spatial information of the current tower crane boom and hook, and the video data is used for auxiliary control and monitoring; A wireless transmission model optimization module is used to input the original sensor data and video data into the shared feature extraction layer of the preset initial model to perform a shared feature extraction operation, determine the corresponding sensor features and video features, input the sensor features into a preset recursive neural network, and determine the corresponding priority classification features, wherein the sensor features are obtained by performing scene labeling, priority labeling, timeliness labeling, and importance labeling on the original sensor data of the tower crane, and then extracting the features; performing a weighted processing operation on the key features in the priority classification features according to the attention algorithm to determine the corresponding high-priority sensor data; input the video features into the preset image processing branch layer; perform a compression processing operation on the video features according to the codec network to determine the corresponding key video image information; input the high-priority sensor data and the key video image information into the output layer of the preset initial model; determine the corresponding loss function according to the comprehensive data after passing through the output layer; perform a parameter update operation on the preset initial model according to the minimum loss function and the back propagation algorithm to determine the corresponding tower crane data transmission model; The wireless data transmission real-time transmission module is used to receive the real-time sensor data and video data of the tower crane, input the real-time sensor data and video data into the tower crane data transmission model, determine the optimal data transmission method, perform data transmission according to the optimal data transmission method, and realize real-time tower crane control.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the data transmission method of the tower group unmanned operation system based on cloud-edge collaboration as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data transmission method of a tower group unmanned operation system based on cloud-edge collaboration as described in any one of claims 1 to 6 are implemented.

Images