5G tower crane remote control method and system based on digital panorama

Abstract

The invention discloses a 5G tower crane remote control method and system based on digital panorama, and relates to the technical field of tower crane control, and the method comprises the steps: dividing areas where static objects and dynamic objects are located, and carrying out the analysis of the dynamic objects by taking the area as a unit; a digital panoramic model of the surrounding environment of a tower crane site is constructed by analyzing capture requirements in areas where static objects and dynamic objects are located, adaptive gridding processing is carried out on the model according to the conditions of the dynamic objects and the static objects in the areas, and the adaptability of capture detail precision of the model is guaranteed. And a reliable basis is provided for subsequent dynamic object identification, tracking, prediction and collision risk assessment. The dynamic object in the digital panoramic model is identified, the motion trail of the dynamic object is predicted, the tracking and prediction accuracy of the dynamic object is improved by setting a search area, the collision risk is analyzed by comparing the motion trail of the dynamic object with the motion trail of the tower crane, the accuracy and safety of remote control of the tower crane are improved, and the collision risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of tower crane control, and particularly to a 5G tower crane remote control method and system based on digital panorama. Background Art

[0002] With the rapid development of 5G technology, its application in the industrial field is becoming increasingly widespread, providing strong technical support for the remote control of tower cranes. The traditional tower crane operation method has problems such as high risk of high-altitude operation and low personnel utilization rate. The 5G technology, with its characteristics of high speed, low latency, and large connection, makes it possible for remote control of tower cranes. Through the 5G network, tower crane status data, control instructions, and video stream data can be transmitted in real time and reliably, enabling operators to remotely control the tower crane on the ground to complete lifting operations. This solution not only improves the working environment of operators, reduces labor intensity, but also effectively improves operation efficiency and safety, opening up a new path for the intelligent development of the tower crane industry. With the continuous maturity and improvement of related technologies, the 5G tower crane remote control solution will be more widely applied and promoted in the future.

[0003] In the prior art, during the tower crane control process, the impact of other dynamic objects in the on-site environment on the tower crane is not considered, resulting in an increased risk of collision between the tower crane and other objects, and lower accuracy and safety of the tower crane remote control, which cannot ensure the safe and stable operation of the tower crane.

[0004] Therefore, how to improve the accuracy and safety of tower crane remote control is a technical problem to be solved currently. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem of lower accuracy and safety of tower crane remote control in the prior art due to the lack of consideration of the impact of other dynamic objects in the on-site environment on the tower crane, and a 5G tower crane remote control method based on digital panorama is proposed, which includes:

[0006] Collect the environmental information around the tower crane site, identify static objects and dynamic objects in the on-site environment around the tower crane, and divide the areas where static objects and dynamic objects are located respectively;

[0007] Analyze the capture requirements in the areas where static objects and dynamic objects are located respectively, and construct a digital panorama model of the on-site environment around the tower crane according to the capture requirements;

[0008] Update the digital panorama model of the on-site environment around the tower crane, identify the dynamic objects in the digital panorama model, predict the movement trajectories of the dynamic objects, and mark the movement trajectories of the dynamic objects in the digital panorama model;

[0009] Plan multiple movement trajectories and registration action parameters of the tower crane according to the work tasks of the tower crane, mark the movement trajectories of the tower crane in the digital panoramic model, compare the movement trajectories of the moving objects and the tower crane to analyze the collision risk, and select the optimal movement trajectory of the tower crane;

[0010] Remotely control the tower crane through the registration action parameters of the tower crane and the optimal movement trajectory of the tower crane.

[0011] In some embodiments of the present application, confirm static objects and moving objects in the surrounding environment of the tower crane site, and divide the areas where the static objects and moving objects are located respectively, including,

[0012] Confirm all categories of objects in the surrounding environment of the tower crane site, distinguish static objects and moving objects, construct a three-dimensional model of the surrounding environment of the tower crane site, collect the position information of static objects and the position information of moving objects, integrate the position information of moving objects to form a position information set, and mark the static objects and moving objects at the corresponding positions in the three-dimensional model of the surrounding environment of the tower crane site based on the position information of static objects and the position information set of moving objects;

[0013] Record the area with only static objects as the static object area, and record the area with moving objects as the candidate area;

[0014] Determine the movement trajectories and size structure information of the moving objects in the candidate area, and analyze the influence degree of the moving objects on the tower crane operation in combination with the movement trajectories and size structure information of the moving objects. Screen the candidate area through the influence degree, and use the screened candidate area as the moving object area.

[0015] In some embodiments of the present application, analyze the capture requirements in the areas where static objects and moving objects are located respectively, including,

[0016] For the static object area, collect the position and size structure information of each static object in the area, and determine the influence degree of each static object on the tower crane operation in combination with the position and size structure information, so as to obtain the influence degree of this static object area on the tower crane operation;

[0017] For the moving object area, calculate the average movement speed and trajectory predictability index of each moving object in the area, and generate the influence degree of this moving object area on the tower crane operation based on the average movement speed, trajectory predictability index, influence degree of the moving object on the tower crane operation, and influence degree of the static object on the tower crane operation;

[0018]

[0019] Among them, is the influence degree of the i1-th dynamic object area on the tower crane operation. n1 and n2 are respectively the number of dynamic objects and the number of static objects in the i1-th dynamic object area. β1 and β2 are respectively the combined weights of dynamic objects and static objects in the i1-th dynamic object area. are respectively the conversion coefficients of the average activity speed and the trajectory predictability index of the i2-th dynamic object in the i1-th dynamic object area. are respectively the average activity speed and the trajectory predictability index of the i2-th dynamic object in the i1-th dynamic object area. is the influence degree of the i2-th dynamic object in the i1-th dynamic object area on the tower crane operation. k1 is a preset constant. is the influence degree of the i3-th static object in the i1-th dynamic object area on the tower crane operation.

[0020] Set the capture requirements of the static object area and the dynamic object area respectively according to the influence degree of the static object area on the tower crane operation and the influence degree of the dynamic object area on the tower crane operation.

[0021] In some embodiments of the present application, a digital panoramic model of the surrounding environment of the tower crane site is constructed according to the capture requirements, including

[0022] Mark the static object area and the dynamic object area on the three-dimensional model of the surrounding environment of the tower crane site, and perform grid density grid processing on the corresponding positions of the three-dimensional model of the surrounding environment of the tower crane site through the capture requirements on the static object area and the dynamic object area, so as to construct a digital panoramic model of the surrounding environment of the tower crane site.

[0023] In some embodiments of the present application, update the digital panoramic model of the surrounding environment of the tower crane site, including

[0024] For the static object area, collect the on-site environment data in the area at a preset cycle, and batch process the on-site environment data, so as to update the situation of the static object area on the digital panoramic model of the surrounding environment of the tower crane site;

[0025] For the dynamic object area, collect the on-site environment data in the area in real time, and perform real-time processing on the on-site environment data through 5G technology, so as to update the situation of the dynamic object area on the digital panoramic model of the surrounding environment of the tower crane site.

[0026] In some embodiments of the present application, identify the dynamic objects in the digital panoramic model and predict the movement trajectories of the dynamic objects, including

[0027] Divide the foreground part and the background part of the video image of the surrounding environment of the tower crane site, extract multiple image features of the dynamic objects in the foreground part, and perform dynamic object recognition by combining the image features;

[0028] Analyze the image features of a dynamic object in multiple frames to determine the motion information of the dynamic object, and use this to drive a dynamic object tracking algorithm to track the dynamic object, and predict the motion trajectory of the dynamic object according to a preset trajectory prediction model.

[0029] In some embodiments of the present application, analyzing the image features of a dynamic object in multiple frames to determine the motion information of the dynamic object, and using this to drive a dynamic object tracking algorithm to track the dynamic object, includes,

[0030] Determine the interfering objects in the background part through the image features of the dynamic object, and analyze the situation of the interfering objects, where the situation of the interfering objects includes the number of interfering objects, the type of interfering objects, the distribution of interfering objects, and the similarity of interfering objects;

[0031] Evaluate the interference degree of the background part based on the number of interfering objects, the type of interfering objects, the distribution of interfering objects, and the similarity of interfering objects;

[0032] Determine the size of the search area by integrating the motion information of the dynamic object and the interference degree of the background part, and drive the dynamic object tracking algorithm to track the dynamic object based on the size of the search area.

[0033] In some embodiments of the present application, compare the motion trajectory of the dynamic object and the motion trajectory of the tower crane to analyze the collision risk, including,

[0034] Perform time alignment processing on the motion trajectory of the dynamic object and the motion trajectory of the tower crane, consider the time error factor to determine an error time length, and segment the motion trajectory of the dynamic object and the motion trajectory of the tower crane according to the error time length to obtain multiple groups of motion trajectories, and each group of motion trajectories includes a motion trajectory segment of the dynamic object and a motion trajectory segment of the tower crane;

[0035] Uniformly set multiple time marking points on each group of motion trajectories, record the time marking points of the motion trajectory segment of the dynamic object as the first marking points, and record the time marking points of the motion trajectory segment of the tower crane as the second marking points. The time of the first marking points and the second marking points is the same and they correspond to each other;

[0036] Calculate the distance between the first marking point and the second marking point at the same time, denoted as the simultaneous distance, and calculate the distance between the first marking point and the second marking points at other times on the same group of motion trajectories, and take the average value as the non-simultaneous distance;

[0037] Generate the collision risk level of each group of motion trajectories according to the simultaneous distance and the non-simultaneous distance;

[0038]

[0039] Among them, is the collision risk level of the j1th group of motion trajectories, m1 is the number of the first or second marker points under the j1th group of motion trajectories, γ1 and γ2 are the combination weights of the simultaneous distance and the asynchronous distance respectively, and σ is the risk conversion coefficient. are the simultaneous distance and the asynchronous distance of the j2th first marker point under the j1th group of motion trajectories respectively. is the sum of their respective minimum values, k2 and k3 are preset constants, and [] is the rounding symbol.

[0040] Integrate the collision risk levels of each group of motion trajectories to determine the collision risk level, and describe the collision risk between the motion trajectories of dynamic objects and the motion trajectories of tower cranes through the collision risk level.

[0041] Correspondingly, the present application also provides a 5G tower crane remote control system based on digital panorama, including

[0042] The first module is used to collect the environmental information around the tower crane site, confirm static objects and dynamic objects in the surrounding environment of the tower crane site, and divide the areas where static objects and dynamic objects are located respectively.

[0043] The second module is used to analyze the capture requirements in the areas where static objects and dynamic objects are located respectively, and construct a digital panorama model of the surrounding environment of the tower crane site according to the capture requirements.

[0044] The third module is used to update the digital panorama model of the surrounding environment of the tower crane site, identify the dynamic objects in the digital panorama model, predict the motion trajectories of the dynamic objects, and mark the motion trajectories of the dynamic objects in the digital panorama model.

[0045] The fourth module is used to plan multiple motion trajectories of the tower crane and registration action parameters according to the working tasks of the tower crane, mark the motion trajectories of the tower crane in the digital panorama model, analyze the collision risk by comparing the motion trajectories of the dynamic objects and the motion trajectories of the tower crane, and screen out the optimal motion trajectories of the tower crane.

[0046] The fifth module is used to remotely control the tower crane through the registration action parameters of the tower crane and the optimal motion trajectories of the tower crane.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1. Divide the areas where static objects and dynamic objects are located respectively, analyze the dynamic objects in units of areas, analyze the capture requirements within the areas where static objects and dynamic objects are located respectively to construct a digital panoramic model of the surrounding environment of the tower crane site, and perform adaptive grid processing on the model according to the situations of dynamic objects and static objects within the areas to ensure the adaptability of the capture detail accuracy of the model, providing a reliable basis for subsequent dynamic object recognition, tracking, prediction, and collision risk assessment.

[0049] 2. Identify the dynamic objects in the digital panoramic model and predict their movement trajectories. By setting a search area, improve the accuracy of tracking and predicting dynamic objects, compare the movement trajectories of dynamic objects and the tower crane to analyze the collision risk, improve the accuracy and safety of remote control of the tower crane, reduce the collision risk, and ensure the safe operation of the tower crane. Description of the Drawings

[0050] Figure 1 It is a schematic flowchart of the 5G tower crane remote control method based on digital panorama proposed by the present invention;

[0051] Figure 2 It is a schematic structural diagram of the 5G tower crane remote control system based on digital panorama proposed by the present invention. Detailed Embodiment

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0053] Refer to Figure 1 , the 5G tower crane remote control method based on digital panorama includes the following steps:

[0054] Step S101, collect the environmental information around the tower crane site, confirm static objects and dynamic objects in the surrounding environment of the tower crane site, and divide the areas where static objects and dynamic objects are located respectively.

[0055] In this embodiment, it is necessary to use a variety of sensors (such as cameras, lidar, infrared sensors, etc.) to collect the surrounding environment of the tower crane site in all directions and at multiple angles. These sensors can capture various object information within the operation area of the tower crane, including static objects (such as buildings, fixed facilities, etc.) and dynamic objects (such as construction workers, other construction machinery, etc.).

[0056] In some embodiments of the present application, confirming static objects and dynamic objects in the surrounding environment of the tower crane site and dividing the areas where static objects and dynamic objects are located respectively includes,

[0057] Identify all categories of objects in the surrounding environment of the tower crane site, distinguish between static and dynamic objects, construct a three-dimensional model of the surrounding environment of the tower crane site, collect the position information of static objects and the position information of dynamic objects, integrate the position information of dynamic objects to form a position information set, and mark the static and dynamic objects at the corresponding positions in the three-dimensional model of the surrounding environment of the tower crane site based on the position information of static objects and the position information set of dynamic objects;

[0058] Denote the area with only static objects as the static object area, and denote the area with dynamic objects as the candidate area;

[0059] Determine the movement trajectory and size structure information of the dynamic objects in the candidate area, and analyze the degree of influence of the dynamic objects on the tower crane operation in combination with the movement trajectory and size structure information of the dynamic objects. Screen the candidate area based on the degree of influence, and use the screened candidate area as the dynamic object area.

[0060] In this embodiment, the candidate area can be an area with only dynamic objects or an area with both dynamic and static objects. The three-dimensional model of the surrounding environment of the tower crane site is an initial three-dimensional simulation model of position objects. According to the past data of the dynamic objects in the candidate area, the movement trajectory and size structure information are obtained by using image processing and machine learning algorithms. According to the characteristics and safety specifications of tower crane operation, an analysis model for the degree of influence of dynamic objects on tower crane operation is established. This model should comprehensively consider factors such as the type, size, speed, position of the dynamic objects, and the relative distance from the tower crane, and output the degree of influence. Use the candidate area with a higher degree of influence as the dynamic object area, and use the candidate area with a lower degree of influence as the dynamic object area.

[0061] Step S102, analyze the capture requirements in the areas where static and dynamic objects are located respectively, and construct a digital panoramic model of the surrounding environment of the tower crane site according to the capture requirements.

[0062] In this embodiment, different capture requirements are set for different areas, and this capture requirement is used to ensure that the movement or activity of dynamic objects can be accurately and quickly captured.

[0063] In some embodiments of the present application, analyzing the capture requirements in the areas where static and dynamic objects are located respectively includes,

[0064] For the static object area, collect the position and size structure information of each static object in the area, and determine the degree of influence of each static object on the tower crane operation in combination with the position and size structure information, so as to obtain the degree of influence of this static object area on the tower crane operation;

[0065] For the dynamic object area, calculate the average activity speed and trajectory predictability index of each dynamic object in the area. Based on the average activity speed of the dynamic objects, the trajectory predictability index, the impact degree of the dynamic objects on the tower crane operation, and the impact degree of the static objects on the tower crane operation, generate the impact degree of this dynamic object area on the tower crane operation;

[0066]

[0067] where I i1 is the impact degree of the i1-th dynamic object area on the tower crane operation, n1 and n2 are the numbers of dynamic objects and static objects in the i1-th dynamic object area respectively, β1 and β2 are the respective combination weights of the dynamic objects and static objects in the i1-th dynamic object area, are the respective conversion coefficients of the average activity speed and trajectory predictability index of the i2-th dynamic object in the i1-th dynamic object area, are the average activity speed and trajectory predictability index of the i2-th dynamic object in the i1-th dynamic object area respectively, is the impact degree of the i2-th dynamic object in the i1-th dynamic object area on the tower crane operation, k1 is a preset constant, is the impact degree of the i3-th static object in the i1-th dynamic object area on the tower crane operation;

[0068] Set the respective capture requirements for the static object area and the dynamic object area according to the impact degree of the static object area on the tower crane operation and the impact degree of the dynamic object area on the tower crane operation respectively.

[0069] In this embodiment, the position of the static object has a direct impact on the capture detail level. If the static object is located in the key area of the tower crane operation, such as the tower crane foundation, important buildings around the construction area, etc., then a higher detail level is required to accurately represent its position and structure. The shape and size of the static object determine its representation method in the model. For static objects with complex shapes and large sizes, such as large buildings and structures, more details are required to accurately describe their external shapes and features. Combine the two to determine the impact degree of the static object area on the tower crane operation.

[0070] In this embodiment, the motion state and speed of moving objects are the key factors determining the capture detail level and update frequency. For fast-moving objects such as construction workers and vehicles, higher update frequencies and finer details are required to capture their motion trajectories and states. The trajectory predictability index is the predictable situation of the activities of moving objects, and mathematical models can be established to describe the motion of moving objects. These models may be linear, non-linear, or machine learning-based models. Historical data is fitted into the models to evaluate the accuracy and applicability of the models. If the models can fit the historical data well and accurately predict future motion trajectories, then the trajectory predictability of the moving objects is relatively high.

[0071] In this embodiment, It represents the correction of the influence degree of the moving object area on the tower crane operation on the sum of the average activity speed and the trajectory predictability index, so as to obtain a more accurate influence situation of the moving object on the tower crane. If the static object in the dynamic area is 0, then This part of the content is 0. The constant k1 is used to balance the correction of the exponential function.

[0072] In some embodiments of the present application, a digital panoramic model of the surrounding environment of the tower crane site is constructed according to the capture requirements, including

[0073] The static object area and the moving object area are marked on the three-dimensional model of the surrounding environment of the tower crane site, and the grid density of the corresponding positions of the three-dimensional model of the surrounding environment of the tower crane site is processed by meshing according to the capture requirements on the static object area and the moving object area, so as to construct a digital panoramic model of the surrounding environment of the tower crane site.

[0074] In this embodiment, generally, the grid density of the static object area can be relatively low because their shapes and positions are relatively stable and do not require too high precision. The grid density of the moving object area can be relatively high to more accurately capture its motion trajectory and shape changes.

[0075] Step S103, update the digital panoramic model of the surrounding environment of the tower crane site, identify the moving objects in the digital panoramic model, predict the motion trajectories of the moving objects, and mark the motion trajectories of the moving objects in the digital panoramic model.

[0076] In this embodiment, part of the area in the digital panoramic model is updated in real time according to 5G technology to complete real-time monitoring and prediction.

[0077] In some embodiments of the present application, updating the digital panoramic model of the surrounding environment of the tower crane site includes

[0078] For the static object area, collect the on-site environmental data in the area according to a preset cycle, and batch process the on-site environmental data to update the situation of the static object area on the digital panoramic model of the surrounding environment of the tower crane;

[0079] For the dynamic object area, collect the on-site environmental data in the area in real time, and perform real-time processing on the on-site environmental data through 5G technology to update the situation of the dynamic object area on the digital panoramic model of the surrounding environment of the tower crane.

[0080] In this embodiment, for the static object area: Since the position and state of the static object are relatively stable, its update frequency can be relatively low. Unless physical changes occur (such as building demolition, new construction, or tree felling, etc.), the static object information does not need to be updated frequently. For the dynamic object area: The position and state of the dynamic object change in real time, so its update frequency needs to be very high. In order to accurately reflect the dynamic changes of the on-site environment, the dynamic object information needs to be updated continuously and in real time.

[0081] In this embodiment, various sensors (such as lidar, cameras, motion capture systems, etc.) are used to collect the on-site environmental data. The collected data is preprocessed, such as denoising, calibration, etc., to improve the data quality. Utilize the high-speed and low-latency characteristics of 5G technology to transmit the collected data to the data processing center in real time. The large bandwidth characteristic of 5G technology can support the rapid transmission of a large amount of data, meeting the requirements of high-frequency updates in the dynamic object area.

[0082] In some embodiments of this application, identify the dynamic objects in the digital panoramic model and predict the movement trajectories of the dynamic objects, including,

[0083] Divide the foreground part and the background part in the video image of the surrounding environment of the tower crane, extract multiple image features of the dynamic objects in the foreground part, and perform dynamic object recognition in combination with the image features;

[0084] Analyze the image features of the dynamic objects in multiple frames to determine the movement information of the dynamic objects, thereby driving the dynamic object tracking algorithm to track the dynamic objects, and predicting the movement trajectories of the dynamic objects according to the preset trajectory prediction model.

[0085] In this embodiment, the foreground part and the background part are divided from the video image of the surrounding environment of the tower crane. This step is usually achieved through image segmentation techniques, such as segmentation based on features such as color, texture, and motion information. The foreground part usually contains moving objects, while the background part is relatively stable with less change. In the foreground part, multiple image features of the moving objects are extracted, such as shape, color, edge, texture, etc. Combining the extracted image features, machine learning or deep learning algorithms are used for the recognition of moving objects. The recognition result will be used as the basis for subsequent tracking and prediction. Motion information includes speed, position, direction, etc. Preset trajectory prediction models such as convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM), etc. can automatically learn the motion features of moving objects and perform trajectory prediction. These methods are suitable for processing complex video and image data and have high prediction accuracy.

[0086] In some embodiments of the present application, the image features of a moving object in multiple frames are analyzed to determine the motion information of the moving object, and based on this, a moving object tracking algorithm is driven to track the moving object, including,

[0087] Determine the interfering objects in the background part through the image features of the moving object, and analyze the situation of the interfering objects. The situation of the interfering objects includes the number of interfering objects, the type of interfering objects, the distribution of interfering objects, and the similarity of interfering objects;

[0088] Evaluate the interference degree of the background part based on the number of interfering objects, the type of interfering objects, the distribution of interfering objects, and the similarity of interfering objects;

[0089] Comprehensively determine the size of the search area based on the motion information of the moving object and the interference degree of the background part, and drive the moving object tracking algorithm to track the moving object based on the size of the search area.

[0090] In this embodiment, during the tracking process, the interfering objects in the background part are determined through the image features of the moving object. Analyze the number, type, distribution, and similarity of the interfering objects to evaluate their impact on the tracking process. Similar interfering objects are screened out in the background through image features, and based on the number, type, distribution, and similarity of the interfering objects, the interference degree of the background part is evaluated. The higher the interference degree, the greater the interference of the background part on the tracking of the moving object, and a larger area needs to be searched. Comprehensively consider the motion information of the moving object and the interference degree of the background part to determine the size of the search area. Based on the determined size of the search area, drive the moving object tracking algorithm (Kalman filter, particle filter, optical flow method, etc.) to track the moving object more precisely. The search area refers to the search range set around the current frame by the algorithm in order to locate the next frame position of the moving object during the tracking process. The size and shape of this range can be dynamically adjusted according to the motion speed and direction of the moving object, etc.

[0091] Step S104: Plan multiple movement trajectories and registration action parameters of the tower crane according to the working tasks of the tower crane, mark the movement trajectories of the tower crane in the digital panoramic model, compare the movement trajectories of the dynamic objects with those of the tower crane to analyze the collision risk, and select the optimal movement trajectories of the tower crane.

[0092] In this embodiment, for movement trajectory planning: Combining the working tasks and operating environment of the tower crane, use path planning algorithms (such as the A* algorithm, Dijkstra algorithm, etc.) to plan multiple feasible movement trajectories of the tower crane. For determining the registration action parameters: According to the movement trajectories of the tower crane and the operation requirements, determine the registration action parameters (such as lifting height, slewing angle, luffing length, etc.) of the tower crane at each position point. Mark the movement trajectories of the tower crane in the digital panoramic model, compare them with the movement trajectories of the dynamic objects, and analyze the collision risk.

[0093] In some embodiments of the present application, comparing the movement trajectories of the dynamic objects with those of the tower crane to analyze the collision risk includes:

[0094] Perform time alignment processing on the movement trajectories of the dynamic objects and the tower crane, consider the time error factor to determine an error time length, and segment the movement trajectories of the dynamic objects and the tower crane according to the error time length to obtain multiple groups of movement trajectories. Each group of movement trajectories includes a movement trajectory segment of the dynamic object and a movement trajectory segment of the tower crane;

[0095] Uniformly set multiple time marker points on each group of movement trajectories, denote the time marker points of the movement trajectory segment of the dynamic object as the first marker points, and denote the time marker points of the movement trajectory segment of the tower crane as the second marker points. The time of the first marker points and the second marker points is the same and they correspond to each other;

[0096] Calculate the distance between the first marker point and the second marker point at the same time, denoted as the simultaneous distance, calculate the distance between the first marker point and the second marker points at other times on the same group of movement trajectories, and take the average value denoted as the non-simultaneous distance;

[0097] Generate the collision risk level of each group of movement trajectories according to the simultaneous distance and the non-simultaneous distance;

[0098]

[0099] Among them, is the collision risk level of the j1-th group of movement trajectories, m1 is the number of the first marker points or the second marker points under the j1-th group of movement trajectories, γ1 and γ2 are the combined weights of the simultaneous distance and the non-simultaneous distance respectively, σ is the risk conversion coefficient, are the simultaneous distance and the non-simultaneous distance of the j2-th first marker point under the j1-th group of movement trajectories respectively, is The sum of their respective minimum values, where k2 and k3 are preset constants, and [] is the rounding symbol;

[0100] Integrate the collision risk levels of each group of motion trajectories to determine the collision risk level, and describe the collision risk between the motion trajectory of the moving object and the motion trajectory of the tower crane through the collision risk level.

[0101] In this embodiment, since the two trajectories of the motion trajectory of the moving object and the motion trajectory of the tower crane are generated based on data prediction, there may be some errors. Here, time error is considered to segment the trajectories. The time error factors include hardware response speed, data transmission delay, data processing and analysis time, etc. Split the two trajectories into multiple groups of motion trajectories for collision analysis.

[0102] In this embodiment, there are two trajectories on each group of motion trajectories. The time marker points (i.e., time points) on the two trajectories are the same, but only the names are different, in order to distinguish the marks on the trajectories. For example, there are five time marker points t1 - t5 on a certain group of motion trajectories, and each time marker point has two marker points on the two trajectories, which are respectively denoted as the first marker point and the second marker point. At the same time, the distance refers to the distance between the first marker point and the second marker point at a certain time point (the same time) among t1 - t5 (the Euclidean distance or other distances between the two models of the moving object and the tower crane), and the non - synchronous distance is the average value of the distances between the first marker point at time point t1 and each second marker point at time points t2 - t5. It is the distance between different moments, and describing the distance between different moments is the risk.

[0103] It can be understood that the distance has positive and negative values. A positive number indicates that there is a distance between the moving object and the tower crane without contact, and a negative number indicates that the moving object and the tower crane have overlapped or crossed.

[0104] In this embodiment, represents the correction of the sum of the two minimum distances to the sum of the synchronous distance and the non - synchronous distance of this group of motion trajectories. k2 and k3 are preset constants, which represent the balance of the size of the correction function and the balance of the overall risk.

[0105] Step S105, remotely control the tower crane through the registration action parameters of the tower crane and the optimal motion trajectory of the tower crane.

[0106] In this embodiment, finally, send the planned registration action parameters and the optimal motion trajectory of the tower crane to the tower crane control system to achieve remote control of the tower crane. Instruction sending: Convert the registration action parameters and the optimal motion trajectory into control instructions and send them to the tower crane control system through the communication network. Remote control execution: After receiving the instructions, the tower crane control system controls the tower crane to operate according to the planned motion trajectory and registration action parameters.

[0107] It can be understood that the movement trajectories of the above-mentioned moving objects and the tower crane will change in real time as the data is updated, and the risk calculation will also change accordingly, so as to accurately cooperate with the movement of the automated tower crane.

[0108] Correspondingly, the present application also provides a 5G remote control system for tower cranes based on digital panoramic view, as Figure 2 shown, including,

[0109] The first module is used to collect the environmental information around the tower crane site, identify static objects and moving objects in the surrounding environment of the tower crane site, and divide the areas where the static objects and moving objects are located respectively;

[0110] The second module is used to analyze the capture requirements in the areas where the static objects and moving objects are located respectively, and construct a digital panoramic model of the surrounding environment of the tower crane site according to the capture requirements;

[0111] The third module is used to update the digital panoramic model of the surrounding environment of the tower crane site, identify the moving objects in the digital panoramic model, predict the movement trajectories of the moving objects, and mark the movement trajectories of the moving objects in the digital panoramic model;

[0112] The fourth module is used to plan multiple movement trajectories of the tower crane and registration action parameters according to the working tasks of the tower crane, mark the movement trajectories of the tower crane in the digital panoramic model, analyze the collision risk by comparing the movement trajectories of the moving objects and the tower crane, and screen out the optimal movement trajectory of the tower crane;

[0113] The fifth module is used to remotely control the tower crane through the registration action parameters of the tower crane and the optimal movement trajectory of the tower crane.

[0114] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0115] 1. Divide the areas where the static objects and moving objects are located respectively, analyze the moving objects in units of areas, analyze the capture requirements in the areas where the static objects and moving objects are located respectively, construct a digital panoramic model of the surrounding environment of the tower crane site, and perform adaptive grid processing on the model according to the situations of the moving objects and static objects in the area to ensure the adaptability of the capture detail accuracy of the model, and provide a reliable basis for subsequent identification, tracking, prediction of moving objects and collision risk assessment.

[0116] 2. Identify the moving objects in the digital panoramic model, predict the movement trajectories of the moving objects, improve the accuracy of tracking and prediction of the moving objects by setting the search area, analyze the collision risk by comparing the movement trajectories of the moving objects and the tower crane, improve the accuracy and safety of the remote control of the tower crane, reduce the collision risk, and ensure the safe operation of the tower crane.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of the present invention.

[0118] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0119] Those skilled in the art can understand that the modules in the system in the implementation scenario can be distributed in the system of the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more systems different from this implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0120] As described above, only the preferred specific implementation manners of the present invention are given, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered within the protection scope of the present invention.

Claims

1. A 5G tower crane remote control method based on digital panorama, characterized in that: include, Collect environmental information around the tower crane site, identify static objects and dynamic objects in the surrounding environment of the tower crane site, and divide the areas where static objects and dynamic objects are located; Analyze the capture requirements of static objects and dynamic objects in their respective areas, and build a digital panoramic model of the tower crane's surrounding environment based on the capture requirements; Update the digital panoramic model of the tower crane's surrounding environment, identify dynamic objects in the digital panoramic model, predict the movement trajectory of the dynamic objects, and mark the movement trajectory of the dynamic objects in the digital panoramic model; According to the working tasks of the tower crane, multiple motion trajectories of the tower crane are planned and the motion parameters are aligned. The motion trajectories of the tower crane are marked in the digital panoramic model. The motion trajectories of the dynamic objects and the tower crane are compared to analyze the collision risk and select the optimal motion trajectory of the tower crane. The tower crane is remotely controlled through the registration action parameters of the tower crane and the optimal motion trajectory of the tower crane.

2. The 5G tower crane remote control method based on digital panorama according to claim 1 is characterized in that: Identify static and dynamic objects in the surrounding environment of the tower crane site, and divide the areas where static and dynamic objects are located, including: Confirm all categories of objects in the tower crane site surrounding environment, distinguish static objects from dynamic objects, build a three-dimensional model of the tower crane site surrounding environment, collect location information of static objects and location information of dynamic objects, integrate the location information of dynamic objects to form a location information set, and mark the static objects and dynamic objects at corresponding positions in the three-dimensional model of the tower crane site surrounding environment by using the location information of static objects and the location information set of dynamic objects; The area where only static objects exist is recorded as the static object area, and the area where dynamic objects exist is recorded as the candidate area; The activity trajectory and size structure information of the dynamic object in the selected area are determined, and the influence of the dynamic object on the tower crane operation is analyzed in combination with the activity trajectory and size structure information of the dynamic object. The selected area is screened according to the influence degree, and the screened selected area is used as the dynamic object area.

3. The 5G tower crane remote control method based on digital panorama according to claim 2 is characterized in that: Analyze the capture requirements of static and dynamic objects in their respective areas, including: For the static object area, the position and size structure information of each static object in the area is collected, and the influence degree of each static object on the tower crane operation is determined by combining the position and size structure information, so as to obtain the influence degree of the static object area on the tower crane operation; For the dynamic object area, the average activity speed and trajectory predictability index of each dynamic object in the area are calculated, and the influence degree of the dynamic object area on the tower crane operation is generated based on the average activity speed of the dynamic objects, the trajectory predictability index, the influence degree of the dynamic objects on the tower crane operation, and the influence degree of the static objects on the tower crane operation; Among them, I i1 is the influence of the ith dynamic object area on the tower crane operation, n1 and n2 are the number of dynamic objects and the number of static objects in the ith dynamic object area, β1 and β2 are the combined weights of dynamic objects and static objects in the ith dynamic object area, are the conversion coefficients of the average activity speed and trajectory predictability index of the i2th dynamic object in the i1th dynamic object area, respectively, and Q1 i2 、Q2 i2 are the average activity speed and trajectory predictability index of the i2nd dynamic object in the i1th dynamic object area, Q3 i2 is the influence degree of the ith dynamic object in the ith dynamic object area on the tower crane operation, k1 is a preset constant, Q4 i3 is the influence degree of the i3th static object in the i1th dynamic object area on the tower crane operation; The capture requirements of the static object area and the dynamic object area are set according to the influence of the static object area on the tower crane operation and the influence of the dynamic object area on the tower crane operation.

4. The 5G tower crane remote control method based on digital panorama according to claim 2 is characterized in that: Build a digital panoramic model of the tower crane's surrounding environment based on capture requirements, including: The static object area and the dynamic object area are marked on the three-dimensional model of the tower crane's surrounding environment. The grid density of the corresponding positions in the three-dimensional model of the tower crane's surrounding environment is gridded according to the capture requirements of the static object area and the dynamic object area, so as to construct a digital panoramic model of the tower crane's surrounding environment.

5. The 5G tower crane remote control method based on digital panorama according to claim 2 is characterized in that: Update the digital panoramic model of the tower crane site surrounding environment, including: For the static object area, the on-site environmental data in the area is collected according to the preset period, and the on-site environmental data is batch processed to update the static object area situation on the digital panoramic model of the tower crane on-site surrounding environment; For dynamic object areas, on-site environmental data in the area are collected in real time, and the on-site environmental data are processed in real time through 5G technology to update the dynamic object area situation on the digital panoramic model of the tower crane's on-site surrounding environment.

6. The 5G tower crane remote control method based on digital panorama according to claim 1 is characterized in that: Identify dynamic objects in the digital panoramic model and predict their motion trajectory, including: Divide the foreground and background parts of the video image of the tower crane's surrounding environment, extract multiple image features of dynamic objects in the foreground part, and perform dynamic object recognition based on the image features; The image features of the dynamic object in multiple frames are analyzed to determine the motion information of the dynamic object, thereby driving the dynamic object tracking algorithm to track the dynamic object and predict the motion trajectory of the dynamic object according to the preset trajectory prediction model.

7. The 5G tower crane remote control method based on digital panorama according to claim 6 is characterized in that: Analyze the image features of dynamic objects in multiple frames to determine the motion information of dynamic objects, thereby driving the dynamic object tracking algorithm to track the dynamic objects, including: Determine the distractors in the background part through the image features of the dynamic object, and analyze the distractor conditions, including the number of distractors, the type of distractors, the distribution of distractors, and the similarity of distractors; The distraction level of the background part is evaluated based on the number of distractors, distractor type, distractor distribution, and distractor similarity; The size of the search area is determined by comprehensively considering the motion information of the dynamic object and the interference degree of the background part, and the dynamic object tracking algorithm is driven based on the size of the search area to track the dynamic object.

8. The 5G tower crane remote control method based on digital panorama according to claim 1 is characterized in that: Compare the motion trajectory of the dynamic object with the motion trajectory of the tower crane to analyze the collision risk, including: The motion trajectory of the dynamic object and the motion trajectory of the tower crane are time-aligned, a time error factor is considered to determine an error time length, and the motion trajectory of the dynamic object and the motion trajectory of the tower crane are segmented according to the error time length to obtain multiple groups of motion trajectories, each group of motion trajectories includes a motion trajectory segment of the dynamic object and a motion trajectory segment of the tower crane; A plurality of time marking points are evenly set on each group of motion trajectories, the time marking points of the motion trajectory segment of the dynamic object are recorded as first marking points, and the time marking points of the motion trajectory segment of the tower crane are recorded as second marking points, and the time of the first marking point and the second marking point are the same and correspond to each other; Calculate the distance between the first marked point and the second marked point at the same time, record it as the simultaneous distance, calculate the distance between the first marked point on the same set of motion trajectories and the second marked point at other times, take the average value and record it as the asynchronous distance; Generate the collision risk level of each group of motion trajectories based on the simultaneous distance and asynchronous distance; in, is the collision risk level of the j1th group of motion trajectories, m1 is the number of the first or second marking points under the j1th group of motion trajectories, γ1 and γ2 are the combined weights of the simultaneous distance and the asynchronous distance, σ is the risk conversion coefficient, are the simultaneous distance and asynchronous distance of the j2th first marked point under the j1th group of motion trajectories, for The sum of their respective minimum values, k2 and k3 are preset constants, and [] is the rounding symbol; The collision risk level of each set of motion trajectories is integrated to determine the collision risk level, and the collision risk between the motion trajectory of the dynamic object and the motion trajectory of the tower crane is described by the collision risk level.

9. The 5G tower crane remote control system based on digital panorama is characterized by: include, The first module is used to collect environmental information around the tower crane site, identify static objects and dynamic objects in the surrounding environment of the tower crane site, and divide the areas where the static objects and dynamic objects are located; The second module is used to analyze the capture requirements of static objects and dynamic objects in their respective areas, and to build a digital panoramic model of the tower crane's surrounding environment based on the capture requirements; The third module is used to update the digital panoramic model of the surrounding environment of the tower crane site, identify dynamic objects in the digital panoramic model, predict the movement trajectory of the dynamic objects, and mark the movement trajectory of the dynamic objects in the digital panoramic model; The fourth module is used to plan multiple tower crane motion trajectories and register motion parameters according to the tower crane's work tasks, mark the tower crane's motion trajectory in the digital panoramic model, compare the motion trajectory of the dynamic object with the tower crane's motion trajectory to analyze the collision risk, and select the optimal tower crane motion trajectory; The fifth module is used to remotely control the tower crane through the registration action parameters of the tower crane and the optimal motion trajectory of the tower crane.

Images