Modeling method, modeling system and remote operation system for lifting operation

Abstract

The application provides a hoisting operation modeling method, a modeling system and a remote operation system. The method comprises the following steps: acquiring a first characteristic value and a second characteristic value of a target object, wherein the first characteristic value is a characteristic collected by a UAV, and the second characteristic value is a characteristic collected by a collection terminal; constructing a digital twin model according to the first characteristic value; and planning a hoisting decision according to the second characteristic value and the digital twin model. The application provides a hoisting operation modeling method, a modeling system and a remote operation system. Multi-dimensional data collection is performed by using a collection terminal and a UAV, the visual field of a work area is fully covered, a monitoring dead angle is avoided, and a hoisting strategy is formulated according to the monitored data, so that the hoisting operation requirements in different scenes are met.

Description

Technical Field

[0001] This invention relates to the field of engineering machinery technology, and in particular to a modeling method, modeling system and remote operating system for lifting operations. Background Technology

[0002] As an important engineering facility, cranes can move and transport large materials vertically and horizontally through actions such as lifting, luffing, and slewing. They can be widely used in construction sites and other scenarios. Traditional cranes require an operator in the control room to operate them, which depends on the operator's skills and experience and has a low level of intelligence.

[0003] To improve the intelligence level of hoisting equipment, existing tracking and monitoring of intelligent cranes relies on cameras and sensors installed on the crane boom and other locations to track and monitor the operation process, status parameters, and environmental parameters. However, there are blind spots in the field of view and monitoring range, resulting in incomplete information collection and biased judgments on the crane operation process. Summary of the Invention

[0004] This invention provides a modeling method for lifting operations to address the shortcomings of existing lifting operation monitoring systems that cannot meet operational needs and pose safety hazards. By collecting data from terminals and drones in multiple dimensions, it achieves full coverage of the operational area, avoids blind spots in monitoring, and formulates lifting strategies based on the monitored data to meet the lifting operation needs in different scenarios.

[0005] The present invention also provides a modeling system for lifting operations.

[0006] The present invention also provides a remote operating system.

[0007] A modeling method for lifting operations according to a first aspect of the present invention includes: an application to a server, the server being connected to lifting equipment, a drone, and a data acquisition terminal within the operation area, the lifting equipment being used to lift a target object within the operation area, and the data acquisition terminal being disposed on the lifting equipment;

[0008] The method includes:

[0009] Obtain a first feature value and a second feature value of the target object, wherein the first feature value is a feature collected by the UAV and the second feature value is a feature collected by the acquisition terminal;

[0010] Construct a digital twin model based on the first feature value;

[0011] The hoisting decision is planned based on the second feature value and the digital twin model.

[0012] According to one embodiment of the present invention, the step of obtaining the first feature value corresponding to the target object specifically includes:

[0013] Obtain a first feature parameter and a second feature parameter corresponding to the target object, wherein the first feature parameter is an image parameter acquired by the UAV, and the second feature parameter is a depth parameter acquired by the UAV;

[0014] The first feature value is generated based on the first feature parameter and the second feature parameter.

[0015] Specifically, this embodiment provides an implementation method for obtaining the first feature value corresponding to the target object. By generating the first feature value based on the first feature parameters and second feature parameters collected by the UAV, a foundation is provided for the construction of a virtual model of the lifting equipment and the surrounding working environment.

[0016] According to one embodiment of the present invention, the step of obtaining the second feature value corresponding to the target object specifically includes:

[0017] The third feature parameter and the fourth feature parameter corresponding to the target object are obtained. The third feature parameter is the image parameter acquired by the acquisition terminal, and the fourth feature parameter is the depth parameter acquired by the acquisition terminal. The acquisition direction of the acquisition terminal and the acquisition direction of the UAV form an acquisition angle.

[0018] The second feature value is generated based on the third feature parameter and the fourth feature parameter.

[0019] Specifically, this embodiment provides an implementation method for obtaining a second feature value corresponding to the target object. The second feature value is generated based on the third and fourth feature parameters collected by the acquisition terminal, providing a basis for the construction of a virtual model of the lifting equipment and the surrounding working environment.

[0020] According to one embodiment of the present invention, the step of constructing a digital twin model based on the first feature value specifically includes:

[0021] The first feature vector and the second feature vector collected by the UAV are obtained, wherein the first feature vector points to the regional features of the lifting equipment and the second feature vector points to the regional features of the target object;

[0022] A first lifting equipment model is generated based on the first feature vector, and a prefabricated lifting equipment model is obtained based on the first lifting equipment model.

[0023] The first lifting equipment model is coupled with the prefabricated lifting equipment model to generate a second lifting equipment model;

[0024] A regional environment model is generated based on the second feature vector;

[0025] The digital twin model is constructed based on the second lifting equipment model and the regional environment model.

[0026] Specifically, this embodiment provides an implementation method for constructing a digital twin model based on the first feature value. By generating a first lifting equipment model based on the first feature value collected by the UAV and coupling it with a prefabricated lifting equipment model, the accuracy of the digital twin model is improved.

[0027] According to one embodiment of the present invention, the step of generating a first lifting equipment model based on the first feature vector and obtaining a prefabricated lifting equipment model based on the first lifting equipment model specifically includes:

[0028] Obtain the coordinate parameters, contour parameters, and model parameters of the lifting equipment;

[0029] The first lifting equipment model is generated based on the coordinate parameters, the contour parameters, and the model parameters;

[0030] The corresponding prefabricated lifting equipment model is determined based on the coordinate parameters, the contour parameters, and the model parameters.

[0031] Specifically, this embodiment provides an implementation method for obtaining a prefabricated lifting equipment model based on the first lifting equipment model. By obtaining the coordinate parameters, contour parameters, and model parameters of the lifting equipment, it is convenient to establish the first lifting equipment model and determine the prefabricated lifting equipment model, thus providing support for improving the accuracy of the first lifting equipment model through the prefabricated lifting equipment model.

[0032] According to one embodiment of the present invention, the step of generating a regional environment model based on the second feature vector specifically includes:

[0033] The first region morphological parameters and the second region morphological parameters collected by the UAV are obtained, wherein the first region morphological parameters correspond to the hoisting start position parameters of the target object, and the second region morphological parameters correspond to the hoisting end position parameters of the target object;

[0034] Acquire hoisting environment parameters, and generate the regional environment model based on the hoisting environment parameters, the first regional morphology parameters, and the second regional morphology parameters, wherein the hoisting environment parameters include at least the wind speed, wind level, wind pressure, and obstacles within the work area.

[0035] Specifically, this embodiment provides an implementation method for generating a regional environment model based on the second feature vector. The starting and ending points of the target object in the work area are determined by the hoisting start position parameters and hoisting end position parameters, and a regional environment model is generated in conjunction with the hoisting environment parameters, thereby improving the safety and accuracy of hoisting operations.

[0036] According to one embodiment of the present invention, the step of planning hoisting decisions based on the second feature value and the digital twin model specifically includes:

[0037] The third feature vector and the fourth feature vector collected by the acquisition terminal are obtained. The third feature vector points to the action characteristics of the lifting equipment, and the fourth feature vector points to the regional characteristics of the target object.

[0038] The digital twin model is driven in real time based on the third feature vector and the fourth feature vector to display the actions of the lifting equipment in the corresponding actual environment in real time in the digital twin space.

[0039] Specifically, this embodiment provides an implementation method for planning hoisting decisions based on the second feature value and the digital twin model. By driving the digital twin model in real time based on the second feature value collected by the acquisition terminal, hoisting decision planning is realized.

[0040] According to one embodiment of the present invention, after the step of driving the digital twin model in real time based on the third feature vector and the fourth feature vector to display the actions of the lifting equipment in the corresponding actual environment in real time in the digital twin space, the method further includes:

[0041] To obtain the appropriate hoisting strategy;

[0042] The lifting decision is planned according to the lifting strategy.

[0043] According to one embodiment of the present invention, after the step of planning the hoisting decision based on the second feature value and the digital twin model, the specific steps include:

[0044] Extract the preset lifting points of the target object from the lifting decision;

[0045] A mimicry hoisting trajectory corresponding to the target object is generated based on the morphological parameters of the first region and the morphological parameters of the second region.

[0046] An environmental digital mode is generated based on the hoisting environment parameters;

[0047] Based on the preset hoisting points, the mimicry hoisting trajectory, and the environmental digital modality, a mimicry hoisting decision is generated corresponding to the target object, and the mimicry hoisting decision is executed.

[0048] Once the mimicry hoisting decision is completed and the hoisting of the target object meets the preset hoisting conditions, the hoisting decision is sent to the lifting equipment.

[0049] Specifically, this embodiment provides an implementation method after planning the hoisting decision based on the second feature value and the digital twin model. By executing the mimicry hoisting decision, the hoisting decision is simulated before execution, ensuring the safety of the hoisting. Problems can also be discovered and resolved in a timely manner during the simulation, improving hoisting efficiency and reducing the existence of hoisting errors and the occurrence of accidents.

[0050] According to one embodiment of the present invention, after the step of planning the hoisting decision based on the second feature value and the digital twin model, the specific steps include:

[0051] Within a continuous time acquisition node, the motion trajectory characteristics of the target object are acquired;

[0052] The instantaneous hoisting trajectory of the target object is generated based on the motion trajectory characteristics, and a judgment is made based on the instantaneous hoisting trajectory and the simulated hoisting trajectory.

[0053] If the deviation rate between the real-time hoisting trajectory and the simulated hoisting trajectory is determined to be greater than the deviation threshold, an alarm is issued.

[0054] Specifically, this embodiment provides another implementation method after planning the hoisting decision based on the second feature value and the digital twin model. By acquiring the motion trajectory characteristics of the target object, the real-time hoisting trajectory of the target object is generated, thereby realizing the monitoring of the hoisting decision of the target object and improving the safety of the hoisting operation.

[0055] According to a second aspect of the present invention, a modeling system for lifting operations is provided, which employs the above-described modeling method for lifting operations.

[0056] According to a third aspect of the present invention, a remote operating system further includes: a remote control platform, wherein the remote control platform is connected to a server, a lifting equipment, a drone and a data acquisition terminal respectively, so as to realize remote planning and hoisting decisions;

[0057] The server contains the aforementioned modeling system for lifting operations.

[0058] Specifically, this embodiment provides an implementation method for a remote control platform, which enables remote control operations by setting up a digital twin space between the remote control platform and the server.

[0059] According to one embodiment of the present invention, it further includes: a cloud control platform, which is connected to multiple work areas to realize the planning and collaborative operation of multiple hoisting decisions;

[0060] Each of the aforementioned work areas includes the remote control platform, the server, the lifting equipment, the drone, and the data acquisition terminal.

[0061] Specifically, this embodiment provides an implementation method for a cloud control platform. By setting up a cloud control platform, it is possible to realize the configuration management of permissions and rules for drivers, vehicles, and remote operation platforms, as well as comprehensive scheduling and monitoring of many-to-many relationships.

[0062] The above-mentioned one or more technical solutions in this invention have at least one of the following technical effects: The modeling method, modeling system and remote operating system for lifting operations provided by this invention achieve full coverage of the field of vision of the operation area by collecting data from a terminal and a drone, avoiding blind spots in monitoring, and formulating lifting strategies based on the monitored data to meet the lifting operation needs in different scenarios.

[0063] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0065] Figure 1 This is a flowchart illustrating the modeling method for lifting operations provided by the present invention;

[0066] Figure 2 This is one of the schematic diagrams of the layout relationship of the modeling system for lifting operations provided by the present invention;

[0067] Figure 3 This is the second schematic diagram of the layout relationship of the modeling system for lifting operations provided by this invention.

[0068] Figure label:

[0069] 10. Server; 20. Drone; 30. Data acquisition terminal; 40. Work area; 50. Remote control platform; 60. Cloud control platform. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] In the description of the embodiments of the present invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of the present invention. In addition, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0072] In some specific embodiments of the present invention, such as Figure 1 As shown, this solution provides a modeling method for lifting operations, including: an application to a server 10, the server 10 being connected to lifting equipment, a drone 20 and a data acquisition terminal 30 within the work area 40, the lifting equipment being used to hoist target objects within the work area 40, and the data acquisition terminal 30 being installed on the lifting equipment;

[0073] The methods include:

[0074] Acquire the first feature value and the second feature value of the target object, wherein the first feature value is the feature collected by the UAV 20 and the second feature value is the feature collected by the acquisition terminal 30;

[0075] Construct a digital twin model based on the first eigenvalue;

[0076] Lifting decisions are planned based on the second eigenvalue and the digital twin model.

[0077] In detail, the present invention provides a modeling method for lifting operations to address the shortcomings of existing lifting operation monitoring systems that cannot meet operational needs and pose safety hazards. By collecting data from multiple dimensions through a data acquisition terminal 30 and a drone 20, the method achieves full coverage of the operational area 40, avoiding blind spots in monitoring. Furthermore, it formulates lifting strategies based on the monitored data to meet the lifting operation needs in different scenarios.

[0078] In some possible embodiments of the present invention, the step of obtaining the first feature value of the corresponding target object specifically includes:

[0079] Obtain the first feature parameter and the second feature parameter of the corresponding target object, wherein the first feature parameter is the image parameter acquired by the UAV 20, and the second feature parameter is the depth parameter acquired by the UAV 20;

[0080] The first feature value is generated based on the first feature parameter and the second feature parameter.

[0081] Specifically, this embodiment provides an implementation method for obtaining the first feature value of the corresponding target object. The first feature value is generated based on the first feature parameters and the second feature parameters collected by the UAV 20, which provides a basis for the construction of a virtual model of the lifting equipment and the surrounding working environment.

[0082] It should be noted that the first feature parameter collected by the UAV 20 includes the image parameters of the lifting equipment and the target object, and the second feature parameter includes the depth parameters of the lifting equipment and the target object.

[0083] In some possible embodiments of the present invention, the step of obtaining the second feature value of the corresponding target object specifically includes:

[0084] The third and fourth feature parameters of the corresponding target object are obtained. The third feature parameter is the image parameter acquired by the acquisition terminal 30, and the fourth feature parameter is the depth parameter acquired by the acquisition terminal 30. The acquisition direction of the acquisition terminal 30 and the acquisition direction of the UAV 20 form an acquisition angle.

[0085] The second feature value is generated based on the third and fourth feature parameters.

[0086] Specifically, this embodiment provides an implementation method for obtaining the second feature value of the corresponding target object. The second feature value is generated based on the third and fourth feature parameters collected by the acquisition terminal 30, which provides a basis for the construction of a virtual model of the lifting equipment and the surrounding working environment.

[0087] It should be noted that the third feature parameter collected by the acquisition terminal 30 includes the image parameters of the lifting equipment and the target object, and the fourth feature parameter includes the depth parameters of the lifting equipment and the target object.

[0088] In some possible embodiments of the present invention, the step of constructing a digital twin model based on a first feature value specifically includes:

[0089] The first feature vector and the second feature vector collected by the UAV 20 are obtained. The first feature vector points to the regional features of the lifting equipment, and the second feature vector points to the regional features of the target object.

[0090] A first lifting equipment model is generated based on the first feature vector, and a prefabricated lifting equipment model is obtained based on the first lifting equipment model.

[0091] The first lifting equipment model is coupled with the prefabricated lifting equipment model to generate the second lifting equipment model;

[0092] Generate a regional environment model based on the second feature vector;

[0093] A digital twin model was constructed based on the second lifting equipment model and the regional environment model.

[0094] Specifically, this embodiment provides an implementation method for constructing a digital twin model based on a first feature value. By generating a first lifting equipment model based on the first feature value collected by the UAV 20 and coupling it with a prefabricated lifting equipment model, the accuracy of the digital twin model is improved.

[0095] In a possible implementation, the first crane model generated by collecting the first feature value through the drone 20 has poor accuracy and is difficult to meet the simulation requirements of the digital twin model. The pre-built crane model is a model of the actual crane constructed at a proportional scale in the digital twin space. Digital twin refers to the establishment of a model corresponding to reality in a digital space scene. The positional relationship, connection relationship, motion relationship and shape parameters of the actual physical parts can be accurately displayed through the digital twin model.

[0096] In some possible embodiments of the present invention, the steps of generating a first lifting equipment model based on a first feature vector and obtaining a prefabricated lifting equipment model based on the first lifting equipment model specifically include:

[0097] Obtain the coordinate parameters, contour parameters, and model parameters of the lifting equipment;

[0098] Generate the first crane model based on coordinate parameters, contour parameters, and model parameters;

[0099] The corresponding prefabricated lifting equipment model is determined based on the coordinate parameters, contour parameters, and model parameters.

[0100] Specifically, this embodiment provides an implementation method for obtaining a prefabricated lifting equipment model based on a first lifting equipment model. By obtaining the coordinate parameters, contour parameters, and model parameters of the lifting equipment, it is convenient to establish the first lifting equipment model and determine the prefabricated lifting equipment model, thus providing support for improving the accuracy of the first lifting equipment model through the prefabricated lifting equipment model.

[0101] In some possible embodiments of the present invention, the step of generating a regional environment model based on the second feature vector specifically includes:

[0102] Acquire the first region morphological parameters and the second region morphological parameters collected by the UAV 20, wherein the first region morphological parameters correspond to the hoisting start position parameters of the target object, and the second region morphological parameters correspond to the hoisting end position parameters of the target object;

[0103] Acquire hoisting environment parameters, and generate a regional environment model based on the hoisting environment parameters, the first region morphology parameters, and the second region morphology parameters. The hoisting environment parameters include at least the wind speed, wind level, wind pressure, and obstacles within the work area 40.

[0104] Specifically, this embodiment provides an implementation method for generating a regional environment model based on a second feature vector. The starting and ending points of the target object within the work area 40 are determined by the hoisting start position parameters and hoisting end position parameters. The regional environment model is generated in conjunction with the hoisting environment parameters, thereby improving the safety and accuracy of the hoisting operation.

[0105] In a possible implementation, the first region morphological parameter and the second region morphological parameter are position parameters that are manually input.

[0106] In a possible implementation, the first region morphological parameter and the second region morphological parameter are pre-marked working area 40 parameters.

[0107] In a possible implementation, the morphological parameters of the first and second regions are input into the digital twin model by marking the corresponding coordinates, thereby realizing the planning of the corresponding hoisting decisions.

[0108] In a possible implementation, by loading hoisting environment parameters, including wind speed, wind level, and wind pressure within the work area 40, into the area environment model, the area environment model can more realistically simulate the environment of the work area 40.

[0109] In a possible implementation, the hoisting environment parameters can be obtained by loading the average value within a continuous data collection time point to ensure the accuracy of the environmental simulation.

[0110] In a possible implementation, the acquisition of hoisting environment parameters can be updated in real time based on the collected data to ensure the immediacy of the environmental simulation.

[0111] In a possible implementation, the acquisition of hoisting environment parameters can be achieved by setting corresponding parameters, i.e., simulating a regional environment model under specific conditions, such as extreme environments like strong winds, heavy rain, and snow. This simulation of specific environments is beneficial for providing data support for planning hoisting strategies under extreme conditions.

[0112] In some possible embodiments of the present invention, the step of planning hoisting decisions based on the second feature value and the digital twin model specifically includes:

[0113] The third feature vector and the fourth feature vector collected by the acquisition terminal 30 are obtained. The third feature vector points to the action features of the lifting equipment, and the fourth feature vector points to the regional features of the target object.

[0114] The digital twin model is driven in real time based on the third and fourth feature vectors to display the actions of the lifting equipment in the corresponding actual environment in real time in the digital twin space.

[0115] Specifically, this embodiment provides an implementation method for planning hoisting decisions based on a second feature value and a digital twin model. By using the second feature value collected by the acquisition terminal 30 to drive the digital twin model in real time, hoisting decision planning is realized.

[0116] In a possible implementation, real-time driving means that the model's movements in the digital twin space are consistent with the crane's movements in the actual scene, and the motion relationships, positional relationships, and connection relationships between the crane model components in the digital twin space are adjusted in real time based on the detected data.

[0117] In a possible implementation, if the boom is detected to rotate 10° in the actual scene by the sensor, the corresponding boom rotation of 10° will also be detected in the corresponding digital twin space. If the crane is detected to rotate 5°, the corresponding crane model in the twin space will also rotate 5°. If the crane is detected to travel 10m, the corresponding crane model in the digital twin space will travel 10m. If the hook is detected to rise 15m, the corresponding hook in the twin space will also rise 15m.

[0118] In a possible implementation, the data acquisition terminal 30 includes an angle sensor, a temperature sensor, and a wind speed sensor, etc., to realize the acquisition angle, acquisition position, adjustment of the acquisition angle, and measurement of wind speed.

[0119] In a possible implementation, the digital twin model uses spatial markers in the planning of lifting strategies. These spatial markers are divided into work coordination points and path smoothing points. Work coordination points are locations where the crane needs to wait for coordination instructions after the hook's automated operation arrives. Path smoothing points are path intervention points where obstacles are considered to need to be avoided during work planning. The work planning simulation currently only considers the coordinated movement of the crane, boom, cable, and hook along the planned path.

[0120] In some possible embodiments of the present invention, after the step of driving the digital twin model in real time based on the third feature vector and the fourth feature vector to display the actions of the lifting equipment in the corresponding actual environment in real time in the digital twin space, the invention further includes:

[0121] To obtain the appropriate hoisting strategy;

[0122] Lifting decisions are planned based on the lifting strategy.

[0123] In some possible embodiments of the present invention, after the step of planning the hoisting decision based on the second feature value and the digital twin model, the specific steps include:

[0124] Extracting the preset lifting points of the target object for lifting equipment in the lifting decision-making process;

[0125] Generate a mimicry hoisting trajectory for the target object based on the morphological parameters of the first and second regions.

[0126] Generate environmental digital modes based on hoisting environment parameters;

[0127] Based on the preset hoisting points, the mimicry hoisting trajectory, and the digital modality of the environment, a mimicry hoisting decision is generated for the corresponding target object, and the mimicry hoisting decision is executed.

[0128] Once the mimicry hoisting decision is completed and the hoisting of the target object meets the preset hoisting conditions, the hoisting decision is sent to the hoisting equipment.

[0129] Specifically, this embodiment provides an implementation method after planning the hoisting decision based on the second feature value and the digital twin model. By executing the mimicry hoisting decision, the hoisting decision is simulated before execution, ensuring the safety of the hoisting. Problems can also be discovered and resolved in a timely manner during the simulation, improving hoisting efficiency and reducing the existence of hoisting errors and the occurrence of accidents.

[0130] In some possible embodiments of the present invention, after the step of planning the hoisting decision based on the second feature value and the digital twin model, the specific steps include:

[0131] Within a continuous time acquisition node, the motion trajectory characteristics of the target object are obtained;

[0132] The instantaneous hoisting trajectory of the target object is generated based on the motion trajectory characteristics, and a judgment is made based on the instantaneous hoisting trajectory and the simulated hoisting trajectory.

[0133] An alarm is issued if the deviation rate between the real-time hoisting trajectory and the simulated hoisting trajectory is greater than the deviation threshold.

[0134] Specifically, this embodiment provides another implementation method after planning the hoisting decision based on the second feature value and the digital twin model. By acquiring the motion trajectory characteristics of the target object, the real-time hoisting trajectory of the target object is generated, thereby realizing the monitoring of the hoisting decision of the target object and improving the safety of the hoisting operation.

[0135] In a possible implementation, multiple instantaneous nodes of the motion trajectory features of the target object are extracted, and an instantaneous hoisting trajectory of the target object is generated based on the multiple instantaneous nodes.

[0136] In some specific embodiments of the present invention, such as Figures 1 to 3 As shown, this solution provides a modeling system for lifting operations, which uses the above-mentioned modeling method for lifting operations.

[0137] In some specific embodiments of the present invention, this solution provides a remote operating system, which further includes a remote control platform 50. The remote control platform 50 is connected to the server 10, the lifting equipment, the drone 20 and the data acquisition terminal 30 respectively, so as to realize remote planning and hoisting decisions.

[0138] The server 10 contains the aforementioned modeling system for lifting operations.

[0139] Specifically, this embodiment provides an implementation of a remote control platform 50, which performs remote control operations by cooperating with the digital twin space of the server 10.

[0140] In a possible implementation, a display is also included, showing the digital twin space on server 10, graphics and scenes captured in real time by a camera.

[0141] In a possible implementation, the remote control platform 50 controls the crane by connecting to onboard motion controllers, such as a boom controller (which controls the movement of the boom) or a chassis controller (which controls the movement of the chassis).

[0142] In a possible implementation, the remote control platform 50 includes remote control software for crawler cranes and multi-channel video playback software, etc.

[0143] In a possible implementation, the data acquisition terminal 30 is a data acquisition device installed on the crane, such as a camera, radar, or various sensors.

[0144] In some possible embodiments of the present invention, it further includes: a cloud control platform 60, which is connected to multiple work areas 40 to realize the planning and collaborative operation of multiple hoisting decisions; wherein each work area 40 has a remote control platform 50, a server 10, hoisting equipment, a drone 20 and a data acquisition terminal 30.

[0145] Specifically, this embodiment provides an implementation method for a cloud control platform 60. By setting up the cloud control platform 60, the configuration management of permissions and rules for drivers, vehicles, and remote operation platforms can be realized, as well as the comprehensive scheduling and monitoring of many-to-many relationships.

[0146] In a possible implementation, the crane's movements in the actual scene can be controlled via a cloud control platform 60, or via a remote control platform 50.

[0147] In a possible implementation, the cloud control platform 60 has greater management authority than the remote control platform 50.

[0148] In a possible implementation, the cloud control platform 60 is equivalent to a platform with access control functions. For example, multiple drivers can drive vehicles through multiple remote control platforms 50. Different drivers have different permissions. For example, the driver with the highest driving permission can operate multiple vehicles, while the driver with a lower level can only operate one or a few vehicles. Taking a crane as an example, different types of cranes generally refer to cranes with different lifting capacities. The higher the level, the larger the crane can be operated, while the lower the level cannot.

[0149] In one application scenario, the modeling system for lifting operations is generally divided into two main parts: the vehicle-side (tracked crane) and the remote-side (command center). A digital twin system is used to construct a digital twin of the tracked crane and its operating environment. The tracked crane is pre-modeled and parameter-bound, while the operating environment is constructed using a UAV for on-site surveying and mapping. The digital twin content is presented using real-time 3D rendering.

[0150] Furthermore, the application system of the vehicle-mounted (tracked crane) is divided into three parts: intelligent boom, intelligent vehicle-mounted and UAV-based surveying.

[0151] Furthermore, the intelligent boom mainly consists of an intelligent gimbal, lidar, vision camera, power supply module, IMU, RTK, and AI computing unit. It is mainly used for sensing and calculating the boom, hook, and load to obtain the precise spatial status of the boom, hook, and load.

[0152] Furthermore, the intelligent vehicle-mounted system mainly consists of actuators such as chassis controllers and upper boom controllers for remote control of the crawler crane, as well as 5G network equipment, video gateways, and vehicle-mounted switches for network communication, and panoramic cameras and IMU inertial navigation systems for crawler crane perception, which together realize the remote control of the crawler crane.

[0153] Furthermore, the UAV mapping system mainly consists of UAV 20, high-precision mapping module, aerial survey 3D mapping computer, etc., and is used for terrain mapping and scene construction at the operation site of tracked cranes.

[0154] Furthermore, the remote (command center) application system is divided into a digital twin system, a remote control platform 50, and a cloud control platform 60. The digital twin system is mainly used for high-precision, high-real-time monitoring of crawler crane operations, and uses operation planning simulation as a demonstration to lay the foundation for future intelligent operations. The remote control platform 50 consists of a crawler crane control console and a multi-channel display system, used for remotely controlling the crawler crane operation.

[0155] Furthermore, the cloud control platform 60 enables configuration management of permissions and rules for drivers, vehicles, and remote operation platforms, as well as comprehensive scheduling and monitoring of many-to-many relationships.

[0156] In the description of the embodiments of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention according to the specific circumstances.

[0157] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "method," "specific method," or "some methods," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or method is included in at least one embodiment or method of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or method. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or methods. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or methods described in this specification, as well as the features of different embodiments or methods.

[0158] Finally, it should be noted that the above embodiments are only for illustrating the present invention and not for limiting the present invention. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention and should be covered within the scope of the claims of the present invention.

Claims

1. A method of modeling a lifting operation, characterized by, The application relates to a method for constructing a digital twin model of a target object and a server (10) applied to the method. The method comprises the following steps: obtaining a first characteristic value and a second characteristic value of the target object, wherein the first characteristic value is a characteristic collected by a UAV (20), and the second characteristic value is a characteristic collected by a collection terminal (30); constructing a digital twin model according to the first characteristic value; planning a hoisting decision according to the second characteristic value and the digital twin model, specifically comprising the following steps: obtaining a third characteristic vector and a fourth characteristic vector collected by the collection terminal (30), wherein the third characteristic vector is directed to an action characteristic of the hoisting device, and the fourth characteristic vector is directed to a region characteristic of the target object; driving the digital twin model in real time according to the third characteristic vector and the fourth characteristic vector to display the action of the hoisting device in the corresponding actual environment in a digital twin space in real time; in the step of constructing the digital twin model according to the first characteristic value, specifically comprising the following steps: obtaining a first characteristic vector and a second characteristic vector collected by the UAV (20), wherein the first characteristic vector is directed to a region characteristic of the hoisting device, and the second characteristic vector is directed to a region characteristic of the target object; generating a first hoisting device model according to the first characteristic vector, and obtaining a prefabricated hoisting device model according to the first hoisting device model; coupling the first hoisting device model and the prefabricated hoisting device model to generate a second hoisting device model; generating a region environment model according to the second characteristic vector; constructing the digital twin model according to the second hoisting device model and the region environment model; in the step of generating the region environment model according to the second characteristic vector, specifically comprising the following steps: obtaining a first region shape parameter and a second region shape parameter collected by the UAV (20), wherein the first region shape parameter corresponds to a hoisting starting position parameter of the target object, and the second region shape parameter corresponds to a hoisting ending position parameter of the target object; obtaining a hoisting environment parameter, and generating the region environment model according to the hoisting environment parameter, the first region shape parameter and the second region shape parameter, wherein the hoisting environment parameter at least comprises a wind speed, a wind level, a wind pressure and an obstacle in the working region (40). in the step of obtaining the first characteristic value corresponding to the target object, specifically comprising the following steps:

2. The modeling method of a hoisting job according to claim 1, characterized in that, obtaining a first characteristic parameter and a second characteristic parameter corresponding to the target object, wherein the first characteristic parameter is an image parameter collected by the UAV (20), and the second characteristic parameter is a depth parameter collected by the UAV (20); generating the first characteristic value according to the first characteristic parameter and the second characteristic parameter. in the step of obtaining the second characteristic value corresponding to the target object, specifically comprising the following steps:

3. The modeling method of a hoisting job according to claim 2, characterized in that, ​ acquire a third characteristic parameter and a fourth characteristic parameter corresponding to the target object, the third characteristic parameter being an image parameter acquired by the acquisition terminal (30), and the fourth characteristic parameter being a depth parameter acquired by the acquisition terminal (30), wherein the acquisition direction of the acquisition terminal (30) and the acquisition direction of the unmanned aerial vehicle (20) form an acquisition included angle; generate the second characteristic value according to the third characteristic parameter and the fourth characteristic parameter.

4. The modeling method of a hoisting job according to claim 1, characterized in that, In the steps of generating a first hoisting equipment model according to the first characteristic vector, and acquiring a prefabricated hoisting equipment model according to the first hoisting equipment model, specifically comprising: acquiring coordinate parameters, contour parameters and model parameters of the hoisting equipment; generating the first hoisting equipment model according to the coordinate parameters, the contour parameters and the model parameters; determining the corresponding prefabricated hoisting equipment model according to the coordinate parameters, the contour parameters and the model parameters.

5. The modeling method of a hoisting job according to claim 1, characterized in that, After the step of planning a hoisting decision according to the second characteristic value and the digital twin model, specifically comprising: extracting a preset hoisting point of the hoisting equipment hoisting the target object in the hoisting decision; generating a quasi-state hoisting trajectory corresponding to the target object according to the first regional morphological parameter and the second regional morphological parameter; generating an environmental digital mode according to the hoisting environment parameter; generating a quasi-state hoisting decision corresponding to the target object according to the preset hoisting point, the quasi-state hoisting trajectory and the environmental digital mode.

6. The modeling method of a hoisting job according to claim 5, characterized in that, After the step of generating a quasi-state hoisting decision corresponding to the target object according to the preset hoisting point, the quasi-state hoisting trajectory and the environmental digital mode, specifically further comprising: executing the quasi-state hoisting decision; determining whether the hoisting of the target object satisfies a preset hoisting condition after the execution of the quasi-state hoisting decision is completed, and then sending the hoisting decision to the hoisting equipment.

7. The modeling method of a hoisting job according to claim 5, characterized in that, After the step of planning a hoisting decision according to the second characteristic value and the digital twin model, specifically comprising: acquiring a motion trajectory feature of the target object in a continuous time acquisition node; generating an instant hoisting trajectory of the target object according to the motion trajectory feature, and judging the instant hoisting trajectory and the quasi-state hoisting trajectory; determining whether the offset rate between the instant hoisting trajectory and the quasi-state hoisting trajectory is greater than an offset threshold, and then issuing an alarm.

8. A modeling system for a lifting operation, characterized by The hoisting operation modeling method of any one of claims 1 to 7 is used for modeling.

9. A teleoperation system, characterized by, Further comprising: a remote control platform (50) connected with the server (10), the hoisting equipment, the unmanned aerial vehicle (20) and the acquisition terminal (30) respectively, so as to realize remote planning of the hoisting decision; wherein the server (10) has the hoisting operation modeling system of claim 8.

10. The teleoperation system of claim 9, wherein, Further comprising: a cloud control platform (60) connected with a plurality of operation regions (40) to realize planning and collaborative operation of a plurality of hoisting decisions. Further comprising: Each of the job areas (40) has the remote control platform (50), the server (10), the hoisting equipment, the unmanned aerial vehicle (20) and the collection terminal (30).

Images