Crane collision early warning method and device based on digital twin platform

Abstract

The invention discloses a crane collision early warning method and device based on a digital twin platform, and the method specifically comprises the following steps: extracting feature vectors of target crane equipment and an existing environment nearby the target crane equipment, importing the extracted features into a platform, and sequentially building a crane equipment model and a model of the existing environment nearby the target crane equipment; the overturning influence range of the horizontal plane and the space plane of the crane equipment in the current state is calculated according to the model; performing highlight model drawing and display on the early warning collision range of the crane equipment and the existing environment according to a judgment result; the method comprises the following steps: creating a line BIM model and a train and large-scale equipment occupation model through existing line data; data are accessed to a supervision system through a Beidou positioning and mechanical Internet of Things system, a mathematical model is generated through a BIM engine, the collision relation with an existing line model is calculated, space position information, output by an algorithm program, of the top end of a suspension arm is sent to a modeling program through a TCP protocol, and whether the suspension arm collides with the existing line or not is judged.

Description

Technical Field

[0001] The present invention relates to the technical field of crane anti-collision, and in particular to a crane collision warning method and device based on a digital twin platform. Background Art

[0002] In the field of collision prevention of existing lines, it is necessary to project the position information of construction machinery and the spatial position of the motion mechanism (such as the crane boom) into the world coordinate system, and create a crane model and an existing structural component model through modeling software, and perform parameterized settings. The crane can be parameterized for values ​​such as leg extension, turntable rotation angle, boom lifting angle, boom extension, hook extension, etc., and the numerical drive model can be adjusted. This application is proposed to achieve early warning of collision with existing lines. Summary of the invention

[0003] The main purpose of the present invention is to provide a crane collision warning method and device based on a digital twin platform, aiming to solve existing technical problems.

[0004] To achieve the above object, the present invention provides a crane collision warning method based on a digital twin platform, which specifically includes the following steps:

[0005] Extract feature vectors of the target crane equipment and its surrounding existing environment, and import the extracted features into the platform to sequentially establish a crane equipment model and a surrounding existing environment model;

[0006] Based on the model, calculate the overturning impact range of the crane equipment in the horizontal and spatial planes in the current state, and determine the overlap with the coverage range of the existing environment;

[0007] Based on the judgment results, the warning collision range of the crane equipment and the existing environment is highlighted and displayed.

[0008] Furthermore, the step of extracting the feature vector of the target crane equipment and its surrounding existing environment specifically includes:

[0009] The input environmental data is downsampled through the CNN network to extract the key information of the simplest features in the multi-feature matrix calculation;

[0010] Transform the model into a lightweight problem by solving contextual data association analysis at the semantic level through Transformer;

[0011] The environmental data of the area near the crane in the digital twin platform is input, and the data extraction of the model feature expression including key information is completed through the encoder.

[0012] Furthermore, after extracting the features, the features are classified. The specific steps include:

[0013] The model uses CNN network to extract features for classification, adopts the commonly used backbone network ResNet, and introduces multi-scale information through the spatial pyramid pooling module of void convolution;

[0014] The bottom-level features of the crane chassis and the high-level features of the boom are further integrated to increase the amount of information contained in the feature vector.

[0015] Furthermore, the model is used to calculate the overturning influence range of the horizontal plane and the spatial plane of the crane equipment in the current state, and the overlap with the coverage range of the existing environment is judged. The specific steps include:

[0016] The crane equipment model and the existing environment model nearby were introduced respectively;

[0017] Express the two models in feature matrices and perform lightweight operations on the feature matrices;

[0018] The crane equipment model is driven to rotate within the current range, and during the process, the projection of the characteristic matrix of the crane equipment model on the characteristic matrix of the existing environment model and the position of the crane equipment model and the existing environment model in the space plane are displayed in real time;

[0019] It is determined in turn whether the characteristic matrix of the crane equipment model overlaps with the characteristic matrix of the existing environment model on the projection horizontal plane and whether the crane equipment model and the existing environment model collide and intersect on the spatial plane.

[0020] Furthermore, it also includes:

[0021] In the early warning process, multiple targets in the existing environment are screened through the pre-set mask area, and candidate targets that appear in the mask area are retained;

[0022] By utilizing the time series relationship and using the assist technology, we can extract the deep features of the target in the hidden layer of the deep network, match the targets selected in the mask, and determine the dynamic illegal intrusion and the time of the illegal intrusion.

[0023] Furthermore, the data source of the digital twin platform comes from the digital chain in the Internet of Things platform, including Beidou, inertial navigation, vision and other data. The platform converts data links of different formats into features in the twin platform.

[0024] A crane collision warning device based on a digital twin platform is applied to the above-mentioned crane collision warning method based on a digital twin platform, comprising:

[0025] Sensors, used to collect data about the target crane equipment and the surrounding environment;

[0026] Digital twin platform, used to model the crane equipment and nearby existing environment data through the collected data;

[0027] A height warning mechanism for warning the height of the target crane equipment; and,

[0028] The horizontal rotation warning mechanism is used to warn the horizontal rotation operation of the target crane equipment.

[0029] Furthermore, the height warning mechanism includes a hanging frame, the top of the hanging frame is connected to the boom of the crane equipment through a pull rope, the boom of the crane equipment is provided with a rope winding mechanism, the bottom of the hanging frame is provided with a contact block, the contact block is provided with a guide rod movably inserted into the inside of the hanging frame, the hanging frame is provided with a spring connected to the guide rod, the end of the guide rod is provided with a counterweight block, the counterweight block is provided with a metal sheet, and the hanging frame is provided with a sensor.

[0030] Furthermore, a rolling block is provided at the bottom of the contact block, and the top of the contact block is rotatably connected to the guide rod.

[0031] Furthermore, the horizontal rotation warning mechanism includes a base arranged on the crane chassis, the base is provided with a guide rail, the guide rail includes a first guide rail and a second guide rail, the first guide rail and the second guide rail are cross-arranged, a movable bracket is provided on the first guide rail and the second guide rail, a scanning device is provided on the movable bracket, the base is arranged on a flip platform, and the flip platform can drive the scanning device to adjust the position on both sides of the crane boom.

[0032] The beneficial effects of the present invention are embodied in:

[0033] The present invention creates a line BIM model, a train and a large equipment occupancy model through existing line data; displays the overall situation through a GIS engine in a hierarchical loading manner; connects data to a supervision system through Beidou positioning and a mechanical Internet of Things system, generates a mathematical model through a BIM engine, calculates the collision relationship with the existing line model, and displays it in the form of a three-dimensional simulation and an envelope box to evaluate and visualize the construction risk of large equipment on the existing line. The spatial position information of the top of the boom output by the algorithm program is sent to the modeling program through the TCP protocol, which determines whether the top collides with the existing line.

[0034] The present invention uses an early warning device to warn the height and horizontal rotation of the crane equipment, and uses a mechanical early warning outside the crane equipment, combined with a system early warning method to further provide safety protection for the operation of the crane equipment and improve safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic flow chart of the collision warning method of the present invention;

[0036] Figure 2 It is a schematic diagram of the structure of the collision warning device (altitude warning mechanism) of the present invention;

[0037] Figure 3 It is a schematic diagram of the structure of the structural warning device (translation warning mechanism) of the present invention;

[0038] Figure 4 This is a structural schematic diagram of the horizontal rotation warning mechanism of the present invention;

[0039] Figure 5 It is a schematic diagram of feature extraction of the present invention;

[0040] Figure 6 It is a schematic diagram of a specific implementation scenario of the present invention;

[0041] Figure 7 It is a schematic diagram of a specific implementation scenario of the present invention;

[0042] Figure 8 It is a schematic diagram of a specific implementation scenario of the present invention.

[0043] Description of reference numerals:

[0044] 100. Sensor; 200. Digital twin platform; 300. Altitude warning mechanism; 301. Hanging frame; 302. Pull rope; 303. Rope winding mechanism; 304. Contact block; 305. Guide rod; 306. Spring; 307. Metal sheet; 308. Collision sensor; 309. Rolling block; 400. Translation warning mechanism; 401. Base; 402. First guide rail; 403. Second guide rail; 404. Moving bracket; 405. Scanning device; 406. Flipping platform. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. In the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] See also Figure 1 and Figure 5-8 The present invention provides a crane collision warning method based on a digital twin platform, which specifically includes the following steps:

[0047] S10: extracting feature vectors of the target crane equipment and its nearby existing environment, and importing the extracted features into the platform to sequentially establish a crane equipment model and a nearby existing environment model (for the convenience of description, the nearby existing environment described below is explained by taking a railway line as an example);

[0048] In one embodiment, the step of extracting the feature vector of the target crane equipment and its surrounding existing environment includes:

[0049] The input environmental data is downsampled through the CNN network to extract the key information of the simplest features in the multi-feature matrix calculation;

[0050] Transformer is used to solve the contextual data association analysis at the semantic level and transform the model into a lightweight problem; a deep neural network is used as the collision detection algorithm. The model combines the CNN convolutional neural network with the transformer attention mechanism to associate the contextual information of the data.

[0051] The environmental data of the area near the crane in the digital twin platform is input, and the data extraction of the model feature expression including key information is completed through the encoder.

[0052] In one embodiment, after extracting features, the features are classified. The specific steps include:

[0053] The model uses CNN network to extract features for classification, adopts the commonly used backbone network ResNet, and introduces multi-scale information through the spatial pyramid pooling module of void convolution;

[0054] The bottom-level features of the crane chassis and the high-level features of the boom are further integrated to increase the amount of information contained in the feature vector.

[0055] S20: Calculate the overturning influence range of the horizontal plane and the spatial plane of the crane equipment in the current state according to the model, and determine the overlap with the coverage range of the existing environment;

[0056] In one embodiment, the overturning influence range of the horizontal plane and the spatial plane of the crane equipment in the current state is calculated based on the model, and the overlap with the coverage range of the existing environment is judged. The specific steps include:

[0057] The crane equipment model and the existing environment model nearby were introduced respectively;

[0058] Express the two models in feature matrices and perform lightweight operations on the feature matrices;

[0059] The crane equipment model is driven to rotate within the current range, and during the process, the projection of the characteristic matrix of the crane equipment model on the characteristic matrix of the existing environment model and the position of the crane equipment model and the existing environment model in the space plane are displayed in real time;

[0060] It is determined in turn whether the characteristic matrix of the crane equipment model overlaps with the characteristic matrix of the existing environment model on the projection horizontal plane and whether the crane equipment model and the existing environment model collide and intersect on the spatial plane.

[0061] S30: Drawing and displaying a highlighted model of the warning collision range of the crane equipment and the existing environment according to the judgment result.

[0062] The digital twin platform based on GIS of the present invention uses a combination of model detection and collision warning algorithms, which can judge and provide collision warnings for illegal intrusions of people, vehicles, and events.

[0063] The digital twin platform combines GIS data, GPS data and attitude data output by the inertial navigation terminal, and drives the twin model based on the actual data collected by sensors.

[0064] Based on deep convolutional neural networks and Transformer mechanisms, real-time extraction of digital chaos platform data is achieved, feature vectors expressing scene conditions are obtained, and collision detection is further performed based on the feature vectors.

[0065] The present invention creates line BIM models, train and large equipment occupancy models through existing line data; displays the overall situation through a GIS engine in a hierarchical loading manner; connects data to a supervision system through Beidou positioning and a mechanical Internet of Things system, generates a mathematical model through a BIM engine, calculates the collision relationship with the existing line model, and displays it in the form of three-dimensional simulation and envelope boxes, so as to evaluate and visualize the construction risks of large equipment on the existing line.

[0066] Finally, the spatial position information of the boom top output by the algorithm program is sent to the modeling program through the TCP protocol, which determines whether the top has collided with the existing line.

[0067] Specifically, the model uses Tiantu or other maps combined with CAD to express the direction of the railway line. Two-dimensional interface machinery and vehicle models: Through modeling software or real photos, draw crane and train space models.

[0068] In one embodiment, the method further includes screening multiple targets in the existing environment through a preset mask area during the early warning process, and retaining candidate targets that appear in the mask area;

[0069] By using the time series relationship and the ass i gn technology, we can extract the deep features of the target in the hidden layer of the deep network, match the targets selected in the mask, and determine the dynamic illegal intrusion and the time of the illegal intrusion. Ass i gn is equivalent to a connection, which generally assigns the value of a variable to another variable continuously, just like connecting the two variables together, so it is habitually used as a connection, such as giving the output of one module to another module as input.

[0070] In one embodiment, the data source of the digital twin platform comes from the digital chain in the Internet of Things platform, including Beidou, inertial navigation, vision and other data. The platform converts data links of different formats into features in the twin platform.

[0071] See also Figure 2-4 The present invention also provides a crane collision warning device based on a digital twin platform, which is applied to the above-mentioned crane collision warning method based on a digital twin platform, including:

[0072] Sensor 100, used to collect data of the target crane equipment and the surrounding existing environment;

[0073] The digital twin platform 200 is used to model the crane equipment and nearby existing environmental data through the collected data;

[0074] A height warning mechanism 300 is used to warn the height of the target crane equipment; and

[0075] The horizontal rotation warning mechanism 400 is used to warn the horizontal rotation operation of the target crane equipment.

[0076] The present invention uses an early warning device to warn the height and horizontal rotation of the crane equipment, and uses a mechanical early warning outside the crane equipment, combined with a system early warning method to further provide safety protection for the operation of the crane equipment and improve safety.

[0077] In one embodiment, the height warning mechanism includes a hanging frame 301, the top of the hanging frame 301 is connected to the boom of the crane equipment through a pull rope 302, the boom of the crane equipment is provided with a rope winding mechanism 303, the bottom of the hanging frame 301 is provided with a contact block 304, the contact block 304 is provided with a guide rod 305 movably inserted into the inside of the hanging frame 301, the hanging frame 301 is provided with a spring 306 connected to the guide rod 305, the end of the guide rod 305 is provided with a counterweight block 306, the counterweight block 306 is provided with a metal sheet 307, and the hanging frame 301 is provided with a collision sensor 308.

[0078] This embodiment is configured such that before the crane equipment is working, according to the required working height, the pull rope 302 is released to make the contact block 304 touch the ground, and the guide rod 305 is compressed to make the spring 306 in a compressed state, so as to keep the metal sheet 307 and the collision sensor 308 in contact. When the crane equipment boom is at a safe working height, the contact block 304 always touches the ground, so that the metal sheet 307 and the collision sensor 308 are in contact. When the crane equipment boom is out of the safe working height, the height of the hanging frame 301 rises, driving the contact block 304 to separate from the ground. The spring 306 pushes the guide rod 305 to move, so that the metal sheet 307 and the collision sensor 308 are separated, triggering an alarm to alert the operator.

[0079] It should be noted that there is a certain buffer distance between the contact block 304 touching the ground and leaving the ground, rather than a fixed-point judgment. The alarm will only be triggered when the crane equipment arm leaves the warning area above the operating height, and the warning area is set according to the actual situation.

[0080] In one embodiment, a rolling block 309 is provided at the bottom of the contact block 304, and the top of the contact block 304 is rotatably connected to the guide rod 305. In this embodiment, the contact block 304 is rotatably connected to the guide rod 305, which can adapt to the change of the ground amplitude when the crane equipment is operated horizontally, ensuring that the judgment of the height warning is not affected when the crane equipment is operated horizontally, and ensuring the stable operation of the device.

[0081] In one embodiment, the horizontal rotation warning mechanism 400 includes a base 401 arranged on the chassis of the crane, and a guide rail is provided on the base 401. The guide rail includes a first guide rail 402 and a second guide rail 403. The first guide rail 402 and the second guide rail 403 are cross-arranged. A movable bracket 404 is provided on the first guide rail 402 and the second guide rail 403. A scanning device 405 is provided on the movable bracket 404. The base 401 is arranged on a flip platform 406. The flip platform 406 can drive the scanning device 405 to adjust the position on both sides of the crane boom.

[0082] This embodiment is configured such that when the crane equipment needs to be rotated horizontally, the base 401 is driven to move in the direction of the rotation by the flip platform 406, and the scanning device 405 is driven to move on the first guide rail 402 and the second guide rail 403 by the moving bracket 404 to scan the space in the direction where the crane equipment is about to rotate horizontally for obstacles, and the scanning information is fed back in time to assist the staff in making early warning judgments on collisions.

[0083] Specifically, the scanning device 405 may adopt an existing image scanning device, collect images in the space, and determine whether there is a collision with the crane equipment after image processing.

[0084] Specifically, the flipping platform 406 can be flipped by a motor drive.

[0085] It should be noted that if the embodiments of the present invention involve directional indications such as up, down, left, right, front, back, etc., then the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture as shown in the accompanying drawings. If the specific posture changes, the directional indication will also change accordingly.

[0086] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or schemes that A and B meet at the same time. In addition, "multiple" refers to more than two. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist.

[0087] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A crane collision warning method based on a digital twin platform, characterized in that: The specific steps include: Extract feature vectors of the target crane equipment and its surrounding existing environment, and import the extracted features into the platform to sequentially establish a crane equipment model and a surrounding existing environment model; Based on the model, calculate the overturning impact range of the crane equipment in the horizontal and spatial planes in the current state, and determine the overlap with the coverage range of the existing environment; Based on the judgment results, the warning collision range of the crane equipment and the existing environment is highlighted and displayed.

2. A crane collision warning method based on a digital twin platform as claimed in claim 1, characterized in that: The specific steps of extracting the feature vector of the target crane equipment and its surrounding existing environment include: The input environmental data is downsampled through the CNN network to extract the key information of the simplest features in the multi-feature matrix calculation; Transform the model into a lightweight problem by solving contextual data association analysis at the semantic level through Transformer; The environmental data of the area near the crane in the digital twin platform is input, and the data extraction of the model feature expression including key information is completed through the encoder.

3. A crane collision warning method based on a digital twin platform as claimed in claim 2, characterized in that: After extracting the features, the features are classified. The specific steps include: The model uses CNN network to extract features for classification, adopts the commonly used backbone network ResNet, and introduces multi-scale information through the spatial pyramid pooling module of void convolution; Further integrate the bottom-level features of the crane chassis with the high-level features of the boom. Increase the amount of information contained in the feature vector.

4. The crane collision warning method based on a digital twin platform as claimed in claim 1, characterized in that: The model is used to calculate the overturning influence range of the horizontal plane and the spatial plane of the crane equipment in the current state, and the overlap with the coverage range of the existing environment is judged. The specific steps include: The crane equipment model and the existing environment model nearby were introduced respectively; Express the two models in feature matrices and perform lightweight operations on the feature matrices; The crane equipment model is driven to rotate within the current range, and during the process, the projection of the characteristic matrix of the crane equipment model on the characteristic matrix of the existing environment model and the position of the crane equipment model and the existing environment model in the space plane are displayed in real time; It is determined in turn whether the characteristic matrix of the crane equipment model overlaps with the characteristic matrix of the existing environment model on the projection horizontal plane and whether the crane equipment model and the existing environment model collide and intersect on the spatial plane.

5. The crane collision warning method based on a digital twin platform as claimed in claim 1, characterized in that: Also includes, In the early warning process, multiple targets in the existing environment are screened through the pre-set mask area, and candidate targets that appear in the mask area are retained; By utilizing the time series relationship and using the assign technology, we can extract the deep features of the target in the hidden layer of the deep network, match the targets selected in the mask, and determine the dynamic illegal intrusion and the time of the illegal intrusion.

6. The crane collision warning method based on a digital twin platform as claimed in claim 1, characterized in that: The data source of the digital twin platform comes from the digital chain in the Internet of Things platform, including Beidou, inertial navigation, vision and other data. The platform converts data links in different formats into features in the twin platform.

7. A crane collision warning device based on a digital twin platform, characterized in that: A crane collision warning method based on a digital twin platform as described in any one of claims 1 to 6, comprising: A sensor (100) for collecting data of the target crane equipment and the surrounding existing environment; A digital twin platform (200) is used to model the crane equipment and nearby existing environmental data using the collected data; A height warning mechanism (300) is used to warn of the height of the target crane equipment; as well as, The horizontal rotation warning mechanism (400) is used to warn of the horizontal rotation operation of the target crane equipment.

8. A crane collision warning device based on a digital twin platform as claimed in claim 7, characterized in that: The height warning mechanism comprises a hanging frame (301), the top of the hanging frame (301) is connected to the boom of the crane equipment through a pull rope (302), the boom of the crane equipment is provided with a rope winding mechanism (303), the bottom of the hanging frame (301) is provided with a contact block (304), the contact block (304) is provided with a guide rod (305) movably inserted into the inside of the hanging frame (301), the hanging frame (301) is provided with a spring (306) connected to the guide rod (305), the end of the guide rod (305) is provided with a counterweight block (306), the counterweight block (306) is provided with a metal sheet (307), and the hanging frame (301) is provided with a collision sensor (308).

9. A crane collision warning device based on a digital twin platform as claimed in claim 8, characterized in that: A rolling block (309) is provided at the bottom of the contact block (304), and the top of the contact block (304) is rotatably connected to the guide rod (305).

10. The crane collision warning device based on the digital twin platform according to claim 7, characterized in that: The horizontal rotation warning mechanism (400) comprises a base (401) arranged on the crane chassis, the base (401) is provided with a guide rail, the guide rail comprises a first guide rail (402) and a second guide rail (403), the first guide rail (402) and the second guide rail (403) are arranged in a cross-shaped manner, a movable bracket (404) is provided on the first guide rail (402) and the second guide rail (403), a scanning device (405) is provided on the movable bracket (404), the base (401) is arranged on a turning platform (406), and the turning platform (406) can drive the scanning device (405) to adjust the position on both sides of the crane boom.

Images