A hoisting construction collision danger early warning method based on AR technology

Abstract

The application discloses a hoisting construction collision danger early warning method based on AR technology, which simulates a running path of a hoisting equipment three-dimensional model and a hoisting path of a beam body three-dimensional model in a three-dimensional real scene model, the hoisting path of the beam body is fitted by a plurality of hoisting position verification points, the AR software on the intelligent mobile terminal continuously monitors whether the beam body is hoisted along the hoisting path during the running of the hoisting equipment and the hoisting of the beam body, and when judging whether the beam body is hoisted along the hoisting path, the AR software considers the offset distance based on the peripheral clustering collision feature points, if the actual distance of the beam body deviating from the corresponding hoisting position verification point is greater than the offset distance, the AR software sends an early warning to the outside. The application has the advantages that: based on the AR software on the intelligent mobile terminal, a simulated hoisting process animation is displayed, and the hoisting track of the beam body can be monitored during the hoisting of the beam body to judge in real time whether there is a danger of collision, and if there is, an early warning is sent to the outside.

Description

Technical Field

[0001] This invention belongs to the field of hoisting construction technology, specifically relating to a hoisting construction collision hazard warning method based on AR technology. Background Technology

[0002] Augmented Reality (AR) technology is a technique that cleverly integrates virtual information with the real world. It widely utilizes multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing technologies to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thus "enhancing" the real world. Currently, AR technology is mainly used in the gaming and film industries, with limited application in actual industrial production and construction.

[0003] At bridge construction sites, hoisting equipment is needed to lift and erect bridge beams. During this beam erection process, there may be collision hazards due to existing structures and the formwork or construction equipment erected on the construction site. However, these construction hazards are often not detected or warned in time during the construction process.

[0004] Therefore, those skilled in the art urgently need a method that can provide timely warnings of dangers such as construction collisions during hoisting operations. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing an AR-based early warning method for collision hazards during hoisting construction. This method simulates the travel path of the hoisting equipment model and the hoisting path of the beam model in a three-dimensional real-world model. The hoisting path of the beam is fitted by several hoisting position verification points. During the process of the hoisting equipment traveling and hoisting the beam, the AR software on the smart mobile terminal continuously monitors whether the hoisting equipment moves along the travel path and whether the beam is hoisted along the hoisting path. When determining whether the beam is hoisted along the hoisting path, the method considers the offset distance based on the clustered collision feature points of the periphery. If the actual distance of the beam deviating from the corresponding hoisting position verification point is greater than the offset distance, the AR software issues an early warning.

[0006] The objective of this invention is achieved through the following technical solutions:

[0007] A method for early warning of collision hazards during hoisting operations based on AR technology, characterized in that the early warning method includes the following steps:

[0008] S1: The environment of the construction site is scanned by a smart mobile terminal with a camera device. Based on the coordinates of the existing structures at the construction site, a three-dimensional real-scene model of the construction site is established. The existing structures in the three-dimensional real-scene model are set as cluster feature points, and objects in the three-dimensional real-scene model with a height of 1.5 meters or more are set as cluster collision feature points.

[0009] S2: Create a 3D model of the hoisting equipment and a 3D model of the beam; simulate the hoisting process in the 3D reality model, simulate the travel path of the 3D model of the hoisting equipment, and mark the hoisting points on the travel path where the 3D model of the hoisting equipment needs to stop moving for hoisting in sequence; simulate the number of the 3D model of the beam to be hoisted and the hoisting path at each hoisting point, and distribute several hoisting position verification points and beam erection position points at intervals on the hoisting path; wherein, based on the beam size, the distance between the hoisting position verification point and the cluster collision feature point is not less than 2.5m, and the distance between the hoisting position verification point and the nearest surrounding cluster collision feature point is S, then the offset distance of the beam at the hoisting position verification point is not greater than S-2.5m;

[0010] S3: Before the hoisting operation, the construction personnel use AR software on a smart mobile terminal to scan and detect the construction site. During the scanning and detection process, they search for the clustered feature points located on the horizontal or vertical plane, and match the clustered feature points with the corresponding three-dimensional real-scene models. The matched three-dimensional real-scene models are displayed on the AR software interface of the smart mobile terminal, and a simulated hoisting process animation of the three-dimensional model of the hoisting equipment and the three-dimensional model of the beam is displayed.

[0011] S4: Fix the smart mobile terminal with a bracket so that the display angle of the three-dimensional real scene model changes with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the shooting screen can monitor the movement of the hoisting equipment and the hoisting and erection of the beam.

[0012] S5: Construction workers operate according to the simulated hoisting process animation:

[0013] S5.1: Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. The AR software on the smart mobile terminal captures the stopped position of the hoisting equipment and determines whether it coincides with the position of the hoisting point on the travel path. If they coincide, proceed to the next hoisting operation; if they do not coincide, issue a warning to cause the hoisting equipment to adjust its position until the stopped position of the hoisting equipment coincides with the position of the first hoisting point on the travel path.

[0014] S5.2: Several beams are placed at the construction site, and each beam is marked with a number on its upper surface; the beams with the corresponding numbers are connected by the hoisting equipment, and the AR software on the smart mobile terminal captures and identifies the numbers on the hoisted beams to determine whether they are the corresponding numbers. If they are not the corresponding numbers, an early warning is issued so that the hoisting equipment can re-hoist the beams with the correct numbers.

[0015] Subsequently, the beam is hoisted onto the bridge using the hoisting equipment. The AR software on the smart mobile terminal detects whether the beam passes each of the hoisting position verification points sequentially during the hoisting process. If it is detected that the beam has not passed any of the verification points, the hoisting movement of the beam is stopped, and the actual position of the beam relative to the verification point is monitored. If the offset distance does not exceed the allowable offset distance, the hoisting movement of the beam continues. If the offset distance exceeds the allowable offset distance, there is a risk of collision, and a warning message is sent to the construction personnel via the AR software on the smart mobile terminal.

[0016] Before the lifting equipment lifts the beam onto the bridge and releases the lifting, the AR software on the smart mobile terminal captures whether the beam has been erected to the beam erection position. If so, the lifting equipment releases the beam; otherwise, the beam is lifted and the erection position is adjusted until it reaches the preset position.

[0017] S5.3: Following the method in steps S5.1-S5.2, control the hoisting equipment to move sequentially to each of the hoisting points and erect the corresponding numbered beams onto the bridge in sequence.

[0018] The smart mobile terminal is a smartphone or tablet computer.

[0019] The hoisting path is fitted by several hoisting position verification points, and the distance between adjacent hoisting position verification points is between 1m and 2m.

[0020] The AR software of the smart mobile terminal communicates with a cloud server, which stores several 3D real-scene models, several 3D models of hoisting equipment, several 3D models of beams, and several animations simulating the hoisting process. When construction personnel scan and detect the construction site using the AR software on the smart mobile terminal, they upload the clustering feature points of the existing structures at the construction site to the cloud server. They then query and compare the detected clustering feature points with the clustering feature points contained in each of the 3D real-scene models to obtain a 3D real-scene model that matches the construction site environment.

[0021] The advantages of this invention are: Based on AR software on a smart mobile terminal, a corresponding 3D real-scene model can be generated by matching the captured construction site footage. This 3D real-scene model can then display a simulated hoisting process animation of the hoisting equipment model and the beam model, allowing hoisting equipment operators to clearly understand the requirements and path planning for the hoisting operation. During the movement of the hoisting equipment, the AR software on the smart mobile terminal uses its camera to capture and monitor the position of the hoisting equipment in real time and compare it with the travel path. If a deviation is detected, a hazard warning is issued. During the hoisting of the beam, the AR software on the smart mobile terminal uses its camera to capture and monitor the position of the beam in real time. If the beam is detected to have deviated from the corresponding hoisting position verification point and the deviation distance is greater than its permissible deviation distance, a collision warning is issued. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the intelligent mobile terminal scanning and detecting the construction site in this invention;

[0023] Figure 2 This is a schematic diagram illustrating how the AR software within the intelligent mobile terminal matches a 3D real-world model and displays an animation of the simulated hoisting process in this invention.

[0024] Figure 3 This is a schematic diagram of the travel path of the hoisting equipment in this invention;

[0025] Figure 4 This is a schematic diagram of the hoisting path of the beam in this invention. Implementation

[0026] The features and other related features of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments, so as to facilitate understanding by those skilled in the art:

[0027] Example: Figure 1-4 As shown, this embodiment specifically relates to a method for early warning of collision hazards during hoisting operations based on AR technology. This method includes the following steps:

[0028] S1: The environment of the construction site is scanned using a smart mobile terminal with a camera, and a three-dimensional real-scene model of the construction site based on the coordinates of the existing structures at the construction site is established. The three-dimensional real-scene model has clustering feature points for feature matching. The clustering feature points are generally taken from the existing structures at the construction site. For example, in this embodiment, bridge piers are used as clustering feature points. Objects with a height of 1.5 meters or more in the three-dimensional real-scene model are set as clustering collision feature points, because objects with a height of 1.5 meters or more may collide with the beams during the hoisting process.

[0029] The three-dimensional reality model established above is stored in the cloud server of the AR software. The AR software communicates with the cloud server and can query or call the various data stored therein in real time.

[0030] It should be noted that the smart mobile terminal used in this step is a common mobile phone or tablet computer, but preferably a professional 3D image scanning device.

[0031] S2: Based on the dimensional information of the lifting equipment and the beam, create 3D models of both the lifting equipment and the beam in modeling software, then import them into a cloud server for storage. Here, the lifting equipment refers to a crawler crane. Simulate the lifting process within the 3D reality model. The simulation includes:

[0032] The simulated 3D model of the hoisting equipment travels on the construction site, such as... Figure 3 As shown, several hoisting points are set on the travel path. The so-called hoisting point refers to the position where the hoisting equipment needs to stop during the movement and travel, and the beam is hoisted and erected on the bridge at the position. Therefore, the number of hoisting points corresponds to the number of beams to be hoisted, and the number of each hoisting point corresponds to the number of the beam. The hoisting equipment stops in sequence according to the hoisting point number.

[0033] The simulation demonstrates the lifting path of the hoisting equipment, which lifts corresponding numbered beams at various lifting points and erects them on the bridge. Different numbers are marked on the upper surface of each beam for AR software on a smart mobile terminal to capture images and identify the numbers, thus confirming that the hoisting equipment is lifting the correct beam. The simulated hoisting path includes several hoisting position verification points and the final beam erection position point. The hoisting position verification points are used to check and monitor the accuracy of the path during the hoisting process to prevent collisions, while the beam erection position point is used to check and monitor the accuracy of the beam's placement on the bridge structure. For example... Figure 4 As shown, it should be noted that the hoisting path is specifically fitted by several hoisting position verification points. In this embodiment, the spacing between adjacent hoisting position verification points is about 1m, which means that the position needs to be monitored and verified every 1m during the beam hoisting process. Based on the beam dimensions, the safe distance S0 between each hoisting position verification point and the clustering collision feature point is not less than 2.5m. The distance S between the hoisting position verification point and its nearest surrounding clustering collision feature point is then considered. Therefore, the possible offset distance of the beam at the hoisting position verification point is not greater than S - S0m.

[0034] The simulated hoisting equipment travel path and beam hoisting path are used to create a simulated hoisting process animation, which is then uploaded to the cloud server of the AR software for storage, so that the AR software can call the simulated hoisting process animation in real time.

[0035] S3: As Figure 1 , 2 As shown, before hoisting operations, the hoisting equipment operators first select a location with a wide field of vision (ensuring they can observe existing structures and the subsequent beam erection process) to scan and inspect the construction site. This is done using AR software on a smart mobile terminal. Since the smart mobile terminal's camera captures the real-world environment frame by frame, the system searches for clustered feature points (such as bridge piers in this embodiment) that appear to be located on common horizontal or vertical surfaces, and uploads the found clustered feature points. Based on the user's perception-based interaction principle, a grid is displayed on the AR software interface to indicate the completion of the environmental scanning and inspection, such as... Figure 2 As shown, this allows users to confirm that the scanning and detection work they are supposed to do has been completed.

[0036] The AR software on the smart mobile terminal communicates with the cloud server. The cluster feature points (bridge piers) found by the AR software at the construction site are uploaded to the cloud server and compared with the cluster feature points contained in several 3D real-scene models stored in the cloud server. When a 3D real-scene model with the same cluster feature points is found, it means that the match is successful. At this time, the matched 3D real-scene model appears on the construction site image layer of the AR software. At the same time, the 3D model of the hoisting equipment and the 3D model of the beam appear on the 3D real-scene model, and the simulated hoisting process animation of the two is displayed. The displayed simulated hoisting process animation can enable the hoisting equipment operators to quickly and clearly understand the hoisting construction steps of the project.

[0037] S4: At least two smart mobile terminals should be prepared at the construction site;

[0038] One is placed on the observation platform and fixed with a bracket. The observation platform should be located at a high position with a wide field of view so as to capture the construction site in real time. The display angle of the 3D real scene model on the AR software should change with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the captured image can monitor the movement of the hoisting equipment and the hoisting and erection of the beam.

[0039] Another device is placed in the cab of the hoisting equipment for the operator to view and use, so that they can receive early warning information in a timely manner.

[0040] S5: As Figure 3 , 4 As shown, the operators of the hoisting equipment follow the animation of the simulated hoisting process to operate the hoisting equipment and the beam:

[0041] S5.1) Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. During this process, the AR software of the smart mobile terminal captures the movement trajectory of the hoisting equipment in real time. When the hoisting equipment stops moving for more than 60 seconds, the AR software is triggered to verify and judge the stopping position of the hoisting equipment, that is, whether the stopping position is located at the simulated first hoisting point. If the two positions coincide, the next hoisting operation is performed; if the two positions do not coincide, an alarm is issued, and the operator controls the hoisting equipment to adjust its position until the stopping position of the hoisting equipment coincides with the first hoisting point on the travel path.

[0042] S5.2) After the hoisting equipment is positioned at the first hoisting point, the construction personnel connect the hoisting equipment to the beam to be hoisted and prepare to hoist it. At this time, the AR software on the smart mobile terminal captures and identifies the number on the beam connected to the hoisting equipment to determine whether it is the corresponding number. If it is not the corresponding number, an alarm is issued and the operator controls the hoisting equipment to re-hoist the beam with the correct number.

[0043] Next, the beams are hoisted onto the bridge using hoisting equipment. AR software on the smart mobile terminal detects whether the beams pass through each hoisting position verification point in sequence during the hoisting process. If it is detected that the beams have not passed through any of the hoisting position verification points, the hoisting and movement of the beams is stopped, and the actual position of the beams relative to the hoisting position verification point is monitored. If the offset distance does not exceed the allowable offset distance, the hoisting and movement of the beams continues; if the offset distance exceeds the allowable offset distance, there is a risk of collision, and a warning message is sent to the operators through the AR software on the smart mobile terminal.

[0044] Before the lifting equipment lifts the beam onto the bridge and releases it, the AR software on the smart mobile terminal detects whether the beam has been erected to the beam erection position. If so, the lifting equipment releases the beam; otherwise, the beam is lifted and the erection position is adjusted until it reaches the preset position. Judging the erection position of the beam is mainly to prevent the beam from overturning after it is on the bridge.

[0045] Taking the beam segment in this embodiment as an example, to prevent the T-beam from overturning after it is placed on the bridge, the construction sequence of side beams first and then middle beams needs to be adopted. For a bridge with 5 beam segments, you can first select 5 beam segments, and then set the sequence as follows: left beam, left secondary side beam, right beam, right secondary side beam, and middle beam. After setting, if the erection sequence is incorrect when erecting 5 beam segments, the operation will be directly stopped.

[0046] S5.3: Following the methods in steps S5.1-S5.2, control the hoisting equipment to move sequentially to each hoisting point and erect the corresponding numbered beams onto the bridge in sequence.

[0047] The advantages of this embodiment are: Based on the AR software on the smart mobile terminal, a corresponding 3D real-scene model can be matched with the captured construction site footage, and the simulated hoisting process animation of the hoisting equipment model and the beam model can be displayed in the 3D real-scene model, so that the hoisting equipment operators can clearly understand the requirements for hoisting construction and path planning; during the movement of the hoisting equipment, the AR software on the smart mobile terminal uses its camera device to capture and monitor the position of the hoisting equipment in real time and compare it with the travel path. If there is a deviation problem, a danger warning will be issued; during the hoisting of the beam, the AR software on the smart mobile terminal uses its camera device to capture and monitor the position of the beam in real time. If the beam is detected to deviate from the corresponding hoisting position verification point and the deviation distance is greater than its allowable deviation distance, a collision warning message will be issued.

Claims

1. An AR technology-based hoisting construction collision danger warning method, characterized by The early warning method includes the following steps: S1: The environment of the construction site is scanned by a smart mobile terminal with a camera device. Based on the coordinates of the existing structures at the construction site, a three-dimensional real-scene model of the construction site is established. The existing structures in the three-dimensional real-scene model are set as cluster feature points, and objects in the three-dimensional real-scene model with a height of 1.5 meters or more are set as cluster collision feature points. S2: Create a 3D model of the hoisting equipment and a 3D model of the beam; simulate the hoisting process in the 3D reality model, simulate the travel path of the 3D model of the hoisting equipment, and mark the hoisting points on the travel path where the 3D model of the hoisting equipment needs to stop moving for hoisting in sequence; simulate the number of the 3D model of the beam to be hoisted and the hoisting path at each hoisting point, and distribute several hoisting position verification points and beam erection position points at intervals on the hoisting path; wherein, based on the beam size, the distance between the hoisting position verification point and the cluster collision feature point is not less than 2.5m, and the distance between the hoisting position verification point and the nearest surrounding cluster collision feature point is S, then the offset distance of the beam at the hoisting position verification point is not greater than S-2.5m; S3: Before the hoisting operation, the construction personnel use AR software on a smart mobile terminal to scan and detect the construction site. During the scanning and detection process, they search for the clustered feature points located on the horizontal or vertical plane, and match the clustered feature points with the corresponding three-dimensional real-scene models. The matched three-dimensional real-scene models are displayed on the AR software interface of the smart mobile terminal, and a simulated hoisting process animation of the three-dimensional model of the hoisting equipment and the three-dimensional model of the beam is displayed. S4: Fix the smart mobile terminal with a bracket so that the display angle of the three-dimensional real scene model changes with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the shooting screen can monitor the movement of the hoisting equipment and the hoisting and erection of the beam. S5: Construction workers operate according to the simulated hoisting process animation: S5.1: Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. The AR software on the smart mobile terminal captures the stopped position of the hoisting equipment and determines whether it coincides with the position of the hoisting point on the travel path. If they coincide, proceed to the next hoisting operation; if they do not coincide, issue a warning to cause the hoisting equipment to adjust its position until the stopped position of the hoisting equipment coincides with the position of the first hoisting point on the travel path. S5.2: Several beams are placed at the construction site, and each beam is marked with a number on its upper surface; the beams with the corresponding numbers are connected by the hoisting equipment, and the AR software on the smart mobile terminal captures and identifies the numbers on the hoisted beams to determine whether they are the corresponding numbers. If they are not the corresponding numbers, an early warning is issued so that the hoisting equipment can re-hoist the beams with the correct numbers. Subsequently, the beam is hoisted onto the bridge using the hoisting equipment. The AR software on the smart mobile terminal detects whether the beam passes each of the hoisting position verification points sequentially during the hoisting process. If it is detected that the beam has not passed any of the verification points, the hoisting movement of the beam is stopped, and the actual position of the beam relative to the verification point is monitored. If the offset distance does not exceed the allowable offset distance, the hoisting movement of the beam continues. If the offset distance exceeds the allowable offset distance, there is a risk of collision, and a warning message is sent to the construction personnel via the AR software on the smart mobile terminal. Before the lifting equipment lifts the beam onto the bridge and releases the lifting, the AR software on the smart mobile terminal captures whether the beam has been erected to the beam erection position. If so, the lifting equipment releases the beam; otherwise, the beam is lifted and the erection position is adjusted until it reaches the preset position. S5.3: Following the method in steps S5.1-S5.2, control the hoisting equipment to move sequentially to each of the hoisting points and erect the corresponding numbered beams onto the bridge in sequence.

2. The method for early warning of collision hazards in hoisting construction based on AR technology according to claim 1, characterized in that... The smart mobile terminal is a smartphone or tablet computer.

3. The method for early warning of collision hazards in hoisting construction based on AR technology according to claim 1, characterized in that... The hoisting path is fitted by several hoisting position verification points, and the distance between adjacent hoisting position verification points is between 1m and 2m.

4. The method for early warning of collision hazards in hoisting construction based on AR technology according to claim 1, characterized in that... The AR software of the smart mobile terminal communicates with a cloud server, which stores several 3D real-scene models, several 3D models of hoisting equipment, several 3D models of beams, and several animations simulating the hoisting process. When construction personnel scan and detect the construction site using the AR software on the smart mobile terminal, they upload the clustering feature points of the existing structures at the construction site to the cloud server. They then query and compare the detected clustering feature points with the clustering feature points contained in each of the 3D real-scene models to obtain a 3D real-scene model that matches the construction site environment.

Images