A hoisting construction danger early warning method based on AR technology

Abstract

The application discloses a hoisting construction danger early warning method based on AR technology, which comprises the following steps: constructing a three-dimensional real scene model of a construction site; manufacturing a three-dimensional model of hoisting equipment and a three-dimensional model of a beam body; simulating a running path of the three-dimensional model of the hoisting equipment and a hoisting path of the hoisted beam body; scanning and detecting the construction site through AR software on a smart mobile terminal, matching clustering feature points of existing structures to corresponding three-dimensional real scene models, and showing a simulated hoisting process animation; moving and hoisting the beam body on a bridge through the hoisting equipment, in the process, the AR software of the smart mobile terminal continuously monitors and captures the moving path of the hoisting equipment and the hoisting path of the beam body and verifies them, and sends an early warning to the outside in case of problems in the verification. The application has the advantages that: based on the AR software on the smart mobile terminal, a simulated hoisting process animation is shown, and the hoisting equipment and the hoisting beam body can be monitored in the process to send an early warning in dangerous situations.

Description

Technical Field

[0001] This invention belongs to the field of hoisting construction technology, specifically relating to a hoisting construction hazard early warning method based on AR technology. Background Technology

[0002] Augmented Reality (AR) technology is a technique that cleverly integrates virtual information with the real world. It widely utilizes multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing technologies to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thus "enhancing" the real world. Currently, AR technology is mainly used in the gaming and film industries, with limited application in actual industrial production and construction.

[0003] At bridge construction sites, hoisting equipment is used to lift and erect bridge beams. During this beam erection process, there may be various construction hazards, such as dangers caused by incorrect beam erection sequence or lifting height exceeding the preset elevation. However, these construction hazards often cannot be detected and warned in time during the construction process.

[0004] Therefore, those skilled in the art urgently need a method that can provide timely warnings of construction hazards during hoisting operations. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a hoisting construction hazard warning method based on AR technology. This warning method simulates the travel path of the hoisting equipment model and the hoisting path of the beam model in a three-dimensional real-scene model. During the process of the hoisting equipment traveling and hoisting the beam, the AR software on the smart mobile terminal continuously monitors whether the hoisting equipment moves along the travel path and whether the beam is hoisted along the hoisting path. If either is not present, the AR software issues a warning.

[0006] The objective of this invention is achieved through the following technical solutions:

[0007] A method for early warning of hoisting construction hazards based on AR technology, characterized in that the early warning method includes the following steps:

[0008] S1: The environment of the construction site is scanned using a smart mobile terminal with a camera device. Based on the coordinates of the existing structures at the construction site, a three-dimensional real-scene model of the construction site is established. The existing structures in the three-dimensional real-scene model are used as clustering feature points.

[0009] S2: Create a 3D model of the hoisting equipment and a 3D model of the beam; simulate the hoisting process in the 3D model, simulate the travel path of the 3D model of the hoisting equipment, and mark the hoisting points on the travel path in sequence where the 3D model of the hoisting equipment needs to stop moving for hoisting; simulate the number of the 3D model of the beam to be hoisted and the hoisting path at each hoisting point, and distribute several hoisting position verification points and beam erection position points at intervals on the hoisting path;

[0010] S3: Before the hoisting operation, the construction personnel use AR software on a smart mobile terminal to scan and detect the construction site. During the scanning and detection process, they search for the clustered feature points located on the horizontal or vertical plane, and match the clustered feature points with the corresponding three-dimensional real-scene models. The matched three-dimensional real-scene models are displayed on the AR software interface of the smart mobile terminal, and a simulated hoisting process animation of the three-dimensional model of the hoisting equipment and the three-dimensional model of the beam is displayed.

[0011] S4: Fix the smart mobile terminal with a bracket so that the display angle of the three-dimensional real scene model changes with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the shooting screen can monitor the movement of the hoisting equipment and the hoisting and erection of the beam.

[0012] S5: Construction personnel operate based on the simulated hoisting process animation:

[0013] S5.1: Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. The AR software on the smart mobile terminal captures the stopped position of the hoisting equipment and determines whether it coincides with the position of the hoisting point on the travel path. If they coincide, proceed to the next hoisting operation; if they do not coincide, issue a warning to cause the hoisting equipment to adjust its position until the stopped position of the hoisting equipment coincides with the position of the first hoisting point on the travel path.

[0014] S5.2: Several beams are placed at the construction site, each beam having a number marked on its upper surface. The beams with the corresponding numbers are hoisted using the hoisting equipment. The AR software on the smart mobile terminal captures and identifies the numbers on the hoisted beams to determine if they correspond to the correct number. If not, a warning is issued, prompting the hoisting equipment to re-hoist the beam with the correct number. Then, the beams are hoisted onto the bridge using the hoisting equipment. The AR software on the smart mobile terminal captures whether the beam passes through each hoisting position verification point sequentially during the hoisting process. If the beam fails to pass two or more hoisting position verification points consecutively during the hoisting process, a warning is issued, causing the beam to stop moving and be adjusted. Before releasing the hoisting equipment from the bridge, the AR software on the smart mobile terminal captures whether the beam has been erected at the beam erection position. If so, the hoisting equipment releases the beam; otherwise, the beam is lifted and the erection position is adjusted until it reaches the preset position.

[0015] S5.3: Following the method in steps S5.1-S5.2, control the hoisting equipment to move sequentially to each of the hoisting points and erect the corresponding numbered beams onto the bridge in sequence.

[0016] The smart mobile terminal is a smartphone or tablet computer.

[0017] The AR software of the smart mobile terminal communicates with a cloud server, which stores several 3D real-scene models, several 3D models of hoisting equipment, several 3D models of beams, and several animations simulating the hoisting process. When construction personnel scan and detect the construction site using the AR software on the smart mobile terminal, they upload the clustering feature points of the existing structures at the construction site to the cloud server. They then query and compare the detected clustering feature points with the clustering feature points contained in each of the 3D real-scene models to obtain a 3D real-scene model that matches the construction site environment.

[0018] The advantages of this invention are: based on AR software on a smart mobile terminal, a corresponding three-dimensional real-scene model can be matched with the captured construction site footage, and the simulated hoisting process animation of the hoisting equipment model and the beam model can be displayed in the three-dimensional real-scene model, so that the hoisting equipment operators can clearly understand the requirements for hoisting construction and path planning; during the movement of the hoisting equipment and the hoisting of the beam, the AR software on the smart mobile terminal uses its camera device to capture and monitor the position of the hoisting equipment and the beam in real time and compare the position with the travel path and the hoisting path. If there is a problem in the monitoring, a danger warning will be issued. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the intelligent mobile terminal scanning and detecting the construction site in this invention;

[0020] Figure 2 This is a schematic diagram illustrating how the AR software within the intelligent mobile terminal matches a 3D real-scene model and displays an animation of the simulated hoisting process in this invention.

[0021] Figure 3 This is a schematic diagram of the travel path in this invention;

[0022] Figure 4 This is a schematic diagram of the hoisting path in this invention. Implementation

[0023] The features and other related features of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments, so as to facilitate understanding by those skilled in the art:

[0024] Example: Figures 1-4 As shown, this embodiment specifically relates to a hoisting construction hazard early warning method based on AR technology. The hoisting construction hazard early warning method includes the following steps:

[0025] S1: The construction site environment is scanned using a smart mobile terminal equipped with a camera. Based on the coordinates of existing structures at the construction site, a 3D reality model of the construction site based on AR technology is established. This 3D reality model includes clustering feature points for feature matching. These clustering feature points are typically taken from existing structures at the construction site; for example, bridge piers are used as clustering feature points in this embodiment. The established 3D reality model is stored in the cloud server of the AR software. The AR software communicates with the cloud server and can query or retrieve the stored data in real time. It should be noted that the smart mobile terminal used in this step is a common mobile phone or tablet computer; preferably, a professional 3D image scanning device.

[0026] S2: Based on the dimensional information of the lifting equipment and the beam, create 3D models of both the lifting equipment and the beam in modeling software, then import them into a cloud server for storage. Here, the lifting equipment refers to a crawler crane. Simulate the lifting process within the 3D reality model. The simulation includes:

[0027] The simulation model of the hoisting equipment's 3D model shows its travel path on the construction site. Several hoisting points are set along this path. A hoisting point is a location where the hoisting equipment needs to stop during its movement, and the beams are hoisted and erected on the bridge at these locations. Therefore, the number of hoisting points corresponds to the number of beams to be hoisted, and the number of each hoisting point corresponds to the number of the beams. The hoisting equipment stops in sequence according to the hoisting point numbers.

[0028] The simulation demonstrates the hoisting path where the hoisting equipment lifts corresponding numbered beams at various lifting points and erects them on the bridge. Different numbers are marked on the upper surface of each beam for AR software on a smart mobile terminal to capture images and identify the numbers, thus confirming that the hoisting equipment is lifting the correct beam. The simulated hoisting path includes several hoisting position verification points and the final beam erection point. The hoisting position verification points are used to check the accuracy of the path during the hoisting process to prevent collisions, while the beam erection point is used to check the accuracy of the beam's placement on the bridge structure.

[0029] The simulated hoisting equipment travel path and beam hoisting path are used to create a simulated hoisting process animation, which is then uploaded to the cloud server of the AR software for storage, so that the AR software can call the simulated hoisting process animation in real time.

[0030] S3: As Figure 1 , 2 As shown, before hoisting operations, the hoisting equipment operators first select a location with a wide field of vision (ensuring they can observe existing structures and the subsequent beam erection process) to scan and inspect the construction site. This is done using AR software on a smart mobile terminal. Since the smart mobile terminal's camera captures the real-world environment frame by frame, the system searches for clustered feature points (such as bridge piers in this embodiment) that appear to be located on common horizontal or vertical surfaces, and uploads the found clustered feature points. Based on the user's perception-based interaction principle, a grid is displayed on the AR software interface to indicate the completion of the environmental scanning and inspection, such as... Figure 2 As shown, this allows users to confirm that the scanning and detection work they are supposed to do has been completed.

[0031] The AR software on the smart mobile terminal communicates with the cloud server. The cluster feature points (bridge piers) found by the AR software at the construction site are uploaded to the cloud server and compared with the cluster feature points contained in several 3D real-scene models stored in the cloud server. When a 3D real-scene model with the same cluster feature points is found, it means that the match is successful. At this time, the matched 3D real-scene model appears on the construction site image layer of the AR software. At the same time, the 3D model of the hoisting equipment and the 3D model of the beam appear on the 3D real-scene model, and the simulated hoisting process animation of the two is displayed. The displayed simulated hoisting process animation can enable the hoisting equipment operators to quickly and clearly understand the hoisting construction steps of the project.

[0032] S4: At least two smart mobile terminals should be prepared at the construction site;

[0033] One is placed on the observation platform and fixed with a bracket. The observation platform should be located at a high position with a wide field of view so as to capture the construction site in real time. The display angle of the 3D real scene model on the AR software should change with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the captured image can monitor the movement of the hoisting equipment and the hoisting and erection of the beam.

[0034] Another device is placed in the cab of the hoisting equipment for the operator to view and use, so that they can receive early warning information in a timely manner.

[0035] S5: As Figure 3 , 4 As shown, the operators of the hoisting equipment follow the animation of the simulated hoisting process to operate the hoisting equipment and the beam:

[0036] S5.1) Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. During this process, the AR software of the smart mobile terminal captures the movement trajectory of the hoisting equipment in real time. When the hoisting equipment stops moving for more than 60 seconds, the AR software is triggered to verify and judge the stopping position of the hoisting equipment, that is, whether the stopping position is located at the simulated first hoisting point. If the two positions coincide, the next hoisting operation is performed; if the two positions do not coincide, an alarm is issued, and the operator controls the hoisting equipment to adjust its position until the stopping position of the hoisting equipment coincides with the first hoisting point on the travel path.

[0037] S5.2) After the hoisting equipment is positioned at the first hoisting point, the construction personnel connect the hoisting equipment to the beam to be hoisted and prepare to hoist it. At this time, the AR software on the smart mobile terminal captures and identifies the number on the beam connected to the hoisting equipment to determine whether it is the corresponding number. If it is not the corresponding number, an alarm is issued and the operator controls the hoisting equipment to re-hoist the beam with the correct number.

[0038] Afterwards, the beams are hoisted onto the bridge using hoisting equipment. The AR software on the smart mobile terminal detects whether the beams pass through each hoisting position verification point in sequence during the hoisting process. If the beams fail to pass through two or more hoisting position verification points consecutively during the hoisting process, there may be a risk of deviating from the predetermined hoisting path and causing a collision. In this case, an early warning is issued to stop the movement of the beams and make adjustments. Before the hoisting equipment lifts the beams onto the bridge and before the hoisting is lifted, the AR software on the smart mobile terminal detects whether the beams have been erected at the beam erection position. If so, the hoisting equipment is lifted from the beams. If not, the beams are raised and the erection position is adjusted until they reach the preset position. Determining the erection position of the beams is mainly to prevent the beams from overturning after they are placed on the bridge.

[0039] Taking the beam segment in this embodiment as an example, to prevent the T-beam from overturning after it is placed on the bridge, the construction sequence of side beams first and then middle beams needs to be adopted. For a bridge with 5 beam segments, you can first select 5 beam segments, and then set the sequence as follows: left beam, left secondary side beam, right beam, right secondary side beam, and middle beam. After setting, if the erection sequence is incorrect when erecting 5 beam segments, the operation will be directly stopped.

[0040] S5.3) Following the methods in steps S5.1-S5.2, control the hoisting equipment to move to each hoisting point in sequence and erect the corresponding numbered beams onto the bridge.

[0041] The advantages of this embodiment are: based on the AR software on the smart mobile terminal, a corresponding three-dimensional real-scene model can be matched with the captured construction site images, and the simulated hoisting process animation of the hoisting equipment model and the beam model can be displayed in the three-dimensional real-scene model, so that the hoisting equipment operators can clearly understand the requirements for hoisting construction and path planning; during the movement of the hoisting equipment and the hoisting of the beam, the AR software on the smart mobile terminal uses its camera device to capture and monitor the position of the hoisting equipment and the beam in real time and compare the position with the travel path and the hoisting path. If there is a problem in the monitoring, a danger warning will be issued.

Claims

1. A method for early warning of hoisting construction hazards based on AR technology, characterized in that... The early warning method includes the following steps: S1: The environment of the construction site is scanned using a smart mobile terminal with a camera device. Based on the coordinates of the existing structures at the construction site, a three-dimensional real-scene model of the construction site is established. The existing structures in the three-dimensional real-scene model are used as clustering feature points. S2: Create a 3D model of the hoisting equipment and a 3D model of the beam; simulate the hoisting process in the 3D model, simulate the travel path of the 3D model of the hoisting equipment, and mark the hoisting points on the travel path in sequence where the 3D model of the hoisting equipment needs to stop moving for hoisting; simulate the number of the 3D model of the beam to be hoisted and the hoisting path at each hoisting point, and distribute several hoisting position verification points and beam erection position points at intervals on the hoisting path; S3: Before the hoisting operation, the construction personnel use AR software on a smart mobile terminal to scan and detect the construction site. During the scanning and detection process, they search for the clustered feature points located on the horizontal or vertical plane, and match the clustered feature points with the corresponding three-dimensional real-scene models. The matched three-dimensional real-scene models are displayed on the AR software interface of the smart mobile terminal, and a simulated hoisting process animation of the three-dimensional model of the hoisting equipment and the three-dimensional model of the beam is displayed. S4: Fix the smart mobile terminal with a bracket so that the display angle of the three-dimensional real scene model changes with the shooting angle of the smart mobile terminal. Adjust the shooting angle of the camera device on the smart mobile terminal so that the shooting screen can monitor the movement of the hoisting equipment and the hoisting and erection of the beam. S5: Construction personnel operate based on the simulated hoisting process animation: S5.1: Control the hoisting equipment to move to the first hoisting point on the travel path and stop moving. The AR software on the smart mobile terminal captures the stopped position of the hoisting equipment and determines whether it coincides with the position of the hoisting point on the travel path. If they coincide, proceed to the next hoisting operation; if they do not coincide, issue a warning to cause the hoisting equipment to adjust its position until the stopped position of the hoisting equipment coincides with the position of the first hoisting point on the travel path. S5.2: Several beams are placed at the construction site, each beam having a number marked on its upper surface. The beams with the corresponding numbers are hoisted using the hoisting equipment. The AR software on the smart mobile terminal captures and identifies the numbers on the hoisted beams to determine if they correspond to the correct number. If not, a warning is issued, prompting the hoisting equipment to re-hoist the beam with the correct number. Then, the beams are hoisted onto the bridge using the hoisting equipment. The AR software on the smart mobile terminal captures whether the beam passes through each hoisting position verification point sequentially during the hoisting process. If the beam fails to pass two or more hoisting position verification points consecutively during the hoisting process, a warning is issued, causing the beam to stop moving and be adjusted. Before releasing the hoisting equipment from the bridge, the AR software on the smart mobile terminal captures whether the beam has been erected at the beam erection position. If so, the hoisting equipment releases the beam; otherwise, the beam is lifted and the erection position is adjusted until it reaches the preset position. S5.3: Following the method in steps S5.1-S5.2, control the hoisting equipment to move sequentially to each of the hoisting points and erect the corresponding numbered beams onto the bridge in sequence.

2. The method for early warning of hoisting construction hazards based on AR technology according to claim 1, characterized in that... The smart mobile terminal is a smartphone or tablet computer.

3. The method for early warning of hoisting construction hazards based on AR technology according to claim 1, characterized in that... The AR software of the smart mobile terminal communicates with a cloud server, which stores several 3D real-scene models, several 3D models of hoisting equipment, several 3D models of beams, and several animations simulating the hoisting process. When construction personnel scan and detect the construction site using the AR software on the smart mobile terminal, they upload the clustering feature points of the existing structures at the construction site to the cloud server. They then query and compare the detected clustering feature points with the clustering feature points contained in each of the 3D real-scene models to obtain a 3D real-scene model that matches the construction site environment.

Images