Intelligent assembling method and device for steel structural component based on multi-source data and medium

Abstract

The invention discloses an intelligent assembly method and device for steel structural members based on multi-source data and a medium. The method comprises the steps that a three-dimensional digital twin model containing all the steel structural members is constructed based on a BIM design model; acquiring point cloud data of a construction site at the current moment, positioning data of each steel structural member and image data of a key control point on each steel structural member through a multi-source data acquisition system; fusing the multi-source data based on a preset data fusion algorithm to obtain state data of each steel structural member in the construction site at the current moment, and estimating a motion track of the corresponding steel structural member at the next moment based on the state data of each steel structural member at the current moment, and whether the corresponding steel structural member has an assembly risk is judged by combining a three-dimensional data twinborn model, and then an early warning notification is generated according to a judgment result. Risk prediction of intelligent assembly of the steel structural member can be realized, and manual operation is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent assembly, and in particular to an intelligent assembly method, device and medium for steel structure components based on multi-source data. BACKGROUND

[0002] The existing steel structure assembly method mainly relies on discrete measurement of total station and manual experience for hoisting and positioning, which has the problems of that the measurement data cannot be synchronized with the virtual model in real time, and it is difficult to realize remote control of assembly; at the same time, the adjustment of the posture of the steel structure component is seriously dependent on the experience of workers, and the assembly risk is large, especially for large and complex steel structure components, since the assembly will produce deviation, large-scale rework is often needed, resulting in low assembly efficiency. SUMMARY

[0003] In order to overcome the defects of the prior art, one of the purposes of the present application is to provide an intelligent assembly method for steel structure components based on multi-source data, which can solve the problems of that the assembly process of the steel structure components cannot be remotely controlled and is dependent on manual work in the prior art.

[0004] The second purpose of the present application is to provide an intelligent assembly device for steel structure components based on multi-source data, which can solve the problems of that the assembly process of the steel structure components cannot be remotely controlled and is dependent on manual work in the prior art.

[0005] The third purpose of the present application is to provide a computer readable storage medium, which can solve the problems of that the assembly process of the steel structure components cannot be remotely controlled and is dependent on manual work in the prior art.

[0006] One of the purposes of the present application is achieved by adopting the following technical solutions:

[0007] The intelligent assembly method for steel structure components based on multi-source data comprises:

[0008] A model construction step: obtaining design data of all steel structure components to be assembled according to project requirements, and constructing a three-dimensional digital twin model containing all steel structure components based on a BIM design model, while presetting theoretical installation coordinates, installation posture, key control points of each steel structure component and connection relationship between steel structure components in the three-dimensional digital twin model;

[0009] A data acquisition step: acquiring point cloud data of a construction site, positioning data of each steel structure component and image data of key control points on each steel structure component at the current time through a multi-source data acquisition system;

[0010] Data fusion steps: Based on a preset data fusion algorithm, the point cloud data of the construction site, the UWB positioning data of each steel structure component, and the image data of each key control point on each steel structure component are fused to obtain the status data of each steel structure component in the construction site at the current moment.

[0011] Early warning judgment steps: Based on the state data of each steel structure component at the current moment, the motion trajectory of the corresponding steel structure component at the next moment is estimated, and the motion trajectory of each steel structure component at the next moment is compared with the ideal installation coordinates of the corresponding steel structure component in the three-dimensional data twin model to determine whether there is an assembly risk of the corresponding steel structure component, and then an early warning notification is generated based on the judgment result.

[0012] Furthermore, the data acquisition steps specifically include: scanning the construction site using a 3D laser scanner installed at a high point or on a mobile platform to obtain point cloud data of the construction site, and then constructing a simulated construction site environment and the macroscopic outline and location range of each steel structure component within the construction site based on the point cloud data; acquiring monitoring data of UWB positioning tags located at the center or suspension point of each steel structure component using UWB base stations installed at the construction site to obtain UWB positioning data of each steel structure component; capturing key feature points of each steel structure component using visual sensors installed at the target installation nodes to obtain real-time image data of each steel structure component, and then obtaining the six-degree-of-freedom attitude of each key control point of each steel structure component; each key control point of the steel structure component is provided with a preset identifier, which includes a preset QR code, a special reflective mark, or a bolt hole.

[0013] Furthermore, prior to the data acquisition step, the system includes a data calibration step: calibrating the 3D laser scanner, vision sensor, and UWB base station by arranging calibration target balls or chessboard grids with known world coordinates in the field; and generating a clock synchronization signal through the system encoder and sending the clock synchronization signal to the 3D laser scanning device, UWB base station, vision sensor, and stress sensor in real time to achieve clock synchronization.

[0014] Furthermore, the data fusion step specifically includes: firstly, based on the state data of each steel structure component at the previous moment and the preset motion model, and the point cloud data of the construction site at the current moment, the state data of each steel structure component at the current moment is predicted; then, based on the UWB positioning data at the current moment, it is used as the observation value of the position data of the corresponding steel structure component and input into the Kalman filter to correct the position of the corresponding steel structure component, so as to realize the positioning of the corresponding steel structure component; then, based on the image data of the key control points of the corresponding steel structure component obtained by the vision sensor, the attitude of each key control point is identified and calculated, and it is input into the Kalman filter to correct the attitude of the corresponding steel structure component; then, based on the positioning and attitude of each steel structure component at the current moment, the global coordinates, attitude data and hoisting speed of each steel structure component at the current moment are obtained, and combined with the timestamp and steel structure component ID, the state dataset of each steel structure component at the current moment is obtained.

[0015] Furthermore, the data fusion step specifically includes: when the image data of the visual sensor cannot be recognized or is inaccurate, the pitch angle and roll angle of the corresponding steel structure component are obtained by acquiring the tilt angle data of the tilt sensor installed on the corresponding steel structure component, so as to correct the attitude of the corresponding steel structure component.

[0016] Furthermore, the assembly risks in the early warning judgment step include deviation risks and collision risks; the early warning judgment step also specifically includes: when there is a deviation risk or collision risk for the corresponding steel structure component, obtaining the deviation value of the corresponding steel structure component or the positional relationship between the corresponding steel structure component and the colliding object, generating an adjustment strategy based on a preset rule base and machine learning model, and pushing the adjustment strategy to the staff so that the staff can control the construction equipment at the construction site according to the adjustment strategy or send control commands to the construction equipment at the corresponding construction site according to the adjustment strategy to achieve automatic control of the construction equipment; the adjustment strategy includes the displacement and rotation of the steel structure component that needs to be adjusted, as well as the adjustment path of the steel structure component.

[0017] Furthermore, the data acquisition step also specifically includes: acquiring the stress changes of the corresponding steel structure component during hoisting or adjustment by using strain sensors installed on the key stress-bearing sections of the corresponding steel structure component;

[0018] The early warning judgment step further includes: loading the stress data and state vector set obtained at the current moment into the three-dimensional digital twin model to display the position information, attitude and stress cloud map of each steel structure component and the corresponding stress monitoring point; at the same time, combining the AI ​​large model to judge whether the stress data of each steel structure component exceeds the limit under the corresponding attitude, and then generating a corresponding early warning notification based on the judgment result.

[0019] Furthermore, the data acquisition step also includes: preprocessing the point cloud data of the construction site at the current moment, the positioning data of each steel structure component, and the image data of the key control points on each steel structure component; the preprocessing includes data clock synchronization and spatial registration.

[0020] The second objective of this invention is achieved by the following technical solution:

[0021] The intelligent assembly device based on multi-source data includes a memory and a processor. The memory stores an intelligent assembly program that runs on the processor. The intelligent assembly program is a computer program. When the processor executes the intelligent assembly program, it implements the steps of the intelligent assembly method based on multi-source data as one of the objectives of this invention.

[0022] The third objective of this invention is achieved by the following technical solution:

[0023] A computer-readable storage medium storing an intelligent assembly program thereon, the intelligent assembly program being a computer program, wherein when the intelligent assembly program is executed by a processor, the steps of an intelligent assembly method based on multi-source data are used to achieve one of the objectives of the present invention.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention introduces a digital twin model and sets up a multi-source data acquisition system at the construction site to obtain multi-source data, enabling dynamic monitoring of each steel structure component during the assembly process. Furthermore, by combining the digital twin model of the components, it can predict potential risks during assembly, ensuring timely detection and safe assembly. This achieves intelligent assembly of steel structure components without excessive reliance on manual labor. Simultaneously, because the assembly process can be monitored in real time, remote control of the assembly of steel structure components can also be achieved, improving assembly efficiency and reducing rework costs. Attached Figure Description

[0026] Figure 1 The flowchart illustrates the intelligent assembly method for steel structure components based on multi-source data provided by this invention. Detailed Implementation

[0027] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0028] Example 1

[0029] This invention combines digital twin models, multi-source sensing, AI decision-making, and point cloud data to achieve data monitoring of steel structure components during assembly, forming an intelligent assembly closed loop capable of proactive real-time correction, thereby improving the assembly efficiency of steel structure components. Specifically, this invention provides a preferred embodiment of an intelligent assembly method for steel structure components based on multi-source data, such as... Figure 1 As shown, it includes:

[0030] Step S1: Obtain the design data of all steel structure components to be assembled according to project requirements, and build a three-dimensional digital twin model containing all steel structure components based on the BIM design model. At the same time, preset the theoretical installation coordinates, installation posture, key control points, and connection relationships between steel structure components in the three-dimensional digital twin model.

[0031] By constructing a three-dimensional digital twin model corresponding to each steel structure component, the geometric information, physical properties, and behavioral rules of each steel structure component are integrated. The geometric information refers to the parameters and dimensional data of the steel structure component, the physical properties include data such as the mass, material, and stiffness of the steel structure component, and the behavioral rules include data such as the installation sequence and hoisting constraints.

[0032] Simultaneously, the theoretical installation coordinates, installation posture, key control points, and connection relationships between steel structure components are pre-set for each steel structure component in the digital twin model. This invention constructs a three-dimensional digital twin model synchronized with the physical steel structure component to synchronize posture changes during the assembly process in real time, promptly detect assembly errors, and provide timely warnings.

[0033] Step S2: Obtain point cloud data of the construction site at the current moment, positioning data of each steel structure component, and image data of key control points on each steel structure component through a multi-source data acquisition system.

[0034] Specifically, the multi-source data acquisition system of this invention includes a 3D laser scanner, a positioning device, and a vision sensor, which are used to acquire point cloud data of the construction site, positioning data of steel structure components, and image data of key control points of steel structure components, respectively, so as to realize synchronous simulation of the construction site during the construction process and monitoring of the position, attitude, and speed of each steel structure component in the construction site. At the same time, it combines the AI ​​decision-making or AI model in the system background to realize the risks existing in the assembly process of steel structure components.

[0035] Specifically, a 3D laser scanner is deployed at a high point or on a mobile platform at the construction site. By performing a macroscopic scan of the entire construction site, it obtains point cloud data of the entire construction site, which is used to reconstruct the environment of the construction site, thereby obtaining the macroscopic outline and location range (i.e., approximate location) of each steel structure component in the construction site.

[0036] The positioning equipment includes UWB positioning base stations and UWB positioning tags. The UWB positioning base stations are fixed around the construction site, and the UWB positioning tags are installed at the center or suspension point of each steel structure component. The UWB positioning base stations can collect the position coordinates of each UWB positioning tag to obtain the position coordinates of each steel structure component, thereby realizing dynamic tracking of the position of the steel structure component.

[0037] Visual sensors, including high-frame-rate industrial cameras, are positioned around the installation location of the target steel structure component to capture image data of key feature points. This image data is then used to calculate the six-degree-of-freedom pose of the key feature points, thereby identifying the posture of the steel structure component. These key feature points can be preset QR codes, affixed reflective markings, or bolt holes, etc.

[0038] Furthermore, this invention relates to multi-source data. Therefore, it is necessary to ensure clock synchronization of devices from different equipment and data sources. This requires preprocessing the acquired point cloud data, positioning data, and image data to achieve data clock synchronization and spatial registration, thereby eliminating spatiotemporal inconsistencies between data and generating a unified, time-aligned flow of on-site status data. Specifically, this invention uses an encoder in the system to generate a clock synchronization signal, which is then synchronously sent to the 3D laser scanner, UWB base station, and vision sensor in real time. Simultaneously, when these devices upload data, they automatically set a timestamp before sending their own collected data. This ensures that when the backend receives data from the multi-source data acquisition system, it synchronizes the data clock based on the timestamp of each monitored data point.

[0039] Furthermore, before starting the measurement, this invention requires pre-calibration of multiple data acquisition terminals within the multi-source data acquisition system to achieve spatial registration of the data. Specifically, by setting up calibration objects with known world coordinates within the construction site, and then using the world coordinates of the calibration target ball or checkerboard, a coordinate transformation matrix is ​​achieved between each data acquisition terminal and the world coordinate system. For example, point cloud data from a 3D laser scanner can be converted to world coordinates through a coordinate transformation matrix. By obtaining the coordinates of the UWB base station in the world coordinate system, the position of each UWB positioning tag can be accurately located in the world coordinate system. This invention, by constructing a world coordinate system and setting up calibration objects within the construction site, can achieve spatial registration of point cloud data, UWB positioning data, and image data, thereby enabling automatic tracking of the position and orientation of steel structure components.

[0040] Step S3: Based on the preset data fusion algorithm, the point cloud data of the construction site, the UWB positioning data of each steel structure component, and the image data of each key control point on each steel structure component are fused to obtain the status data of each steel structure component in the construction site at the current moment.

[0041] This invention uses a data fusion algorithm to fuse collected multi-source data to calculate the global coordinates, attitude, and hoisting speed of each steel structure component on the construction site, thereby constructing a state vector set for each steel structure component at the current moment.

[0042] Furthermore, step S3 also includes: firstly, predicting the state data of each steel structure component at the current moment based on the state data of each steel structure component at the previous moment, the preset motion model, and the point cloud data of the construction site at the current moment; then, using the UWB positioning data at the current moment as the observation value of the position data of the corresponding steel structure component and inputting it into the Kalman filter to correct the position of the corresponding steel structure component, so as to realize the positioning of the corresponding steel structure component; then, identifying and solving the attitude of each key control point based on the image data of the key control points of the corresponding steel structure component obtained by the vision sensor, and inputting it into the Kalman filter to correct the attitude of the corresponding steel structure component; then, based on the positioning and attitude of each steel structure component at the current moment, obtaining the global coordinates, attitude data, and hoisting speed of each steel structure component at the current moment, and combining the timestamp and steel structure component ID to obtain the state dataset of each steel structure component at the current moment.

[0043] The preset motion model refers to a model pre-built based on artificial intelligence. By using data such as the position, hoisting speed, and attitude of the steel structure components at the previous moment, the position, attitude, and other state data of the steel structure components at the next moment can be predicted. At the same time, it also combines the point cloud data of the current 3D laser scanning data to achieve the preliminary positioning of each steel structure component at the current moment.

[0044] When the system receives the UWB positioning data at the current moment, it can use it as the observation value of the position data of the corresponding steel structure component and input it into a filter, such as an extended Kalman filter or an unscented Kalman filter, to correct the predicted position data of the corresponding steel structure component and thus achieve the positioning of the corresponding steel structure component.

[0045] Similarly, when the system receives the image data from the vision sensor at the current moment, it calculates the pose of each key feature point based on the image data, and then inputs it as the pose observation value of the corresponding steel structure component into the Kalman filter to correct the predicted pose of the steel structure component.

[0046] By fusing and correcting the above data, and combining it with the positioning and attitude of each steel structure component at the current moment, the state data of each steel structure component at the current moment can be obtained, including global coordinates, attitude data, and hoisting speed. This data, combined with the timestamp and steel structure component ID, yields the state dataset of each steel structure component at the current moment. The hoisting speed refers to the moving speed of the steel structure component, which can be obtained from the hoisting equipment or calculated from the position data of the steel structure component at previous and subsequent moments and the data acquisition time interval.

[0047] Specifically, in practical use, the visual sensor may acquire inaccurate or fail to recognize image data due to equipment obstruction or other reasons. To ensure the correction of the steel structure component's attitude, this invention also installs a tilt sensor on the steel structure component to obtain its tilt angle data. When the visual sensor fails to recognize image data or the recognized image data is inaccurate, the tilt angle data can be activated to correct the attitude of the corresponding steel structure component. Since the pitch and roll angles of the steel structure component can be absolutely predicted, continuous attitude correction of the steel structure component can be achieved, which plays a crucial role, especially when the visual sensor is temporarily obstructed.

[0048] Specifically, when steel structure components are being hoisted normally at the construction site, due to good alignment, the vision sensor can identify all key control points on the steel structure component, and the acquired image data is clear. Therefore, in the Kalman filter, the noise covariance of the vision sensor's observations is set very small, allowing image data to be used as the dominant source of attitude correction for the steel structure component. However, when the steel structure component passes through a supporting space frame or other equipment, causing the vision sensor's camera to be obstructed, the vision sensor may only identify one key control point on the steel structure component, and the image may be blurry. To ensure the accuracy of the correction data, this invention uses tilt sensor data to correct the attitude of the steel structure component. Specifically, this invention dynamically increases the noise covariance of the vision sensor's observations in the Kalman filter to increase the weight of the vision sensor's image data. This allows the system to combine UWB positioning data and tilt sensor tilt data to correct the state data of the steel structure component. Simultaneously, when a steel structure component protrudes from an obstruction or support, the image data acquisition by the vision sensor can be restarted to correct the posture of the steel structure component. Using the newly acquired high-precision visual data, UWB and prediction errors that may have accumulated during the obstruction period are quickly corrected, allowing the system state to rapidly converge back to the true high-precision state. In other words, this invention can identify the reliability of each data source by setting a confidence level within the system, and then dynamically adjust the noise covariance of the corresponding observation in the Kalman filter based on the confidence level. This allows for the selection of an appropriate combination to correct the state data of the steel structure component, ensuring posture correction even when the vision sensor is unavailable.

[0049] Specifically, this invention can evaluate the usability of image data from a visual sensor by setting quantitative indicators, including image sharpness, signal-to-noise ratio (SNR), feature extraction reliability, and reprojection error. Image sharpness can be measured by calculating the image gradient. The SNR is lowered when there is insufficient lighting or excessive interference, as the noise level in the image is assessed. Feature extraction reliability can be determined by calculating the number of identified key control points or the recognition rate, such as the ratio of successfully identified key control points to a preset number of key control points. That is, a low recognition rate indicates that the steel structure components are heavily obstructed, resulting in fewer key control points being acquired from the image data; therefore, a tilt sensor can be activated to correct the posture. Reprojection error involves backprojecting the calculated 3D pose back into the 2D image and calculating the average pixel error compared to the actual pixels. For example, an average pixel error of <0.5 pixels indicates excellent image quality; conversely, an average pixel error indicates poor image quality.

[0050] In addition, for UWB positioning data, this invention can also determine the availability of UWB positioning data based on signal quality. Specific quantitative indicators include signal-to-noise ratio (SNR), multipath effect indication (MPI), and geometric accuracy. Among them, SNR refers to the SNR value of the UWB received signal, which directly reflects whether the signal is pure. MPI is determined by channel impulse response analysis to determine whether there is strong multipath interference. The geometric accuracy factor is a coefficient calculated based on the spatial geometry of the UWB positioning tag and multiple UWB positioning base stations.

[0051] For tilt sensors, the usability of the tilt sensor data can be determined by the stability of the tilt angle data. For example, the standard deviation of high-frequency vibrations in the readings within a short time window can be calculated. For instance, a sudden change in the tilt angle data of the tilt sensor may occur due to a sudden movement of the crane or a strong wind.

[0052] The confidence level of a single sensor can be calculated by combining multiple evaluation metrics. For example, in calculating the confidence level of a vision sensor, metrics with different dimensions can be mapped to the 0-1 range, and then the scores of each quantified metric can be calculated. Finally, a weighted fusion method can be used to calculate the confidence level. Furthermore, when determining the threshold for judging whether a sensor is good or bad, extensive offline calibration experiments can be conducted before the system is put into use to determine the sensor's performance baseline, optimal operating range, and the weights and threshold parameters in the aforementioned calculation formula. For instance, the vision system can be tested under various lighting conditions to determine a reasonable threshold for sharpness scores.

[0053] Step S4: Based on the state vector set of each steel structure component at the current moment, predict the motion trajectory of the corresponding steel structure component at the next moment, and compare the predicted motion trajectory of each steel structure component at the next moment with the ideal installation coordinates of the corresponding steel structure component in the three-dimensional data twin model to determine whether there is an assembly risk for the corresponding steel structure component, and then generate an early warning notification based on the judgment result.

[0054] This invention predicts the motion trajectory of steel structure components and compares it with the corresponding project establishment and installation coordinates of the steel structure components in the constructed three-dimensional data twin model to determine whether there are assembly risks for the corresponding steel structure components, thereby realizing the assembly monitoring of steel structure components.

[0055] Specifically, the assembly risks in this invention include deviation risk and collision risk. Deviation risk refers to the discrepancy between the installation position of the steel structure component, obtained from its predicted motion trajectory, and its ideal installation coordinates. Collision risk refers to the potential collision between the installation position of the steel structure component, obtained from its predicted motion trajectory, and other steel structure components or equipment. Therefore, when a steel structure component faces deviation or collision risk, the deviation value of the corresponding steel structure component or its positional relationship with the colliding object is obtained. An adjustment strategy is generated based on a preset rule base and a machine learning model, and this strategy is pushed to the workers. This allows the workers to control the construction equipment at the construction site according to the adjustment strategy or to send control commands to the construction equipment at the corresponding construction site to achieve automatic control of the construction equipment. The adjustment strategy includes the displacement and rotation of the steel structure component to be adjusted, as well as the adjustment path of the steel structure component.

[0056] This invention enables timely risk prediction for steel structural components with assembly risks, allowing for prompt identification of potential risks and timely guidance or control of hoisting equipment to take corresponding adjustment measures. This ensures correct and safe assembly of steel structural components, achieving true intelligent assembly and reducing manual control over the assembly process and inspection of the assembly results. More preferably, this invention also uses strain sensors installed on the key stress sections of the corresponding steel structural components to acquire stress changes during hoisting or adjustment. Specifically, the stress data and state vector set acquired at the current moment are loaded into a three-dimensional digital twin model to display the position information, attitude, and stress cloud map of each steel structural component at the corresponding stress monitoring point. Simultaneously, an AI model is used to determine whether the stress data of each steel structural component exceeds the limit under the corresponding attitude, and then a corresponding early warning notification is generated based on the judgment result.

[0057] Normally, when steel structure components are assembled properly, their stress and deformation data are within a preset reasonable range. Once the stress data (stress and deformation data) is detected to exceed the reasonable range, it can be determined that the assembly of the steel structure components is illegal. For example, tilting or incorrect assembly will cause the stress or deformation data of the steel structure components to exceed the limit. Therefore, this invention also sets stress sensors at key stress points to monitor the corresponding forces. Once the stress data is detected to exceed the limit, an early warning notification is generated to promptly remind the construction personnel so that necessary measures can be taken.

[0058] This invention achieves real-time correction of the intelligent assembly of steel structure components by deeply integrating three-dimensional digital twin models, multi-data perception, and artificial intelligence and AI decision-making. It can automatically synchronize the assembly process of steel structure components and combine AI large models to simulate and predict motion trajectories and mechanical states in advance, thereby realizing risk prevention in the assembly of steel structure components, greatly reducing the risks in the assembly process, ensuring assembly safety, avoiding rework, and reducing rework costs.

[0059] Example 2

[0060] Based on Embodiment 1, the present invention also provides another embodiment: an intelligent assembly device based on multi-source data, including a memory and a processor. The memory stores an intelligent assembly program that runs on the processor. The intelligent assembly program is a computer program, and when the processor executes the intelligent assembly program, it performs the following steps:

[0061] Model building steps: Obtain the design data of all steel structure components to be assembled according to project requirements, and build a three-dimensional digital twin model containing all steel structure components based on the BIM design model. At the same time, preset the theoretical installation coordinates, installation posture, key control points and connection relationships between steel structure components in the three-dimensional digital twin model.

[0062] Data acquisition steps: Acquire point cloud data of the construction site at the current moment, positioning data of each steel structure component, and image data of key control points on each steel structure component through a multi-source data acquisition system;

[0063] Data fusion steps: Based on a preset data fusion algorithm, the point cloud data of the construction site, the UWB positioning data of each steel structure component, and the image data of each key control point on each steel structure component are fused to obtain the status data of each steel structure component in the construction site at the current moment.

[0064] Early warning judgment steps: Based on the state data of each steel structure component at the current moment, the motion trajectory of the corresponding steel structure component at the next moment is estimated, and the motion trajectory of each steel structure component at the next moment is compared with the ideal installation coordinates of the corresponding steel structure component in the three-dimensional data twin model to determine whether there is an assembly risk of the corresponding steel structure component, and then an early warning notification is generated based on the judgment result.

[0065] Furthermore, the data acquisition steps specifically include: scanning the construction site using a 3D laser scanner installed at a high point or on a mobile platform to obtain point cloud data of the construction site, and then constructing a simulated construction site environment and the macroscopic outline and location range of each steel structure component based on the point cloud data; obtaining UWB positioning data of each steel structure component by acquiring monitoring data from UWB positioning tags located at the center or suspension point of each steel structure component using UWB base stations installed at the construction site; capturing real-time image data of each steel structure component by using visual sensors installed at the target installation nodes to photograph key feature points of each steel structure component, and then obtaining the six-degree-of-freedom attitude of each key control point of each steel structure component; each key control point of the steel structure component is equipped with a preset identifier, which includes a preset QR code, a special reflective mark, or a bolt hole.

[0066] Furthermore, prior to the data acquisition step, the system includes a data calibration step: calibrating the 3D laser scanner, vision sensor, and UWB base station by arranging calibration target balls or chessboard grids with known world coordinates in the field; and generating a clock synchronization signal through the system encoder and sending the clock synchronization signal to the 3D laser scanning equipment, UWB base station, vision sensor, and stress sensor in real time to achieve clock synchronization.

[0067] Furthermore, the data fusion steps specifically include: First, based on the state data of each steel structure component at the previous moment and the preset motion model, and the point cloud data of the construction site at the current moment, the state data of each steel structure component at the current moment is predicted. Then, based on the UWB positioning data at the current moment, it is used as the observation value of the position data of the corresponding steel structure component and input into the Kalman filter to correct the position of the corresponding steel structure component, so as to realize the positioning of the corresponding steel structure component. Then, based on the image data of the key control points of the corresponding steel structure component obtained by the vision sensor, the attitude of each key control point is identified and calculated, and it is input into the Kalman filter to correct the attitude of the corresponding steel structure component. Finally, based on the positioning and attitude of each steel structure component at the current moment, the global coordinates, attitude data and hoisting speed of each steel structure component at the current moment are obtained, and combined with the timestamp and steel structure component ID, the state dataset of each steel structure component at the current moment is obtained.

[0068] Furthermore, the data fusion step also specifically includes: when the image data of the visual sensor cannot be recognized or is inaccurate, the pitch angle and roll angle of the corresponding steel structure component are obtained by acquiring the tilt angle data of the tilt sensor installed on the corresponding steel structure component, so as to correct the attitude of the corresponding steel structure component.

[0069] Furthermore, the assembly risks in the early warning judgment step include deviation risks and collision risks; the early warning judgment step also specifically includes: when there is a deviation risk or collision risk for the corresponding steel structure component, obtaining the deviation value of the corresponding steel structure component or the positional relationship between the corresponding steel structure component and the colliding object, generating an adjustment strategy based on a preset rule base and machine learning model, and pushing the adjustment strategy to the staff so that the staff can control the construction equipment on the construction site according to the adjustment strategy or send control commands to the construction equipment on the corresponding construction site according to the adjustment strategy to achieve automatic control of the construction equipment; the adjustment strategy includes the displacement and rotation of the steel structure component that needs to be adjusted, as well as the adjustment path of the steel structure component.

[0070] Furthermore, the data acquisition steps also specifically include: obtaining stress changes of the corresponding steel structure component during hoisting or adjustment by using strain sensors installed on the key stress-bearing sections of the corresponding steel structure component;

[0071] The early warning judgment steps also include: loading the stress data and state vector set acquired at the current moment into a three-dimensional digital twin model to display the position information, attitude, and stress cloud map of each steel structure component and the corresponding stress monitoring point; at the same time, combining the AI ​​large model to judge whether the stress data of each steel structure component exceeds the limit under the corresponding attitude, and then generating a corresponding early warning notification based on the judgment result.

[0072] Furthermore, the data acquisition steps also include: preprocessing the point cloud data of the construction site at the current moment, the positioning data of each steel structure component, and the image data of the key control points on each steel structure component; the preprocessing includes data clock synchronization and spatial registration.

[0073] Example 3

[0074] Based on Embodiment 1, the present invention also provides another embodiment: a computer-readable storage medium storing an intelligent assembly program thereon. The intelligent assembly program is a computer program, and when executed by a processor, it performs the following steps:

[0075] Model building steps: Obtain the design data of all steel structure components to be assembled according to project requirements, and build a three-dimensional digital twin model containing all steel structure components based on the BIM design model. At the same time, preset the theoretical installation coordinates, installation posture, key control points and connection relationships between steel structure components in the three-dimensional digital twin model.

[0076] Data acquisition steps: Acquire point cloud data of the construction site at the current moment, positioning data of each steel structure component, and image data of key control points on each steel structure component through a multi-source data acquisition system;

[0077] Data fusion steps: Based on a preset data fusion algorithm, the point cloud data of the construction site, the UWB positioning data of each steel structure component, and the image data of each key control point on each steel structure component are fused to obtain the status data of each steel structure component in the construction site at the current moment.

[0078] Early warning judgment steps: Based on the state data of each steel structure component at the current moment, the motion trajectory of the corresponding steel structure component at the next moment is estimated, and the motion trajectory of each steel structure component at the next moment is compared with the ideal installation coordinates of the corresponding steel structure component in the three-dimensional data twin model to determine whether there is an assembly risk of the corresponding steel structure component, and then an early warning notification is generated based on the judgment result.

[0079] Furthermore, the data acquisition steps specifically include: scanning the construction site using a 3D laser scanner installed at a high point or on a mobile platform to obtain point cloud data of the construction site, and then constructing a simulated construction site environment and the macroscopic outline and location range of each steel structure component based on the point cloud data; obtaining UWB positioning data of each steel structure component by acquiring monitoring data from UWB positioning tags located at the center or suspension point of each steel structure component using UWB base stations installed at the construction site; capturing real-time image data of each steel structure component by using visual sensors installed at the target installation nodes to photograph key feature points of each steel structure component, and then obtaining the six-degree-of-freedom attitude of each key control point of each steel structure component; each key control point of the steel structure component is equipped with a preset identifier, which includes a preset QR code, a special reflective mark, or a bolt hole.

[0080] Furthermore, prior to the data acquisition step, the system includes a data calibration step: calibrating the 3D laser scanner, vision sensor, and UWB base station by arranging calibration target balls or chessboard grids with known world coordinates in the field; and generating a clock synchronization signal through the system encoder and sending the clock synchronization signal to the 3D laser scanning equipment, UWB base station, vision sensor, and stress sensor in real time to achieve clock synchronization.

[0081] Furthermore, the data fusion steps specifically include: First, based on the state data of each steel structure component at the previous moment and the preset motion model, and the point cloud data of the construction site at the current moment, the state data of each steel structure component at the current moment is predicted. Then, based on the UWB positioning data at the current moment, it is used as the observation value of the position data of the corresponding steel structure component and input into the Kalman filter to correct the position of the corresponding steel structure component, so as to realize the positioning of the corresponding steel structure component. Then, based on the image data of the key control points of the corresponding steel structure component obtained by the vision sensor, the attitude of each key control point is identified and calculated, and it is input into the Kalman filter to correct the attitude of the corresponding steel structure component. Finally, based on the positioning and attitude of each steel structure component at the current moment, the global coordinates, attitude data and hoisting speed of each steel structure component at the current moment are obtained, and combined with the timestamp and steel structure component ID, the state dataset of each steel structure component at the current moment is obtained.

[0082] Furthermore, the data fusion step also specifically includes: when the image data of the visual sensor cannot be recognized or is inaccurate, the pitch angle and roll angle of the corresponding steel structure component are obtained by acquiring the tilt angle data of the tilt sensor installed on the corresponding steel structure component, so as to correct the attitude of the corresponding steel structure component.

[0083] Furthermore, the assembly risks in the early warning judgment step include deviation risks and collision risks; the early warning judgment step also specifically includes: when there is a deviation risk or collision risk for the corresponding steel structure component, obtaining the deviation value of the corresponding steel structure component or the positional relationship between the corresponding steel structure component and the colliding object, generating an adjustment strategy based on a preset rule base and machine learning model, and pushing the adjustment strategy to the staff so that the staff can control the construction equipment on the construction site according to the adjustment strategy or send control commands to the construction equipment on the corresponding construction site according to the adjustment strategy to achieve automatic control of the construction equipment; the adjustment strategy includes the displacement and rotation of the steel structure component that needs to be adjusted, as well as the adjustment path of the steel structure component.

[0084] Furthermore, the data acquisition steps also specifically include: obtaining stress changes of the corresponding steel structure component during hoisting or adjustment by using strain sensors installed on the key stress-bearing sections of the corresponding steel structure component;

[0085] The early warning judgment steps also include: loading the stress data and state vector set acquired at the current moment into a three-dimensional digital twin model to display the position information, attitude, and stress cloud map of each steel structure component and the corresponding stress monitoring point; at the same time, combining the AI ​​large model to judge whether the stress data of each steel structure component exceeds the limit under the corresponding attitude, and then generating a corresponding early warning notification based on the judgment result.

[0086] Furthermore, the data acquisition steps also include: preprocessing the point cloud data of the construction site at the current moment, the positioning data of each steel structure component, and the image data of the key control points on each steel structure component; the preprocessing includes data clock synchronization and spatial registration.

[0087] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for intelligent assembly of steel structural members based on multi-source data, characterized in that, The intelligent assembly method comprises: A model construction step: obtaining design data of all steel structure components to be assembled according to project requirements, and constructing a three-dimensional digital twin model comprising all steel structure components based on a BIM design model, while presetting theoretical installation coordinates, installation postures, key control points of each steel structure component and connection relationships between steel structure components in the three-dimensional digital twin model; A data acquisition step: acquiring point cloud data of a construction site, positioning data of each steel structure component and image data of key control points on each steel structure component at the current time through a multi-source data acquisition system; A data fusion step: fusing the point cloud data of the construction site, the UWB positioning data of each steel structure component and the image data of each key control point on each steel structure component based on a preset data fusion algorithm to obtain state data of each steel structure component in the construction site at the current time; A pre-warning judgment step: estimating the motion trajectory of the corresponding steel structure component at the next time based on the state data of each steel structure component at the current time, comparing the motion trajectory of each steel structure component at the next time with the ideal installation coordinates of the corresponding steel structure component in the three-dimensional data twin model to judge whether the corresponding steel structure component has an assembly risk, and generating a pre-warning notification according to the judgment result.

2. The method for intelligent assembly of steel structural members based on multi-source data according to claim 1, characterized in that, The data acquisition step specifically comprises: scanning the construction site by a three-dimensional laser scanner installed on a high point or a mobile platform of the construction site to acquire point cloud data of the construction site, and then constructing a simulated construction site environment and a macroscopic contour and position range of each steel structure component in the construction site according to the point cloud data of the construction site; acquiring monitoring data of a UWB positioning tag arranged on a center or a hoisting point of each steel structure component by a UWB base station installed in the construction site to acquire UWB positioning data of each steel structure component; acquiring image data of each steel structure component by a visual sensor installed at a target installation node to capture key feature points of each steel structure component, and then acquiring a six-degree-of-freedom posture of each key control point of each steel structure component; the key control points of each steel structure component are provided with preset identifiers, and the preset identifiers include preset two-dimensional codes, special reflective markers or bolt holes.

3. The method of claim 2, wherein, The data acquisition step further comprises a data calibration step: arranging a calibration target ball or a checkerboard with known world coordinates in the site to realize system calibration of the three-dimensional laser scanner, the visual sensor and the UWB base station; and generating a clock synchronization signal by a system encoder and sending the clock synchronization signal to the three-dimensional laser scanning device, the UWB base station, the visual sensor and the stress sensor in real time to synchronize the clocks.

4. The method of claim 1, wherein, The data fusion step specifically comprises: firstly, predicting the state data of each steel structure member at the current time based on the state data of each steel structure member at the previous time and the preset motion model, and the point cloud data of the construction site at the current time; then, taking the UWB positioning data at the current time as the observation value of the position data of the corresponding steel structure member and inputting it into the Kalman filter to correct the position of the corresponding steel structure member, so as to realize the positioning of the corresponding steel structure member; then, identifying and calculating the attitude of each key control point of the corresponding steel structure member according to the image data of the key control point obtained by the visual sensor, and inputting it into the Kalman filter to correct the attitude of the corresponding steel structure member; and then, obtaining the global coordinate, attitude data and hoisting speed of each steel structure member at the current time based on the positioning and attitude of each steel structure member at the current time, and combining the time stamp and the steel structure member ID to obtain the state data of each steel structure member at the current time.

5. The method of claim 4, wherein, The data fusion step also specifically comprises: when the image data of the visual sensor cannot be recognized or recognized accurately, the pitch angle and roll angle of the corresponding steel structure member are obtained by acquiring the inclination data of the inclination sensor installed on the corresponding steel structure member, so as to correct the attitude of the corresponding steel structure member.

6. The method of claim 1, wherein, The assembly risk in the early warning judgment step includes deviation risk and collision risk; The early warning judgment step also specifically comprises: when there is deviation risk or collision risk of the corresponding steel structure member, obtaining the deviation value of the corresponding steel structure member or the positional relationship between the corresponding steel structure member and the collision object, and generating an adjustment strategy based on a preset rule base and a machine learning model, and pushing the adjustment strategy to the worker to make the worker control the construction equipment of the construction site according to the adjustment strategy or send a control instruction to the construction equipment of the corresponding construction site according to the adjustment strategy to realize automatic control of the construction equipment; the adjustment strategy includes the displacement and rotation amount of the steel structure member to be adjusted and the adjustment path of the steel structure member.

7. The method for intelligent assembly of steel structural members based on multi-source data according to claim 1, characterized in that, The data acquisition step also specifically comprises: acquiring the stress change of the corresponding steel structure member during hoisting or adjustment by the strain sensor installed on the key stress section of the corresponding steel structure member; The early warning judgment step also includes: loading the stress data and state vector set obtained at the current time into the three-dimensional digital twin model to display the position information, attitude of each steel structure member and stress cloud diagram of the corresponding stress monitoring point, and simultaneously combining an AI large model to judge whether the stress data is out of limit under the corresponding attitude of each steel structure member, and then generating a corresponding early warning notification according to the judgment result.

8. The method of claim 1, wherein, The data acquisition step also includes: preprocessing the point cloud data of the construction site at the current time, the positioning data of each steel structure member and the image data of the key control point on each steel structure member; the preprocessing includes data clock synchronization and spatial registration.

9. An intelligent assembly device based on multi-source data, comprising a memory and a processor, the memory having stored thereon an intelligent assembly program running on the processor, the intelligent assembly program being a computer program, characterized in that, The processor executes the intelligent assembly program to realize the steps of the intelligent assembly method based on multi-source data according to any one of claims 1-8.

10. A computer readable storage medium having stored thereon an intelligent assembly program, the program comprising: The intelligent assembly program is a computer program, and the intelligent assembly program, when executed by a processor, implements the steps of the intelligent assembly method based on multi-source data according to any one of claims 1-8.

Images