Steel structure positioning system and method based on computer vision and AI

Abstract

The invention provides a steel structure positioning system and method based on computer vision and AI, and the system comprises a preset module, a shooting module, an image processing module, a sensor module and an adjustment module, and the preset module is used for setting the preset rectangular structure form of a steel structure vestibule and the accurate poses of four corresponding angular points. The shooting module is used for shooting a first image of the steel structure vestibule in real time, the image processing module is used for obtaining the real-time deviation angle of the steel structure vestibule, the sensor module is used for obtaining the real-time poses of four angular points of the steel structure vestibule and obtaining the real-time pose difference value of the four angular points, and the adjusting module is used for adjusting the steel structure vestibule in real time. The real-time deviation angle is smaller than a preset deviation angle threshold value, and the real-time pose difference values of the four angular points are all smaller than a preset pose difference value threshold value. According to the method, all the joints can be ensured to be qualified in the one-time installation process, so that the consistency and stability of steel structure vestibule positioning are ensured in the long-distance and high-complexity environment.

Description

Technical Field

[0001] The present invention relates to the technical field of steel structure positioning, and in particular to a steel structure positioning system and method based on computer vision and AI. Background Art

[0002] In modern construction projects, steel structures are widely used in the construction of large buildings due to their advantages such as high strength, light weight, durability and recyclability. Especially in the installation process of long-distance steel structure corridors, accurate positioning is the key to ensuring the quality of the project. However, due to human factors and equipment limitations, the traditional theodolite measurement method is difficult to ensure high-precision positioning consistency. The existing solutions mainly achieve the positioning of steel structures through manual measurement and adjustment. Although this method can achieve a certain positioning function, there are problems such as low accuracy, low efficiency and high cost in operation. In addition, due to the strong subjectivity of manual operation, the consistency and stability of the positioning results may also be insufficient. The existing steel structure positioning methods mainly rely on manual operation, which is not only time-consuming and labor-intensive, but also due to the influence of human factors, the accuracy and stability of the positioning results may be insufficient. When facing long-distance and highly complex environments, the existing methods are difficult to ensure the consistency and stability of positioning. When processing large amounts of image data and performing complex calculations, these methods may have problems such as slow processing speed and low accuracy.

[0003] Chinese invention patent CN111677294A discloses a method for intelligent installation of steel structures based on digital twin technology. Before the construction of the steel structure, Revit is used to model the steel structure to obtain a steel structure model; the steel structure model is imported into Ansys for mechanical performance analysis. After the calculation is passed, a construction animation is made for the steel structure nodes, and the construction process is technically explained through VR glasses; during the construction process, the model established in Revit is strictly followed, and each steel structure installation construction step is recorded in time by a 3D scanner to form a point cloud data inverse modeling for comparison with the design model, and the installation of the steel structure entity is adjusted to ensure the consistency of construction with the design; after the steel structure entity is assembled, a light sensor is arranged at each connection node, and the data image of the sensor is used to timely check the construction quality problems and make rectifications; from design to construction, the monitoring of each stage realizes the mutual mapping of virtual reality to achieve digital twins, thereby improving the quality and efficiency of steel structure installation and construction.

[0004] In summary, the currently disclosed intelligent steel structure installation method based on digital twin technology mainly targets the problems of poor installation quality and complicated process of existing steel structure beams and columns. After the steel structure is assembled, light sensors are arranged at the connection nodes to timely check the construction quality problems through the data images of the sensors and make rectifications, instead of ensuring the quality of all nodes in one installation process. Moreover, the relevant technology cannot meet the consistency and stability of the installation and positioning of steel structure corridors in long-distance and highly complex environments. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a steel structure positioning system and method based on computer vision and AI, which can ensure the quality of all nodes during a single installation process, so as to ensure the consistency and stability of the steel structure corridor positioning in a long-distance and high-complexity environment.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] Steel structure positioning system based on computer vision and AI, including:

[0008] A preset module is used to set the preset rectangular structure of the steel structure corridor and the accurate positions of the four corner points corresponding to the preset rectangular structure;

[0009] A shooting module, used for shooting a first image of the steel structure corridor in real time;

[0010] An image processing module is used to preprocess the first image to obtain a second image, identify the real-time rectangular structure form of the steel structure corridor based on the second image, and obtain the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form;

[0011] The sensor module is used to obtain the real-time posture of the four corner points of the steel structure corridor, and obtain the real-time posture difference of the four corner points based on the accurate posture and real-time posture of the four corner points;

[0012] The adjustment module is used to make real-time adjustments to the steel structure corridor until the real-time offset angle is less than a preset offset angle threshold and the real-time posture differences of the four corner points are all less than a preset posture difference threshold.

[0013] Furthermore, the posture includes attitude information and position information, the sensor module includes an inertial measurement unit and a global positioning system, the inertial measurement unit is used to measure the attitude information of the four corner points of the steel structure corridor when moving, the attitude information includes angular velocity and acceleration information, and the global positioning system is used to provide the position information of the four corner points of the steel structure corridor.

[0014] Furthermore, the adjustment module includes a magnetic suction cup, a bracket, a handle and a pushing member, the magnetic suction cup is used to adsorb the steel structure corridor, the bracket is rotatably connected to the magnetic suction cup, the pushing member includes a pushing seat and a pushing shaft, one side of the bracket is fixed to the handle, and the other side is fixed to the pushing seat, the handle is used to hook the steel structure corridor, and the pushing shaft is fixed to the steel structure support frame.

[0015] Further, the handle includes a first U-shaped arm and a second U-shaped arm, the first U-shaped arm is fixed to the bracket, the first U-shaped arm includes a first end and a second end, the second U-shaped arm includes a third end and a fourth end, the first end and the third end are connected by a first straight rod, the second end and the fourth end are connected by a second straight rod, the angle between the first straight rod and the first end is 110-150 degrees, the angle between the third end and the first straight rod is 110-150 degrees, the angle between the second straight rod and the second end is 110-150 degrees, and the angle between the fourth end and the second straight rod is 110-150 degrees.

[0016] The steel structure positioning method based on computer vision and AI is applied to the above-mentioned steel structure positioning system based on computer vision and AI, and includes the following steps:

[0017] Set the preset rectangular structure of the steel structure corridor and the accurate positions of the four corner points corresponding to the preset rectangular structure;

[0018] Real-time capture of the first image of the steel structure corridor;

[0019] Preprocessing the first image to obtain a second image, identifying the real-time rectangular structure form of the steel structure corridor based on the second image, and obtaining the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form;

[0020] Obtain the real-time poses of the four corner points of the steel structure corridor, and obtain the real-time pose differences of the four corner points based on the accurate poses and real-time poses of the four corner points;

[0021] The steel structure corridor is adjusted in real time until the real-time offset angle is less than the preset offset angle threshold and the real-time posture differences of the four corner points are all less than the preset posture difference threshold.

[0022] Furthermore, the method of preprocessing the first image to obtain the second image is:

[0023] Grayscale transformation and median filtering denoising processing.

[0024] Furthermore, the method of identifying the real-time rectangular structure form of the steel structure corridor based on the second image and obtaining the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form is:

[0025] The Canny edge detection algorithm is used to identify the edge of the steel structure corridor, and the structural morphological features of the edge of the steel structure corridor are extracted by morphological operation, wherein the morphological operation includes expansion and corrosion;

[0026] SIFT and SURF algorithms are used to extract key feature points of structural morphological features, where the key feature points are the four corner points of the steel structure corridor, and a real-time rectangular structural morphology is established based on the extracted four corner points of the steel structure corridor;

[0027] The preset four sides of the steel structure corridor are obtained based on the preset rectangular structure form, and the actual four sides of the steel structure corridor are obtained based on the real-time rectangular structure form. According to the angles between the preset four sides and the actual four sides corresponding to each other, the values ​​of the four angles are averaged to obtain the real-time rectangular structure form and the preset rectangular structure form to obtain the real-time offset angle of the steel structure corridor.

[0028] Furthermore, it also includes establishing a deep learning model, and the specific method is as follows:

[0029] Collect a certain amount of image data of the steel structure corridor installation process to establish an image set;

[0030] Manually annotate the image set and mark out the steel structure corridor;

[0031] The VGG-16 convolutional neural network is used for model training;

[0032] The trained convolutional neural network is used to recognize the real-time rectangular structure of the steel corridor.

[0033] The real-time offset angle of the steel structure corridor is obtained based on the real-time rectangular structure shape and the preset rectangular structure shape.

[0034] Furthermore, the direction and amplitude that need to be adjusted are determined based on the real-time offset angle and real-time posture error, and accurate correction instructions are generated by combining the proportional, integral and differential parts of the PID controller to make real-time adjustments to the steel structure corridor.

[0035] Furthermore, a virtual model of the steel structure corridor is established in 3ds Max, and a virtual body is added to each corner point of the steel structure corridor in the virtual model. The displacement changes of the four corner points of the steel structure corridor are tracked one by one by the four virtual bodies to obtain the real-time posture of the four corner points of the steel structure corridor.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] The present invention is to achieve high-precision positioning and correction of steel structure corridors in long-distance and highly complex environments. The system realizes the automatic positioning and correction of steel structures by combining shooting modules, image processing modules, sensor modules and adjustment modules. System integration and optimization improve the system operation efficiency and positioning accuracy through multiple tests and calibrations, and enhance the robustness and anti-interference ability of the system. Through the above technical means, the present invention realizes high-precision positioning and correction of steel structures, improves work efficiency and accuracy, reduces errors in manual operations, adapts to complex construction environments and reduces costs. This technology has broad application prospects and market demand in the fields of construction engineering, machinery manufacturing and intelligent robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0039] Figure 1 It is a schematic diagram of a steel structure positioning system based on computer vision and AI of the present invention;

[0040] Figure 2 It is a structural schematic diagram of the present invention applied to on-site adjustment of a steel structure corridor;

[0041] Figure 3 for Figure 2 A magnified schematic diagram of point A;

[0042] Figure 4 It is a schematic diagram of the rapid alignment system of the present invention;

[0043] Figure 5 The figure is a flow chart of the steel structure positioning method based on computer vision and AI of the present invention.

[0044] Among them, 1. shooting module; 2. sensor module; 3. adjustment module; 31. magnetic suction cup; 32. bracket; 33. handle; 331. first U-shaped arm; 3311. first end; 3312. second end; 332. second U-shaped wall; 3321. third end; 3322. fourth end; 333. first straight rod; 334. second straight rod; 34. pushing member; 341. pushing shaft; 342. pushing seat; 35. hydraulic station; 4. steel structure corridor; 5. steel structure support frame; DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0046] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0047] It should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make the disclosure thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0048] Example 1

[0049] Embodiment 1 provides a steel structure positioning system based on computer vision and AI, such as Figure 1 and Figure 2 As shown, including:

[0050] A preset module is used to set the preset rectangular structure of the steel structure corridor 4 and the accurate positions and postures of the four corner points corresponding to the preset rectangular structure;

[0051] A shooting module 1 is used to shoot a first image of the steel structure corridor 4 in real time;

[0052] An image processing module, used for preprocessing the first image to obtain a second image, identifying the real-time rectangular structure form of the steel structure corridor 4 based on the second image, and obtaining the real-time offset angle of the steel structure corridor 4 based on the real-time rectangular structure form and the preset rectangular structure form;

[0053] Sensor module 2 is used to obtain the real-time posture of 44 corner points of the steel structure corridor, and obtain the real-time posture difference of the four corner points based on the accurate posture and real-time posture of the four corner points;

[0054] The adjustment module 3 is used to adjust the steel structure corridor 4 in real time until the real-time offset angle is less than a preset offset angle threshold and the real-time posture differences of the four corner points are all less than a preset posture difference threshold.

[0055] The steel structure positioning system based on computer vision and AI provided in Example 1 is used to achieve high-precision positioning and correction of the steel structure corridor 4 in a long-distance and high-complexity environment. The system realizes the automatic positioning and correction of the steel structure by combining the shooting module 1, the image processing module, the sensor module 2 and the adjustment module 3. System integration and optimization improve the system operation efficiency and positioning accuracy through multiple tests and calibrations, and enhance the robustness and anti-interference ability of the system. Through the above technical means, the present invention realizes high-precision positioning and correction of steel structures, improves work efficiency and accuracy, reduces errors in manual operations, adapts to complex construction environments and reduces costs. This technology has broad application prospects and market demands in the fields of construction engineering, machinery manufacturing and intelligent robots.

[0056] In the shooting module 1 of this embodiment, a number of high-definition cameras are installed on the steel structure corridor 4 and / or the steel structure support frame 5. These cameras should have the characteristics of high resolution, high frame rate and low latency, so as to capture real-time images. At the same time, they are equipped with high-precision servo motors or hydraulic systems to perform correction and adjustment operations on the steel structure.

[0057] In the sensor module 2 of this embodiment, the posture includes attitude information and position information. The sensor module 2 includes an inertial measurement unit (IMU) and a global positioning system (GPS sensor) to achieve precise angle and position measurement. The inertial measurement unit is used to measure the attitude information of the 44 corner points of the steel structure corridor when moving. The attitude information includes angular velocity and acceleration information. The global positioning system is used to provide the position information of the 44 corner points of the steel structure corridor.

[0058] Specifically, the inertial measurement unit (IMU) contains a three-axis gyroscope, a three-axis accelerometer, and a magnetometer to measure angular velocity and acceleration. The global positioning system (GPS sensor) provides high-precision position data. After the IMU and GPS data are fused, more accurate position and attitude information is obtained. Before actual application, the IMU and GPS need to be calibrated to ensure accurate measurement data.

[0059] In the adjustment module 3 of this embodiment, if Figure 2-Figure 4 As shown, the adjustment module 3 includes a magnetic suction cup 31, a bracket 32, a handle 33 and a pusher 34. The magnetic suction cup 31 is used to adsorb the steel structure corridor 4. The bracket 32 ​​is rotatably connected to the magnetic suction cup 31. The pusher 34 includes a pushing seat 342 and a pushing shaft 341. One side of the bracket 32 ​​is fixed to the handle 33, and the other side is fixed to the pushing seat 342. The handle 33 is used to hook the steel structure corridor 4, and the pushing shaft 341 is fixed to the steel structure support frame 5.

[0060] Specifically, the magnetic suction cup 31 adopts an electrically controlled permanent magnetic suction cup, which realizes the generation or disappearance of the magnetic field through the electric control principle, so as to flexibly absorb or release the load. The electrically controlled permanent magnetic suction cup is operated by a control unit, and the control unit is integrated on the handle 33 for easy operation.

[0061] The pusher 34 is a hydraulic cylinder, which is powered by a hydraulic station 35. The hydraulic cylinder provides the necessary support force and position adjustment to ensure the smooth and precise alignment of the steel structure corridor 4. The adjustment module 3 is installed at the closest position between the steel structure corridor 4 and the steel structure support frame 5. The steel structure corridor 4 is fixed by an electric-controlled permanent magnetic suction cup, and the hydraulic cylinder is precisely adjusted to achieve rapid alignment. When disassembling, the state of the electric-controlled permanent magnetic suction cup is switched by the control unit, and the lever effect is used for rapid disassembly.

[0062] like Figure 3 As shown, the handle 33 includes a first U-shaped arm 331 and a second U-shaped arm 332. The first U-shaped arm 331 is fixed to the bracket 32. The first U-shaped arm 331 includes a first end 3311 and a second end 3312. The second U-shaped arm 332 includes a third end 3321 and a fourth end 3322. The first end 3311 and the third end 3321 are connected through a first straight rod 333. The second end 3312 and the fourth end 3322 are connected through a second straight rod 334. The angle between the first straight rod 333 and the first end 3311 is 110-150 degrees. The third end 3321 and the first straight rod 333 are connected. The angle between them is 110-150 degrees, the angle between the second straight rod 334 and the second end 3312 is 110-150 degrees, and the angle between the fourth end 3322 and the second straight rod 334 is 110-150 degrees. By setting the first U-shaped arm 331, the second U-shaped arm 332, the first straight rod 333 and the second straight rod 334, the steel structure corridor 4 can be hooked, and then the real-time offset angle and the real-time posture difference of the steel structure corridor 4 can be adjusted to ensure that the real-time offset angle is less than the preset offset angle threshold and the real-time posture differences of the four corner points are all less than the preset posture difference threshold.

[0063] Specifically, one end of the first straight rod 333 is fixed to the first end 3311, and the other end is fixed to the third end 3321. One end of the second straight rod 333 is fixed to the second end 3312, and the other end is fixed to the fourth end 3322. The angle of the bracket 32 ​​is adjusted by telescopically pushing the shaft 341, and then the angle of the first U-shaped arm 331 is adjusted, thereby adjusting the angle of the second U-shaped arm 332 and the steel structure corridor 4, and adjusting the real-time offset angle and real-time posture difference of the steel structure corridor 4 to ensure that the real-time offset angle is less than the preset offset angle threshold and the real-time posture difference of the four corner points are all less than the preset posture difference threshold.

[0064] Example 2

[0065] Embodiment 2 provides a steel structure positioning method based on computer vision and AI, which is applied to the above-mentioned steel structure positioning system based on computer vision and AI, such as Figure 5 As shown, the following steps are included:

[0066] Step S1: setting the preset rectangular structure of the steel structure corridor 4 and the accurate positions of the four corner points corresponding to the preset rectangular structure;

[0067] Step S2: capturing a first image of the steel structure corridor 4 in real time;

[0068] Step S3: preprocessing the first image to obtain a second image, identifying the real-time rectangular structure form of the steel structure corridor 4 based on the second image, and obtaining the real-time offset angle of the steel structure corridor 4 based on the real-time rectangular structure form and the preset rectangular structure form;

[0069] Step S4: obtaining the real-time poses of 44 corner points of the steel structure corridor, and obtaining the real-time pose differences of the four corner points based on the accurate poses and real-time poses of the four corner points;

[0070] Step S5: Make real-time adjustments to the steel structure corridor 4 until the real-time offset angle is smaller than a preset offset angle threshold and the real-time posture differences of the four corner points are smaller than a preset posture difference threshold.

[0071] In step S1 of this embodiment, a virtual model of the steel structure corridor 4 is established in 3ds Max, and then the preset rectangular structure of the steel structure corridor 4 and the accurate positions and postures of the four corner points corresponding to the preset rectangular structure are set.

[0072] Specifically, virtual bodies are added to each corner point of the virtual model, and these virtual bodies will be used to track the displacement changes of the corner points. The virtual body is bound to the corner point using the binding tool of 3ds Max to ensure that the virtual body moves with the corner point. The displacement data of the virtual body during the entire hoisting process is recorded using the animation and trajectory recording functions of 3ds Max. The real-time position and posture of the 44 corner points of the steel structure corridor 4 are obtained by tracking the displacement changes of the 4 corner points of the steel structure corridor 4 one by one through 4 virtual bodies.

[0073] In terms of installation and monitoring in the actual environment, IMU (Inertial Measurement Unit) and GPS (Global Positioning System) sensors are installed on the corridor to achieve accurate angle and coordinate measurement, ensuring that the coordinate system of the recording module is consistent with the coordinate system in 3ds Max. During the hoisting process, the displacement data of the corner points is monitored in real time and compared with the preset data in 3ds Max, and the hoisting position is adjusted to ensure that the final position is aligned with the designed coordinates.

[0074] In step S2 of this embodiment, grayscale transformation, median filter denoising and Canny edge detection are performed to improve image quality and feature extraction accuracy. The Canny edge detection algorithm can accurately capture edge details in an image and enhance the accuracy of adjustment and subsequent analysis.

[0075] Specifically, the method of preprocessing the first image to obtain the second image is: grayscale transformation and median filtering denoising processing.

[0076] In step S2 of this embodiment, the real-time rectangular structure form of the steel structure corridor 4 is identified based on the second image, and the method for obtaining the real-time offset angle of the steel structure corridor 4 based on the real-time rectangular structure form and the preset rectangular structure form is:

[0077] The Canny edge detection algorithm is used to identify the edge of the steel structure corridor 4, and the structural morphological features of the edge of the steel structure corridor 4 are extracted by morphological operations, and the morphological operations include expansion and corrosion;

[0078] SIFT and SURF algorithms are used to extract key feature points of structural morphological features, where the key feature points are the four corner points of the steel structure corridor 4. A real-time rectangular structural morphology is established based on the extracted four corner points of the steel structure corridor 4.

[0079] The Canny edge detection algorithm is a classic image processing technique that implements edge detection through five main steps: first, Gaussian smoothing is performed on the image to reduce the impact of noise; second, the gradient strength and direction of each pixel in the image are calculated; then, the edge is refined through non-maximum suppression, retaining only the local maximum value; then, high and low dual threshold detection is applied to divide the gradient strength into strong edges, weak edges, and non-edges; finally, strong edges and weak edges are connected to form the final edge detection result. The Canny algorithm is known for its high detection rate, high positioning accuracy, and low response redundancy. It can effectively strike a balance between noise suppression and edge positioning, and is suitable for a variety of application scenarios such as object recognition, image segmentation, and contour extraction.

[0080] The Canny edge detection algorithm is used to identify the edge of the steel structure, and morphological operations (such as expansion and corrosion) are applied to further extract the structural morphological features. At the same time, SIFT and SURF algorithms are used to extract key feature points, which will help in subsequent matching and positioning work.

[0081] SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) are two commonly used image feature extraction algorithms, which are widely used in image recognition and matching tasks. SIFT achieves scale and rotation invariance by detecting and describing key points in the image, and has high robustness and matching accuracy, but the computational complexity is high. SURF improves the computational speed while retaining high matching accuracy, making it suitable for real-time applications. The main difference between the two lies in the computational efficiency and feature descriptor dimension: SIFT descriptor is 128-dimensional, SURF is 64-dimensional, SURF is more efficient but slightly less robust than SIFT in complex scenes.

[0082] The four preset sides of the steel structure corridor 4 are obtained based on the preset rectangular structure form, and the actual four sides of the steel structure corridor 4 are obtained based on the real-time rectangular structure form. According to the angles between the preset four sides and the actual four sides corresponding to each other, the values ​​of the four angles are averaged to obtain the real-time rectangular structure form and the preset rectangular structure form to obtain the real-time offset angle of the steel structure corridor 4.

[0083] In step S2 of this embodiment, a deep learning model is also established. The specific method is as follows:

[0084] Collect a certain amount of image data of the installation process of the steel structure corridor 4 to establish an image set;

[0085] Manually annotate the image set and mark out the steel structure corridor 4;

[0086] The VGG-16 convolutional neural network is used for model training;

[0087] The trained convolutional neural network is used to recognize the real-time rectangular structure of the steel structure corridor 4;

[0088] The real-time offset angle of the steel structure corridor 4 is obtained based on the real-time rectangular structure form and the preset rectangular structure form.

[0089] In this embodiment, a large amount of image data of the installation process of the steel structure corridor 4 is collected and manually annotated to form a training data set. The VGG-16 convolutional neural network architecture is selected for model training. Through data enhancement techniques (such as rotation, flipping and scaling), more variant samples are generated to enhance the generalization ability of the model.

[0090] VGG-16 is a deep convolutional neural network model proposed by the Visual Geometry Group (VGG) of the University of Oxford. It is widely used for its outstanding performance in image recognition tasks. The model consists of 13 convolutional layers and 3 fully connected layers, with a total of 16 layers deep. The convolutional layer uses a 3×3 convolution kernel with a stride of 1 and a padding of 1 to ensure that the spatial dimensions of the input and output remain unchanged. Each convolutional layer is usually followed by a ReLU activation function. The pooling layer after the convolutional layer uses a 2×2 maximum pooling with a stride of 2 to reduce the spatial dimension, the number of parameters and the computational complexity. The fully connected layer integrates the features extracted by the convolutional layer and outputs the classification result. Although the structure is simple, VGG-16 performs well on large image datasets due to its depth and about 138 million parameters. This model is often used for tasks such as image classification, object detection, and image segmentation, and requires a lot of computing resources when training. Its advantages are high performance and simple structure, but also because the model is large, the computing resource requirements are high. In the steel structure positioning and real-time adjustment system, VGG-16 can process the image data collected by the camera, generate corresponding correction instructions by analyzing the real-time position and angle, and improve the system automation and accuracy.

[0091] In step S4 of this embodiment, the accurate poses and real-time poses of the four corner points are respectively subtracted to obtain the real-time pose difference values ​​of the four corner points.

[0092] In step S5 of this embodiment, during the installation of the steel structure corridor 4, the camera image data is input and processed in real time. The real-time offset angle and real-time posture error of the steel structure corridor 4 in the current section are predicted by combining the trained VGG-16 model with the real-time posture of the 44 corner points of the steel structure corridor 4. According to the real-time offset angle and the real-time posture error, the direction and amplitude to be adjusted are determined, and the steel structure corridor 4 is adjusted in real time. Combined with the proportional, integral and differential parts of the PID controller, accurate correction instructions are generated to control the hydraulic cylinder to perform correction operations and adjust the posture and angle of the steel structure corridor 4. After each correction operation, the image data is reacquired, and the analysis and correction steps are repeated to form a closed-loop control mechanism.

[0093] (PID controller consists of three parts: proportional (P), integral (I) and differential (D). The proportional part quickly adjusts the system output according to the size of the error, but may cause the system to have a certain steady-state error. The integral part eliminates the steady-state error by accumulating the error to ensure that the system eventually reaches the target value, but it may slow down the system response. The differential part makes advance corrections by predicting the trend of error changes, reducing oscillation and overshoot, but is sensitive to noise. By reasonably adjusting the weights of these three parts, precise control of the system can be achieved.

[0094] In this embodiment, multi-segment linkage correction: multiple cameras work together to capture the relative position and angle information of each section of the steel structure corridor 4. A deep learning model is used to identify the real-time rectangular structure form, and the real-time offset angle of the steel structure corridor 4 is obtained based on the real-time rectangular structure form and the preset rectangular structure form, and the overall adjustment is made through the adjustment system to ensure that all segments are on the same straight line.

[0095] In this embodiment, an integrated system is also constructed, which includes a preset module, an image processing module, a shooting module 1, a sensor module 2 and an adjustment module 3. Through multiple tests and calibrations, the collaborative work of each module is optimized to improve the system operation efficiency and positioning accuracy. The robustness and anti-interference ability of the system are enhanced to adapt to complex construction environments. In addition, the data is exported and analyzed later. By comparing the displacement data of the virtual model with the actual recorded displacement data, the errors and deviations in the hoisting process are analyzed, and a report containing displacement data, error analysis and adjustment suggestions is generated to ensure that the installation process meets the design requirements.

[0096] In summary, the present invention has the following advantages:

[0097] (1) The present invention and method use computer vision technology, deep learning algorithm, PID controller, high-precision sensors (IMU and GPS) and virtual reality technology (3ds Max) to solve the problem of accurate alignment and adjustment during the installation of steel structures, ensuring the accurate installation of steel structures and the consistency of the overall structure. It realizes automated, high-precision positioning and correction of steel structures, significantly improves installation efficiency and quality, and reduces errors and risks of manual operations.

[0098] (2) Use edge detection algorithms to extract key features of steel structures. SIFT and SURF algorithms are used to further extract key feature points, ensuring high accuracy of image feature extraction. These technical means can accurately identify and locate the edges and morphological features of steel structures, providing high-quality data for subsequent deep learning model training.

[0099] (3) The present invention collects a large amount of image data of the steel structure installation process and manually annotates it to form a training data set. The VGG-16 convolutional neural network architecture is selected, and more variant samples are generated through data enhancement techniques (such as rotation, flipping and scaling) to enhance the generalization ability of the model. The trained model can accurately predict the position, angle and deviation of the steel structure in real-time image analysis.

[0100] (4) Combine the proportional, integral and differential parts of the PID controller to generate accurate correction instructions and control the high-precision servo motor or hydraulic system to perform the correction operation. Through the feedback of real-time image data, the analysis and correction steps are repeated to form a closed-loop control mechanism to ensure the accurate positioning and adjustment of the steel structure.

[0101] (5) The present invention installs multiple high-definition cameras on the steel structure corridor 4 to capture the relative position and angle information of each section of the steel structure. Through the collaborative work of multiple cameras and deep learning model calculation, the deviation of each section of the steel structure can be adjusted as a whole to ensure that all sections are on the same straight line, thereby achieving high-precision laying of continuous steel structures.

[0102] (6) A virtual body is established on the 3ds Max corridor model to track the displacement changes of the corner points, record the displacement data of the virtual body during the entire hoisting process, and synchronize the IMU (inertial measurement unit) and GPS (global positioning system) sensor information during the actual installation process to ensure that the coordinate system of the recording module is consistent with the coordinate system in 3ds Max. During the hoisting process, the displacement data of the corner points is monitored in real time and compared with the preset data in 3ds Max to adjust the hoisting position to ensure that the final position is aligned with the design coordinates.

[0103] (7) An integrated system including a preset module, an image processing module, a shooting module 1, a sensor module 2, and an adjustment module 3 was constructed. Through multiple tests and calibrations, the collaborative work of each module was optimized to improve the system's operating efficiency and positioning accuracy. The robustness and anti-interference ability of the system were enhanced to ensure its stable operation in a complex construction environment.

[0104] Through the above innovations, the present invention can provide a high-precision, high-efficiency steel structure positioning and correction system, effectively improve the accuracy and stability of the steel structure corridor 4 installation, reduce the error of manual operation, improve the construction efficiency and quality, adapt to complex construction environments and reduce costs. This technology has broad application prospects and huge market demand in the fields of construction engineering, machinery manufacturing, intelligent robots, etc.

[0105] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. Steel structure positioning system based on computer vision and AI, It is characterized in that include: A preset module is used to set the preset rectangular structure of the steel structure corridor and the accurate positions of the four corner points corresponding to the preset rectangular structure; A shooting module, used for shooting a first image of the steel structure corridor in real time; An image processing module is used to preprocess the first image to obtain a second image, identify the real-time rectangular structure form of the steel structure corridor based on the second image, and obtain the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form; The sensor module is used to obtain the real-time posture of the four corner points of the steel structure corridor, and obtain the real-time posture difference of the four corner points based on the accurate posture and real-time posture of the four corner points; The adjustment module is used to make real-time adjustments to the steel structure corridor until the real-time offset angle is less than a preset offset angle threshold and the real-time posture differences of the four corner points are all less than a preset posture difference threshold.

2. According to the steel structure positioning system based on computer vision and AI according to claim 1, Features: The posture includes attitude information and position information. The sensor module includes an inertial measurement unit and a global positioning system. The inertial measurement unit is used to measure the attitude information of the four corner points of the steel structure corridor when moving. The attitude information includes angular velocity and acceleration information. The global positioning system is used to provide the position information of the four corner points of the steel structure corridor.

3. According to the steel structure positioning system based on computer vision and AI according to claim 1, Features: The adjustment module includes a magnetic suction cup, a bracket, a handle and a pushing member. The magnetic suction cup is used to adsorb the steel structure corridor. The bracket is rotatably connected to the magnetic suction cup. The pushing member includes a pushing seat and a pushing shaft. One side of the bracket is fixed to the handle, and the other side is fixed to the pushing seat. The handle is used to hook the steel structure corridor, and the pushing shaft is fixed to the steel structure support frame.

4. According to the steel structure positioning system based on computer vision and AI as claimed in claim 3, Features: The handle includes a first U-shaped arm and a second U-shaped arm, the first U-shaped arm is fixed to the bracket, the first U-shaped arm includes a first end and a second end, the second U-shaped arm includes a third end and a fourth end, the first end and the third end are connected by a first straight rod, the second end and the fourth end are connected by a second straight rod, the angle between the first straight rod and the first end is 110-150 degrees, the angle between the third end and the first straight rod is 110-150 degrees, the angle between the second straight rod and the second end is 110-150 degrees, and the angle between the fourth end and the second straight rod is 110-150 degrees.

5. A steel structure positioning method based on computer vision and AI, applied to a steel structure positioning system based on computer vision and AI as described in any one of claims 1 to 4, It is characterized in that The following steps are involved: Set the preset rectangular structure of the steel structure corridor and the accurate positions of the four corner points corresponding to the preset rectangular structure; Real-time capture of the first image of the steel structure corridor; Preprocessing the first image to obtain a second image, identifying the real-time rectangular structure form of the steel structure corridor based on the second image, and obtaining the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form; Obtain the real-time poses of the four corner points of the steel structure corridor, and obtain the real-time pose differences of the four corner points based on the accurate poses and real-time poses of the four corner points; The steel structure corridor is adjusted in real time until the real-time offset angle is less than the preset offset angle threshold and the real-time posture differences of the four corner points are all less than the preset posture difference threshold.

6. The steel structure positioning method based on computer vision and AI according to claim 5, It is characterized in that The method of preprocessing the first image to obtain the second image is: Grayscale transformation and median filtering denoising processing.

7. The steel structure positioning method based on computer vision and AI according to claim 5, It is characterized in that The method for identifying the real-time rectangular structure form of the steel structure corridor based on the second image and obtaining the real-time offset angle of the steel structure corridor based on the real-time rectangular structure form and the preset rectangular structure form is as follows: The Canny edge detection algorithm is used to identify the edge of the steel structure corridor, and the structural morphological features of the edge of the steel structure corridor are extracted by morphological operation, wherein the morphological operation includes expansion and corrosion; SIFT and SURF algorithms are used to extract key feature points of structural morphological features, where the key feature points are the four corner points of the steel structure corridor, and a real-time rectangular structural morphology is established based on the extracted four corner points of the steel structure corridor; The preset four sides of the steel structure corridor are obtained based on the preset rectangular structure form, and the actual four sides of the steel structure corridor are obtained based on the real-time rectangular structure form. According to the angles between the preset four sides and the actual four sides corresponding to each other, the values ​​of the four angles are averaged to obtain the real-time rectangular structure form and the preset rectangular structure form to obtain the real-time offset angle of the steel structure corridor.

8. The steel structure positioning method based on computer vision and AI according to claim 7, Features: It also includes building a deep learning model by: Collect a certain amount of image data of the steel structure corridor installation process to establish an image set; Manually annotate the image set and mark out the steel structure corridor; The VGG-16 convolutional neural network is used for model training; The trained convolutional neural network is used to recognize the real-time rectangular structure of the steel corridor. The real-time offset angle of the steel structure corridor is obtained based on the real-time rectangular structure shape and the preset rectangular structure shape.

9. The steel structure positioning method based on computer vision and AI according to claim 7, Features: According to the real-time offset angle and real-time posture error, the direction and amplitude that need to be adjusted are determined. Combined with the proportional, integral and differential parts of the PID controller, accurate correction instructions are generated to make real-time adjustments to the steel structure corridor.

10. The steel structure positioning method based on computer vision and AI according to claim 5, Features: A virtual model of the steel structure corridor is established in 3ds Max, and a virtual body is added to each corner point of the steel structure corridor in the virtual model. The displacement changes of the four corner points of the steel structure corridor are tracked one by one by four virtual bodies to obtain the real-time posture of the four corner points of the steel structure corridor.

Images