A pile position automatic correction system based on Beidou and AI vision

Abstract

The application relates to the technical field of automatic control of engineering machinery, and discloses a pile position automatic correction system based on Beidou and AI vision, which comprises a Beidou inertial navigation combined positioning unit, a positive optical reference field generating unit, an AI vision sensing unit and a predictive control unit. The system solves the pose deviation by combining Beidou global guidance with the precision measurement of the projected optical pattern, overcomes the problems of insufficient accuracy and reliability of traditional positioning methods, adopts an online-updatable neural network hybrid dynamics model, can autonomously compensate for the model drift introduced by working condition changes according to the motion error, ensures the long-term accuracy and adaptability of the model predictive control algorithm, in addition, the system reconstructs the local three-dimensional environment through stereo vision to identify obstacles and converts the obstacles into real-time safety constraints, realizes active intelligent obstacle avoidance, and significantly improves the safety of automatic operation.

Description

Technical Field

[0001] This invention relates to the field of automated control technology for construction machinery, specifically to an automatic pile position correction system based on BeiDou and AI vision. Background Technology

[0002] Piling machinery is a key piece of equipment in foundation engineering construction, and the accuracy of its pile positioning directly affects the quality and safety of the building. With technological advancements, automation and intelligence of piling machinery have become industry trends. In existing automatic pile positioning technologies, a common method is to use a Global Navigation Satellite System (GNSS, such as BeiDou). However, while GNSS positioning provides global coordinates, its accuracy is typically only at the meter or sub-meter level and is easily affected by environmental factors such as tall buildings blocking the view and multipath effects, making it difficult to meet the centimeter-level accuracy requirements for the final pile position. Another method utilizes machine vision technology to guide the piling machine alignment by recognizing pre-set physical markers on the ground (such as crosshairs, paint dots, or embedded rebar). The drawback of this method is that the complex construction site environment, often covered with mud, water, and debris, makes the physical markers easily contaminated, obscured, or damaged, leading to vision system recognition failures or decreased accuracy, severely impacting the reliability and applicability of automated correction. Current technologies fail to effectively combine the convenience of global guidance with the accuracy of local measurements to provide a robust, high-precision positioning solution even in complex site environments.

[0003] Furthermore, at the implementation level of automated control algorithms for piling machines, existing control systems mostly rely on a fixed, idealized dynamic model of the piling machine. These models, when established, typically fail to comprehensively encompass all the complex nonlinear factors encountered in actual operations. For example, when a piling machine travels on soil or gravel surfaces with varying densities, the slip ratio of its tracks and the coefficient of friction with the ground change significantly; the performance of the hydraulic system also drifts with fluctuations in oil temperature; and wear and tear on mechanical components caused by prolonged operation alters the system's dynamic response characteristics. The fixed dynamic model cannot adapt to these dynamic changes, leading to increasingly larger deviations between model predictions and actual motion. This reduces the accuracy and robustness of the control system, and may even cause oscillations or instability, making it difficult to maintain efficient and stable operational performance under varying working conditions over the long term.

[0004] Furthermore, operational safety is a crucial prerequisite for automated equipment. Currently, the safety measures for automated piling machinery are not yet perfect, mainly relying on passive safety mechanisms such as remote monitoring by operators and emergency stop buttons. Although some equipment is equipped with simple proximity sensors (such as ultrasonic or infrared sensors), these sensors can usually only trigger emergency braking, interrupting the entire operation process. They lack intelligent path planning and adjustment capabilities. When the piling machine encounters temporary construction personnel, scattered materials, or other equipment on the automatically corrected movement path, the existing technology lacks a mechanism that can integrate dynamically changing local environmental information into the control decision-making closed loop in real time. It cannot achieve proactive and smooth obstacle avoidance, thus making it difficult to guarantee absolute safety in complex scenarios of human-machine mixed operations, limiting its fully automated application in real construction environments. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an automatic pile position correction system based on BeiDou and AI vision, which solves the problems of insufficient positioning accuracy and reliability, poor adaptability of control models to dynamic working conditions, and low safety due to lack of active obstacle avoidance capabilities in traditional pile position alignment technology under complex construction environments.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The first aspect of this invention provides an automatic pile position correction system based on BeiDou and AI vision, comprising: The BeiDou inertial navigation system (INS) integrated positioning unit is used to acquire the real-time global pose of the pile driver in the global coordinate system. This real-time global pose describes the position and orientation of the pile driver's body coordinate system within the global coordinate system. The BeiDou INS integrated positioning unit provides the global position and attitude information of the pile driver.

[0007] An active optical reference field generation unit is used to calculate and project an optical reference graphic onto the ground area where the target pile is located, based on the preset target pile coordinates and the real-time global pose. The active optical reference field generation unit dynamically adjusts the position of the projected graphic according to the real-time global pose of the pile driver to form a positioning reference on the ground.

[0008] The AI ​​visual perception unit is used to simultaneously acquire images of the optical reference pattern and the pile body on the piling machine, and to determine the pose deviation between the pile body and the optical reference pattern through image processing. The AI ​​visual perception unit uses image analysis to accurately and quantitatively measure the relative position and pose deviation of the pile body relative to the ground optical reference pattern.

[0009] The predictive control unit, equipped with a pile driver dynamics model, generates corrective control commands for driving the pile driver's movement and attitude adjustment based on the posture deviation and the pile driver dynamics model, using a model predictive control algorithm. The predictive control unit receives posture deviation data from the AI ​​visual perception unit and, in conjunction with the pile driver dynamics model, generates motion commands for the pile driver's actuators through optimized calculations.

[0010] Preferably, the optical reference pattern projected by the active optical reference field generation unit is a coded optical pattern, which embeds coded information for verifying or transmitting instructions. The coded information is used to ensure the integrity of the optical pattern or to transmit additional control instructions.

[0011] Preferably, the predictive control unit is further configured to: predict the motion trajectory of the pile driver in one or more future control cycles based on the pile driver dynamics model; predict future changes in the observation perspective of the AI ​​visual perception unit based on the motion trajectory; and, based on the future changes in the observation perspective, send a feedforward command to the active optical reference field generation unit to project the pre-distortion corrected optical reference image. The pre-distortion correction can compensate for optical image distortion caused by changes in the observation perspective, thereby improving the accuracy of visual perception.

[0012] Preferably, the pre-set pile driver dynamics model in the predictive control unit includes a mechanistic model portion describing the basic physical motion laws of the pile driver, and a data-driven model portion serving as a neural network model. The neural network model is used to learn and compensate for the error between the predicted values ​​of the mechanistic model portion and the actual motion state of the pile driver. By learning from actual data, the neural network model can improve the prediction accuracy and adaptability of the pile driver dynamics model.

[0013] Preferably, the predictive control unit uses the model predictive control algorithm to generate the correction control command. The specific steps include: in each control cycle, based on the current pose deviation determined by the AI ​​visual perception unit, which characterizes the degree of deviation between the pile and the optical reference image, solving a finite-time domain optimization problem to obtain the optimal control sequence that minimizes the pose deviation of the pile relative to the optical reference image in the prediction time domain, and executing only the first control command in the optimal control sequence to control the pile driver.

[0014] Preferably, the system is further configured to perform local environment perception before executing correction control commands. This local environment perception includes: the active optical reference field generation unit projecting structured light onto the local environment surrounding the target pile location; the AI ​​visual perception unit acquiring the structured light image and reconstructing a three-dimensional model of the local environment for obstacle identification; and the predictive control unit converting the obstacle information into safety constraints in the model predictive control algorithm. This local environment perception enhances the operational safety of the system in complex construction sites.

[0015] In one specific embodiment, the AI ​​visual perception unit includes a site observation camera and a pile recognition camera. The installation poses of the site observation camera and the pile recognition camera on the pile driver are known relative to the pile driver's coordinate system. During the local environment perception, the site observation camera and the pile recognition camera constitute a stereo vision system, which works in conjunction with structured light to reconstruct the 3D model. The stereo vision system can provide depth information to assist in the reconstruction of the 3D model.

[0016] Furthermore, the process by which the AI ​​visual perception unit determines the pose deviation specifically includes: processing the image of the optical reference graphic acquired by the site observation camera to locate the coordinates of its optical center in the site observation camera coordinate system, i.e., the optical center coordinates; processing the image of the pile acquired by the pile recognition camera to segment the pile outline and calculate the coordinates of its physical center in the pile recognition camera coordinate system, i.e., the physical center coordinates; transforming both the optical center coordinates and the physical center coordinates to the pile driver body coordinate system, and calculating the pose deviation based on the relative positional relationship between the two in the pile driver body coordinate system. This process achieves the calculation of the pose deviation between the pile and the reference graphic through multi-sensor data fusion and coordinate transformation.

[0017] Preferably, the system further includes a state estimation module, which employs an extended Kalman filter to fuse the real-time global pose output by the BeiDou inertial navigation integrated positioning unit and the pose deviation determined by the AI ​​visual perception unit, providing a more accurate state estimate of the pile driver for the predictive control unit. The state estimation module, through multi-source information fusion, provides more accurate pile driver state information than a single sensor, improving the robustness of the control system.

[0018] Preferably, the system further includes an online model update module, configured to: continuously collect the actual motion state of the pile driver measured by the BeiDou inertial navigation combined positioning unit and the AI ​​visual perception unit during the pile position correction process; and update the network weights and bias parameters of the neural network model online based on the error between the actual motion state of the pile driver and the predicted value of the pile driver dynamics model. The online model update module enables the system to continuously learn and adapt in actual operation, improving the long-term performance stability of the system.

[0019] The second aspect of this invention provides an automatic pile position correction system based on BeiDou and AI vision, which has the functions of local environment perception and safety constraint generation.

[0020] Before executing correction control commands, the system performs local environmental perception. This local environmental perception includes: the active optical reference field generation unit projecting structured light into the local environment surrounding the target pile location; the AI ​​visual perception unit acquiring the structured light image and reconstructing a 3D model of the local environment for obstacle identification; and the predictive control unit converting the obstacle information into safety constraints in the model predictive control algorithm. This function ensures the safe operation of the pile driver during the correction process by identifying potential obstacles before operation and incorporating them into the constraints of the control algorithm.

[0021] In one specific embodiment, the AI ​​visual perception unit includes a site observation camera and a pile recognition camera. The installation poses of the site observation camera and the pile recognition camera on the pile driver are known relative to the coordinate system of the pile driver itself. During the local environment perception, the site observation camera and the pile recognition camera constitute a stereo vision system, which works in conjunction with the structured light to reconstruct the three-dimensional model.

[0022] The third aspect of this invention provides an automatic pile position correction system based on BeiDou and AI vision, which has a prediction-based visual distortion correction function.

[0023] The predictive control unit is configured to: predict the movement trajectory of the pile driver in one or more future control cycles based on the pile driver dynamics model; predict future changes in the observation perspective of the AI ​​visual perception unit based on the movement trajectory; and, based on the future changes in the observation perspective, send a feedforward command to the active optical reference field generation unit to project the optical reference image that has undergone pre-distortion correction. This function, by predicting future observation conditions and performing pre-correction, can improve the recognition accuracy and positioning accuracy of the optical reference image by the AI ​​visual perception unit.

[0024] Preferably, the pre-set dynamic model of the pile driver in the predictive control unit includes a mechanism model part for describing the basic physical motion law of the pile driver, and a data-driven model part as a neural network model. The neural network model is used to learn and compensate for the error between the predicted value of the mechanism model part and the actual motion state of the pile driver.

[0025] The fourth aspect of this invention provides an automatic pile position correction system based on BeiDou and AI vision, which has an online adaptive update function for the pile driver dynamic model.

[0026] The system also includes an online model update module, configured to: continuously collect the actual motion state of the pile driver measured by the BeiDou inertial navigation system and the AI ​​visual perception unit during the pile position correction process; and update the network weights and bias parameters of the neural network model online based on the error between the actual motion state of the pile driver and the predicted value of the pile driver dynamics model. This function enables the pile driver dynamics model to maintain high accuracy under different working conditions and equipment wear, thereby improving the long-term performance and robustness of the model-based predictive control algorithm.

[0027] This invention provides an automatic pile position correction system based on BeiDou navigation and AI vision. It has the following beneficial effects: 1. This invention obtains the initial pose of the pile driver in the global coordinate system through the Beidou inertial navigation combined positioning unit, and completes the coarse positioning guidance. Then, the active optical reference field generation unit projects a clear and stable optical reference image onto the ground based on the target pile position. This optical reference image provides the AI ​​visual perception unit with a high-contrast, accurate reference benchmark that is not affected by the site ground conditions, enabling the AI ​​visual perception unit to accurately calculate the pose deviation between the pile body and the target point. This combination of global pose guidance and local visual precision measurement overcomes the problems of insufficient positioning accuracy of a single signal source and unclear reference objects at the construction site, and can achieve automatic correction with centimeter-level or even higher precision.

[0028] 2. The pile driver dynamics model adopted in this invention combines a mechanistic model describing physical laws with a neural network model as a data-driven model. The system integrates an online model update module. During the calibration process, this module can continuously compare the actual movement state of the pile driver measured by Beidou and AI vision with the predicted value of the model, and use the error between them to update the parameters of the neural network model online. This mechanism enables the system to learn autonomously and compensate for model errors introduced by factors such as changes in geological conditions, fluctuations in hydraulic oil temperature, and mechanical wear of equipment. This ensures that the predictive control unit always makes decisions based on a high-precision model, thereby improving the system's adaptability and long-term stability of control performance in different operating environments.

[0029] 3. This invention introduces a local environment perception step before the predictive control unit executes the correction control command. Structured light is projected by an active optical reference field generation unit, and images are acquired by an AI visual perception unit to reconstruct a local three-dimensional environment model around the target pile location. This model is used to identify potential obstacles such as construction workers, materials, or equipment. The identified obstacle information is converted into safety constraints in the model predictive control algorithm by the predictive control unit. This design enables the system to actively avoid obstacles when planning the pile driver's movement path, embedding safety considerations into every optimization solution of the control algorithm, thereby effectively preventing collision risks and significantly improving the safety of automated operations. Attached Figure Description

[0030] Figure 1 This is a block diagram of the overall system structure of the present invention; Figure 2 This is a schematic diagram showing the relationship between the physical components of the piling machine and the coordinate system of the present invention; Figure 3 Flowchart for determining pose deviation of the AI ​​visual perception unit of the present invention; Figure 4 This is a flowchart illustrating the execution of the Model Predictive Control (MPC) algorithm of this invention. Figure 5 This is a schematic diagram of the overall system workflow of the present invention. Detailed Implementation

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see the appendix Figure 1 This is a system overall structure block diagram according to an embodiment of the present invention. The embodiment of the present invention provides an automatic pile position correction system based on Beidou and AI vision, including: a Beidou inertial navigation combined positioning unit 10, an active optical reference field generation unit 20, an AI vision perception unit 30, and a prediction control unit 40. The BeiDou inertial navigation combined positioning unit 10, the active optical reference field generation unit 20, and the AI ​​visual perception unit 30 all establish data communication connections with the prediction control unit 40. In one embodiment, this communication connection can be achieved through a controller local area network (CAN) bus or Ethernet. The prediction control unit 40, as the control core of the system, receives measurement data from the BeiDou inertial navigation combined positioning unit 10 and the AI ​​visual perception unit 30, and sends control commands to the active optical reference field generation unit 20 and the pile driver actuator.

[0033] In the aforementioned communication network, different data are transmitted with different priorities and frequencies. For example, the global pose from the BeiDou inertial navigation integrated positioning unit 10. and pose deviation from AI visual perception unit 30 As high-frequency key data, it is broadcast at a frequency of 10Hz to 100Hz; while the pre-distortion correction command sent by the foresight control unit 40 to the active optical reference field generation unit 20 and the correction control command sent to the pile driver actuator are also sent at the same control frequency, while other low-frequency data such as system status monitoring and parameter configuration are transmitted with lower priority.

[0034] In one embodiment, the system may further include a state estimation module 50 and a model online update module 60. The state estimation module 50 and the model online update module 60 may operate as functional modules within the processor of the predictive control unit 40. The state estimation module 50 receives data from the BeiDou inertial navigation integrated positioning unit 10 and the AI ​​visual perception unit 30, and its output is provided to the predictive control unit 40. The model online update module 60 receives measurement data from the BeiDou inertial navigation integrated positioning unit 10 and the AI ​​visual perception unit 30 and uses it to update the pile driver dynamics model configured internally within the predictive control unit 40.

[0035] Please see the appendix Figure 2 To accurately describe the pose relationships involved in this invention, the following coordinate system is defined: Global coordinate system , recorded as This coordinate system is a fixed inertial reference system, such as a local horizontal coordinate system established with the target pile location as the origin and east, north, and vertically upward as axes. All global positions and attitudes are described in this coordinate system.

[0036] Pile driver body coordinate system , recorded as This coordinate system is fixed to a specific reference point on the piling machine body, such as its center of rotation. All movements and measurements of the piling machine can ultimately be reduced to this coordinate system for a unified description.

[0037] Site observation camera coordinate system , recorded as The coordinate system is fixed to the optical center of the site observation camera in the AI ​​visual perception unit 30, which is used to observe the ground optical reference shape.

[0038] Pile body recognition camera coordinate system , recorded as The coordinate system is fixed to the optical center of the pile recognition camera in the AI ​​visual perception unit 30, which is used to identify the outline of the pile.

[0039] In one embodiment of the present invention, the transformation relationship between coordinate systems is described by a homogeneous transformation matrix, where a transformation matrix is ​​used to transform the coordinate system from the target coordinate system {... } to source coordinate system { The homogeneous transformation matrix of} Represented as: ; in, Represents a homogeneous transformation matrix. This represents a 3×3 rotation matrix; Represents rotation; superscript and subscript Its meaning and The definitions are the same; This represents a 3×1 translation vector; Represents position; superscript and subscript Its meaning and The definitions are the same as those in the text. It is the last row of the matrix, specifically in the form of ; It is a 1×3 zero vector; It is a scalar.

[0040] The function of the Beidou inertial navigation combined positioning unit 10 is to determine the coordinate system of the pile driver body in real time. Relative to global coordinate system The pose, i.e., the homogeneous transformation matrix This matrix contains the three-dimensional position and three-dimensional attitude information of the piling machine in the global coordinate system.

[0041] Site observation camera coordinate system coordinate system with pile driver body The relative poses between them, and the coordinate system of the pile recognition camera coordinate system with pile driver body The relative poses between them are a fixed relationship determined by the physical installation position of the cameras, and these two homogeneous transformation matrices... and It is known and obtained in advance through offline calibration.

[0042] Based on the above definition, a coordinate system for a site observation camera... Points measured below It can be transformed to the global coordinate system. In, its coordinates The calculation process is as follows: ; Similarly, a coordinate system for recognizing cameras on piles... Points measured below In the global coordinate system coordinates in The calculation process is as follows:

[0043] ; in, Representing a point in three-dimensional space, it is usually represented by a coordinate vector; This represents a homogeneous transformation matrix; When used as a superscript or subscript, it refers to the global coordinate system. ; Refers to the coordinate system of the pile driver body ; Refers to the coordinate system of the site observation camera ; The coordinate system of the pile body recognition camera ; For a point in the global coordinate system The coordinates below; A point is directly measured by a site observation camera and placed in its own coordinate system. The coordinates expressed below; A point is directly measured by the pile recognition camera and placed in its own coordinate system. The coordinates expressed below; This indicates that the coordinates of a point are transferred from the pile driver's body coordinate system. Transform to global coordinate system The transformation matrix represents the real-time pose (position and orientation) of the piling machine in the global space. This indicates that the coordinates of a point are transferred from the coordinate system of the site observation camera. Transform to the pile driver body coordinate system The transformation matrix is ​​fixed because the camera is rigidly mounted on the pile driver, and this transformation relationship can be determined through a one-time calibration. This indicates that the coordinates of a point are transferred from the pile body to the camera coordinate system. Transform to the pile driver body coordinate system The transformation matrix is ​​also a fixed transformation relationship.

[0044] Through these coordinate transformation relationships, the system can unify the local measurement information of different sensors in their respective coordinate systems into a global coordinate system. or pile driver body coordinate system The fusion processing, deviation calculation, and control decision-making constitute the mathematical basis for the accurate correction achieved in this invention.

[0045] In one embodiment, the BeiDou inertial navigation integrated positioning unit 10 consists of a module that integrates a multi-frequency, multi-constellation GNSS receiver and a microelectromechanical system (MEMS) inertial measurement unit (IMU). The GNSS receiver can simultaneously receive signals from multiple satellite navigation systems, including BeiDou, GPS, and GLONASS, to provide absolute position information. The IMU includes a three-axis accelerometer and a three-axis gyroscope for measuring the three-dimensional angular velocity and linear acceleration of the piling machine body. The unit internally operates an extended Kalman filter to tightly couple and fuse the low-frequency but error-free position information provided by GNSS with the high-frequency but integral-drift attitude and acceleration information provided by the IMU. Through this fusion, the unit can continuously output the motion-compensated and smoothed coordinate system of the piling machine body at a frequency of 100Hz or higher. In the global coordinate system Real-time global pose The data is then sent to the predictive control unit 40 via the CAN bus.

[0046] In this embodiment, the active optical reference field generation unit 20 employs a digital light processing (DLP) projector. This projector possesses industrial-grade protection capabilities and sufficient light throughput to ensure that the projected optical reference graphics have clear contrast on the ground under different ambient lighting conditions. The prediction control unit 40 uses the preset target pile coordinates and the real-time global pose measured by the BeiDou inertial navigation combined positioning unit 10. The system calculates the exact plane position to which the optical reference graphic should be projected. To achieve the encoding function, the projected optical reference graphic is not a simple crosshair or dot, but a composite graphic. For example, the center is a high-brightness cross or circular reference mark, surrounded by one or more rings of binary coded dot matrix or marks resembling ArUco codes. These codes embed the unique identification number (ID) or timestamp information of the target pile position, which is verified by the AI ​​visual perception unit 30 during image processing to prevent misidentification of projections from other light sources or adjacent pile drivers.

[0047] To achieve prediction-based visual distortion correction, the predictive control unit 40 will predict the pile driver pose at the future time t+k. The image is sent to the active optical reference field generation unit 20, which generates a pre-distorted image based on the predicted pose. Specifically, a set of ideal reference points to be formed on the ground is assumed to be in the global coordinate system. The homogeneous coordinate set below is The projector is in the coordinate system of the pile driver body. The installation pose is known. Therefore, the predicted pose of the projector at a future time is: Ideal reference point in the projector prediction coordinate system coordinates below The following transformation is used to obtain: ; in, It is a 4×1 homogeneous coordinate vector, representing a target reference point in the global coordinate system. The position in the middle; It is a 4×4 homogeneous transformation matrix, representing the projector's position relative to the global coordinate system at future prediction times. The position; yes The inverse matrix represents the coordinate system from the global coordinate system. To the projector prediction coordinate system Transformation; It is a 4×1 homogeneous coordinate vector, representing the corresponding position of the target reference point in the projector's predicted coordinate system.

[0048] Subsequently, the active optical reference field generation unit 20, based on its internally calibrated projector intrinsic parameter model (e.g., pinhole model and distortion parameters), calculates... A reverse projection calculation is performed to generate a two-dimensional image. When this two-dimensional image is projected, since the projection process is the inverse of the above calculation process, the final image formed on the ground will be an ideal reference image without distortion, even if the pile driver body is tilted or located on uneven ground.

[0049] Please see the appendix Figure 3 The AI ​​visual perception unit 30 includes hardware components and image processing algorithms running on it. The hardware components include a site observation camera and a pile recognition camera, both of which are industrial cameras using a global shutter to avoid motion blur when acquiring moving images. Their installation poses are... and It is accurately measured and stored through hand-eye calibration methods.

[0050] In a specific configuration, the site observation camera uses a wide-angle lens with a large field of view (e.g., 90 degrees or more) to ensure that the optical reference graphics on the ground can be captured completely under different postures of the pile driver. The pile recognition camera uses a lens with a long focal length and a small field of view (e.g., 30 degrees) to achieve magnified observation of the cross-section of the top of the pile, thereby improving the pixel-level accuracy of pile contour segmentation and physical center positioning. The line laser, which is coaxially mounted with the pile recognition camera, has a known preset angle between its laser emission plane and the vertical axis of the camera to form the optimal laser triangulation geometry.

[0051] The process of determining the pose deviation is as follows: The site observation camera acquires ground images containing optical reference graphics. The algorithm first binarizes the images, then filters out noise through morphological operations, and then uses methods such as contour discovery and geometric moment calculation or Hough circle transformation to accurately locate the center reference mark of the optical reference graphics in the coordinate system of the site observation camera. Pixel coordinates on the image plane Combined with the intrinsic parameter matrix of the site observation camera and by and The calculated height of the camera above the ground can be used to reconstruct the three-dimensional coordinate system of the optical center within the field observation camera. coordinates below .

[0052] Simultaneously, the pile recognition camera captures images of the top of the pile. The algorithm uses a pre-trained U-Net or other semantic segmentation neural network model to perform pixel-level segmentation of the pile cross-section in the image, obtaining a precise pile contour. By calculating the centroid of the region enclosed by this contour, the physical center of the pile in the pile recognition camera coordinate system is obtained. Pixel coordinates on the image plane To obtain its three-dimensional coordinates, a line laser coaxial with the camera projects a laser line onto the top of the pile. By analyzing the offset of the laser stripe in the image (i.e., laser triangulation), the depth value of the pile surface relative to the camera can be calculated, thereby reconstructing the three-dimensional coordinate system of the pile's physical center within the pile recognition camera. coordinates below .

[0053] Finally, the two 3D coordinates measured in their respective camera coordinate systems are transformed into the pile driver body coordinate system. Down: ; ; in, As a subscript, it is an abbreviation for optics, specifically referring to an optical reference center; As a subscript, it is an abbreviation for pile body, specifically referring to the physical center of the pile body; It is the optical reference center point In the coordinate system of the pile driver The coordinate vector below; It is the optical reference center point Coordinate system of site observation camera The coordinate vector below; It is the physical center point of the pile. In the coordinate system of the pile recognition camera The coordinate vector below; It is the physical center point of the pile. In the coordinate system of the pile driver The coordinate vector below; other symbols have the same meaning as above, in the pile driver body coordinate system. Below, pose deviation This is the vector difference between the physical center of the pile and the optical reference center: ; The obtained pose deviation vector Includes and Translational deviation in direction and possible The height deviation in the direction is sent to the predictive control unit 40 as the core input of the model predictive control algorithm. The directional deviation can be obtained by comparing the main direction of the pile profile with the directional information encoded in the optical reference figure.

[0054] The predictive control unit 40 employs an industrial PC or embedded computing platform equipped with a central processing unit (CPU) and a graphics processing unit (GPU). As a centralized data processing and decision-making center, it receives the global pose of the BeiDou inertial navigation integrated positioning unit 10 via a bus interface. Pose deviation of AI visual perception unit 30 This unit runs the piling machine dynamics model and model predictive control algorithm, which will be detailed in later chapters. The calculation results of the algorithm, i.e. the optimal control sequence, are converted into specific control commands, such as the control voltage signal of the electro-hydraulic proportional valve that drives the piling machine tracks, or the pulse / direction signal of the motor driver that drives the upper body to rotate. These commands are sent to the programmable logic controller (PLC) at the bottom of the piling machine through a digital-to-analog converter card or communication bus. The PLC directly drives the corresponding actuator to complete the closed-loop correction of the piling machine's position and posture.

[0055] To clarify the core algorithm and working principle executed by the predictive control unit 40 in this invention, the following will describe in detail the construction of the pile driver dynamics model, the model predictive control algorithm, the local environment perception, and the online update mechanism.

[0056] In one embodiment of the present invention, the pile driver dynamics model configured in the predictive control unit 40 is a hybrid model combining mechanism and data. This model is used to accurately predict the future motion state of the pile driver under a given control input. The state-space expression of this hybrid model can be discretized as follows: ; in, Indicates at discrete time The state vector of the pile driver; Indicates at discrete time The control input vector applied to the piling machine; Indicates the next moment Predicted pile driver state vector; The mechanism model function describes the basic motion law of the pile driver based on the principles of vehicle kinematics. This represents a data-driven neural network model function used to learn and compensate for mechanistic models. The model errors are caused by complex nonlinear dynamics that cannot be described, such as ground slippage, actuator delay, and frictional changes. Representing a neural network model The set of all trainable parameters, including the weights and biases of each layer of the network, in one embodiment, A multilayer perceptron (MLP) structure with several hidden layers can be used, with the current state as its input. and control input The output is the compensation value for each component of the state vector.

[0057] Please see the appendix Figure 4 The predictive control unit 40, within each control cycle, solves an optimal control problem in the finite time domain based on the aforementioned hybrid dynamics model to generate correction control commands. This optimal control problem is formalized as follows: ; Subject to the following constraints: 1. 2. ; 3. ; in, Indicates from the current moment Beginning, Future A control sequence to be optimized with a control step size; The objective function to be minimized is, in physical terms, to make the piling machine's trajectory as close as possible to the target pose while consuming as little control energy as possible within the prediction time domain. Indicates the length of the prediction time domain; Indicates from the current moment Starting from the current state, predict the state of the future time k+i; The target state vector is represented in this invention. Its two-dimensional position is determined by the target pile coordinates, and the target heading angle can be preset or determined by the AI ​​visual perception unit 30. , , These are positive semidefinite weight matrices for state error, control input, and terminal state error, used to adjust the relative importance of different performance indicators in the optimization problem; , Represents the control input vector The upper and lower limits represent the capability limitations of the physical actuator; This represents the state constraints of the system; in particular, this term is used to introduce safety constraints obtained from local environment perception. To be from arrive The sum of all terms represents the cumulative cost over the entire prediction time domain.

[0058] Before system deployment, the neural network, as part of the data-driven model, It requires offline pre-training. The offline training dataset is collected by operating the piling machine under various representative working conditions (such as ground with different hardness and different slopes) and simultaneously recording the actual movement trajectory of the piling machine and the corresponding control input commands measured by high-precision external measuring equipment (such as a laser tracker). Through this offline data, the neural network... initial parameters The model is trained to a level that can roughly compensate for errors in the mechanistic model. Subsequent online updates are fine-tuning and real-time adaptations based on this pre-trained model.

[0059] The predictive control unit 40 solves the above optimization problem using a numerical optimization algorithm (such as Sequence Quadratic Programming, SQP) to obtain the optimal control sequence. Then, only the first element of the sequence This is sent as the actual control command to the piling machine actuator in the next control cycle. The system measures the new state of the piling machine and repeats the entire optimization process described above. This rolling optimization method enables the control to continuously adapt to changes in system state and external disturbances.

[0060] Before performing corrective control, the system of the present invention can perform local environmental perception to generate the aforementioned state constraints. The process is as follows: The active optical reference field generation unit 20 projects a known structured light pattern (such as a multi-line laser or Gray code pattern) around the target pile location. The site observation camera and pile recognition camera in the AI ​​visual perception unit 30 then form a stereo vision system, simultaneously acquiring structured light images modulated by the obstacles. By matching identical structured light feature points in the left and right images and performing triangulation using the calibrated intrinsic and extrinsic parameters of the binocular cameras, a 3D point cloud of the local environment can be reconstructed. After processing the point cloud data using a clustering algorithm (such as DBSCAN), individual obstacles are identified. Each obstacle is approximately enclosed as a circle or convex polygon on a two-dimensional plane. For example, for an obstacle in a global coordinate system... The center was identified as the center. , radius is A circular obstacle, and its corresponding state constraint function. It can be defined as: ; in, This is a safety distance set up to ensure safety redundancy; and Represents two-dimensional plane coordinates, subscript Represents obstacles; It is the sum of two radii, representing the total radius of the safe zone; This is the square of the distance between the center of the pile driver and the center of the obstacle. Incorporating this constraint into the MPC optimization problem ensures that the planned path always maintains a safe distance from the obstacle. To further improve control accuracy, the state estimation module 50 uses an extended Kalman filter (EKF) to fuse multi-source sensor information. The EKF prediction step uses the aforementioned hybrid dynamics model for state prediction. When the global pose measurement value is received from the BeiDou inertial navigation integrated positioning unit 10... and pose deviation measurements from AI visual perception unit 30 At this point, EKF enters the update step, which calculates the residuals between the predicted and actual values ​​corresponding to these two measurements, and then updates the predicted state vector based on the residuals and the Kalman gain. and its covariance matrix The system makes corrections and outputs a more accurate and smoother state estimate that incorporates the advantages of each sensor, which serves as the input to the MPC algorithm.

[0061] The online model update module 60 is responsible for enabling the adaptive operation of the pile driver dynamics model. During the pile position correction process, this module continuously records the high-precision real state sequence output by the state estimation module 50, and calculates the real state change rate. With mechanism model prediction part The difference, this difference is the neural network The target compensation value to be learned under the current operating conditions : ; in, Represent an error vector. Vector symbols, representing errors, will be used in module 60. As input, The target output forms a training sample, which is then used to refine the neural network parameters through stochastic gradient descent (SGD) or its variants. Perform small-step online updates: ; in, , The parameter set of the neural network at time... and The value, For learning rate, Represents a gradient vector. For the entire update term, according to the principle of gradient descent, the parameters... Update along the direction of the steepest descent of the loss function (i.e., the negative gradient direction), with the step size determined by the learning rate. control; Using the loss function, this online update process enables the dynamic model to continuously adapt to changes in the environment and the device's own state, ensuring the long-term performance of the control algorithm.

[0062] See attached document Figure 5 The following will use a complete pile location correction task as an example to illustrate the workflow of the system of the present invention in detail: Step S100: System Initialization and Target Input After the system is powered on, each unit module completes a self-test. The operator then uses the human-machine interface to input a series of three-dimensional coordinates of the target pile locations. The construction design documents are loaded into the system, and the predictive control unit 40 reads the coordinates of the first target pile location. As the target point of the current task.

[0063] Step S200: Coarse positioning and establishment of optical reference field The pile driver moves to the target pile location based on manual operation by the operator or on the global pose provided by the Beidou inertial navigation integrated positioning unit 10. The approximate area is determined during this process, and the BeiDou inertial navigation integrated positioning unit 10 continuously measures and outputs the location of the pile driver in the global coordinate system to the prediction control unit 40. Real-time pose The predictive control unit 40 is based on and target pile coordinates Real-time calculation of distance from the projector position to the target point The projection direction required for the projection on the ground is determined, and then the foresight control unit 40 instructs the active optical reference field generation unit 20 to project an coded optical reference image onto the calculated ground position.

[0064] Step S300: Local Environment Perception and Safety Constraint Generation Before initiating precise closed-loop correction, the system performs a local environment perception. The active optical reference field generation unit 20 projects a preset structured light pattern onto the ground area surrounding the target pile location. The site observation camera and pile recognition camera in the AI ​​visual perception unit 30, acting as a stereo vision system, simultaneously acquire structured light images modulated by environmental objects. The processor inside the AI ​​visual perception unit 30 calculates the disparity map using a stereo matching algorithm and reconstructs a 3D point cloud of the local environment by combining it with the calibrated camera parameters. After filtering and clustering, this point cloud data identifies the location and size of potential obstacles. The predictive control unit 40 converts the information of each obstacle into one or more forms such as... The nonlinear inequality constraints are applied and loaded into the solver of the Model Predictive Control (MPC) optimization problem.

[0065] Step S400: The closed-loop correction control system enters a high-frequency closed-loop correction control cycle, which operates on a set control period. (For example, 10 milliseconds) This process is repeated continuously until the task completion condition is met. A single iteration of the loop includes: Step S410 (Visual Perception): The AI ​​visual perception unit 30 simultaneously acquires images of the optical reference shape and the pile body, locates the optical center and the physical center of the pile body respectively through image processing algorithms, and calculates the coordinates of the pile driver body at the current moment according to the coordinate transformation process. pose deviation vector .

[0066] Step S420 (State Estimation): The state estimation module 50 receives global pose measurement values ​​from the BeiDou inertial navigation integrated positioning unit 10 and pose deviation measurement values ​​from the AI ​​visual perception unit 30. The extended Kalman filter (EKF) running inside the module 50 fuses the data from these two sources and outputs an optimized and more accurate state estimate of the pile driver. .

[0067] Step S430 (Predictive and Control Solution): The predictive control unit 40 uses state estimates... As the initial state of the MPC algorithm, it utilizes a hybrid dynamics model in the prediction time domain. The system predicts the future trajectory of the piling machine and, based on this predicted trajectory, proactively generates a pre-distortion correction command, which is sent to the active optical reference field generation unit 20 to compensate for visual distortion that may be caused by changes in the piling machine's pose at the next moment. Subsequently, the predictive control unit 40 constructs and solves a finite-time domain optimization problem that includes state tracking error, control energy consumption, and safety constraints generated in step S300, to obtain the optimal control sequence. .

[0068] Step S440 (Instruction Execution): The predictive control unit 40 selects the optimal control sequence... Extract the first control command It then converts the signal into a physical control signal for the corresponding pile driver actuator (such as a hydraulic valve or motor driver), driving the pile driver to make a slight positional adjustment.

[0069] Step S500: Online Update of the Dynamic Model While the closed-loop correction loop in step S400 is being executed, the online model update module 60 works in parallel. This module caches a short period of time the high-precision state sequence output by the state estimation module 50 and the actual control command sequence executed by the system. It uses this data to calculate the prediction error of the mechanistic model and uses this error as a training label to fine-tune the parameters of the neural network part in the hybrid dynamic model online through the backpropagation algorithm. This process ensures that the dynamic model can continuously track and adapt to the slow changes in the characteristics of the piling machine itself and the external environment.

[0070] Step S600: Task Completion Judgment At the end of each closed-loop correction cycle, the predictive control unit 40 checks the pose deviation measured by the AI ​​visual perception unit 30. At the end of each closed-loop correction cycle, the predictive control unit 40 checks the pose deviation measured by the AI ​​visual perception unit 30. To avoid misjudgments caused by sensor noise, the system calculates the moving average of the pose deviation modulus over the most recent N (e.g., N=50) control cycles. When this moving average remains below a preset accuracy threshold (e.g., 2 cm) for a preset period of time (e.g., 1 consecutive second), the system determines that the current pile position correction task is complete. Furthermore, the system sets a maximum correction time (e.g., 120 seconds). If the convergence condition is not met within this time, the system will issue a timeout alarm, prompting the operator to intervene and check. The anticipatory control unit 40 will stop sending correction commands, instruct the active optical reference field generation unit 20 to turn off projection, and prompt the operator to proceed with the next pile driving operation or switch to the next target pile position. And repeat steps S200 to S600.

Claims

1. An automatic pile position correction system based on BeiDou and AI vision, characterized in that, include: The Beidou inertial navigation combined positioning unit is used to obtain the real-time global pose of the pile driver in the global coordinate system. The real-time global pose is used to describe the position and orientation of the pile driver body coordinate system in the global coordinate system. An active optical reference field generation unit is used to calculate and project an optical reference pattern onto the ground area where the target pile is located, based on the preset target pile coordinates and the real-time global pose. The AI ​​visual perception unit is used to simultaneously acquire images of the optical reference pattern and the pile body on the pile driver, and to determine the pose deviation between the pile body and the optical reference pattern through image processing. The predictive control unit, configured with a pile driver dynamics model, is used to generate correction control commands for driving pile driver movement and attitude adjustment based on the posture deviation and the pile driver dynamics model, employing a model predictive control algorithm. The predictive control unit is further configured to: Based on the pile driver dynamics model, the motion trajectory of the pile driver in one or more future control cycles is predicted. Based on the motion trajectory, predict the future change in the observation perspective of the AI ​​visual perception unit; Based on the future change in observation angle, a feedforward command is sent to the active optical reference field generation unit to project the optical reference image that has undergone pre-distortion correction; The pre-set dynamic model of the pile driver in the predictive control unit includes: The mechanism model part describes the basic physical motion law of the pile driver, and the data-driven model part is used as a neural network model. The neural network model is used to learn and compensate for the error between the predicted value of the mechanism model part and the actual motion state of the pile driver. The pile driver dynamics model also includes an online model update module, which is configured as follows: During the pile position correction process, the actual movement status of the pile driver is continuously collected by the Beidou inertial navigation combined positioning unit and the AI ​​visual perception unit. Based on the error between the actual motion state of the pile driver and the predicted value of the pile driver dynamics model, the network weights and bias parameters of the neural network model are updated online.

2. The automatic pile position correction system based on BeiDou and AI vision according to claim 1, characterized in that, The optical reference pattern projected by the active optical reference field generation unit is a coded optical pattern, which contains coded information for verifying or transmitting instructions.

3. The automatic pile position correction system based on BeiDou and AI vision according to claim 1, characterized in that, The predictive control unit uses the model predictive control algorithm to generate the correction control command, specifically including: In each control cycle, based on the current pose deviation determined by the AI ​​visual perception unit, which characterizes the degree of deviation between the pile and the optical reference image, a finite-time domain optimization problem is solved to obtain the optimal control sequence that minimizes the pose deviation of the pile relative to the optical reference image in the prediction time domain. The piling machine is controlled by executing only the first control command in the optimal control sequence.

4. The automatic pile position correction system based on BeiDou and AI vision according to claim 1, characterized in that, Before executing the correction control command, the predictive control unit also needs to perform local environment perception, which includes: The active optical reference field generation unit projects structured light into the local environment surrounding the target pile location; The AI ​​visual perception unit acquires structured light images and reconstructs a three-dimensional model of the local environment for the purpose of identifying obstacles; The predictive control unit converts the obstacle information into safety constraints in the model predictive control algorithm.

5. The automatic pile position correction system based on BeiDou and AI vision according to claim 4, characterized in that, The AI ​​visual perception unit includes a site observation camera and a pile recognition camera. The installation poses of the site observation camera and the pile recognition camera on the pile driver are known relative to the coordinate system of the pile driver body. When performing the local environment perception, the site observation camera and the pile recognition camera constitute a stereo vision system and work together with the structured light to complete the reconstruction of the three-dimensional model.

6. The automatic pile position correction system based on BeiDou and AI vision according to claim 5, characterized in that, The process by which the AI ​​visual perception unit determines the pose deviation specifically includes: The image of the optical reference pattern acquired by the site observation camera is processed to locate the coordinates of its optical center in the site observation camera coordinate system, i.e., the optical center coordinates. The image of the pile captured by the pile recognition camera is processed to segment the pile outline and calculate the coordinates of its physical center in the pile recognition camera coordinate system, i.e., the physical center coordinates. The optical center coordinates and the physical center coordinates are both transformed to the pile driver body coordinate system, and the pose deviation is calculated based on their relative positional relationship in the pile driver body coordinate system.

7. The automatic pile position correction system based on BeiDou and AI vision according to claim 1, characterized in that, It also includes a state estimation module, which uses an extended Kalman filter to fuse the real-time global pose output by the Beidou inertial navigation combined positioning unit and the pose deviation determined by the AI ​​visual perception unit, so as to provide a more accurate state estimation of the pile driver for the predictive control unit.

Images