Support splicing equipment for subway foundation pit construction based on binocular vision human eye bionics

Abstract

The invention discloses supporting and splicing equipment for metro foundation pit construction based on binocular vision human eye bionics, and relates to the field of metro foundation pit construction, the supporting and splicing equipment comprises a walking mechanism, a workbench, a sensor system, a first mechanical arm, a second mechanical arm, a sensor system and a control system; the control system processes data of the sensor system and controls the bionic eye to horizontally, vertically and accurately rotate, and the grabbing, conveying and butt joint states of the mechanical arm are detected and adjusted.

Description

Technical Field

[0001] This invention relates to the field of subway support machinery technology, and in particular to a support splicing device for subway foundation pit construction based on binocular vision bionics. Background Technology

[0002] With the rapid development of urban rail transit construction, subway foundation pit engineering is gradually moving towards deeper and larger spans. Especially in urban areas where the cut-and-cover method is used, temporary excavation and support of the foundation pit are required under the road surface system to reduce the impact on surface traffic and the surrounding environment. Foundation pit support, as a core component of foundation pit construction, provides safety assurance for the construction of subway underground stations and lines, and its construction quality directly affects the safety and progress of the project. Traditional foundation pit support operations rely on hoisting equipment and manual labor to transport and splice steel pipes. This model has certain limitations: when there are obstructions above the support or when the support location is far from the edge of the foundation pit, the hoisting equipment cannot complete the support work as intended; in confined spaces, machinery and construction personnel work together, leading to frequent construction accidents; the positioning of the supporting steel pipes relies on manual alignment, resulting in long installation times for a single pipe and large positioning errors.

[0003] Some rail transit construction plans have stations located in densely populated areas of ancient city centers, surrounded by numerous buildings and experiencing high traffic volume. Constructing station foundation pits presents numerous challenges, including limited space, stringent traffic control requirements, sensitive surrounding environment, and strict environmental regulations. Therefore, the cut-and-cover method is proposed for construction. To prevent the surrounding soil from collapsing, a retaining structure is required. In one rail transit construction plan, an 800mm thick diaphragm wall is used as the retaining structure, with four supports along the depth of the foundation pit. The first support is reinforced concrete; the second, third, and fourth supports are steel supports. The horizontal spacing of the reinforced concrete supports is 9m. The steel supports are 800mm thick, with the third and fourth supports being... The steel supports for 609 are arranged horizontally at 2m / 4m intervals. Ordinary construction methods are insufficient to meet the requirements. To improve automation, a solution combining a robotic arm and vision processing is urgently needed to reduce labor intensity and construction risks. An invention patent, CN111645077A, entitled "Ground Monitoring System and Method for Live-Line Operation Robot in Power Distribution Networks," discloses a robot for power operations using a robotic arm and camera. The camera is mounted on the robot body to monitor the movement of the robotic arm. However, this method often results in the robotic arm self-obstructing key parts, failing to meet real-time perception requirements. Summary of the Invention

[0004] In view of this, the present invention provides a support splicing device for subway foundation pit construction based on binocular vision and human eye bionics.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A support splicing device for subway foundation pit construction based on binocular vision and human eye bionics includes: a walking mechanism for moving the entire device; a workbench mounted on the walking mechanism for mounting a sensor system and a first robotic arm and a second robotic arm; the first and second robotic arms for collaboratively gripping, transporting, and docking support steel pipes to improve the stability of support construction; a sensor system including a lidar, a GPS sensor, a piezoresistive thin-film sensor, and two bionic eye cameras; and a control system including an industrial computer, an STM32 control board, and a driver for processing data from the sensor system and controlling the precise horizontal and vertical rotation of the bionic eyes, as well as detecting and adjusting the gripping, transporting, and docking status of the robotic arms.

[0007] Preferably, the bionic eye camera is a dual-motor cable-driven bionic eye, comprising: a camera, an eyeball structure, a rotating shaft, and a fixed connecting rod; the eyeball is mounted and fixed on the vertical axis of the rotating shaft; the horizontal axis of the rotating shaft is mounted on the fixed connecting rod, which is fixed to the worktable by bolts; the camera is mounted on the eyeball structure; the eyeball structure is driven to rotate by a flexible cable; the flexible cable is fixed to the pulley of the drive motor; the rotation of the motor causes the length of the flexible cable to change, thereby simulating the elongation and contraction of the extraocular muscles, thus realizing the rotation of the eyeball.

[0008] Preferably, a flexible cable fixing groove is reserved on the vertical axis of the rotating shaft; a central hole is reserved in the horizontal axis of the rotating shaft for the flexible cable to pass through along its vertical axis; bearings are installed on both the vertical and horizontal axes to reduce friction; the eyeball is composed of an anterior eyeball and a posterior eyeball; a flexible cable mounting hole is reserved in the anterior eyeball; a nut is pre-embedded in the anterior eyeball and connected to the posterior eyeball by threads; both eyeballs have bearing mounting grooves so that the two eyeballs can be installed and fixed on the vertical axis; one end of the flexible cable controlling vertical rotation is fixed in the flexible cable fixing groove on the vertical axis, and the other end away from the flexible cable fixing groove is fixed on a corresponding reel on the vertical axis; one end of the flexible cable controlling horizontal rotation is fixed in the reserved flexible cable mounting hole in the anterior eyeball, and the other end away from the reserved flexible cable mounting hole passes through the central hole reserved in the horizontal axis and is arranged and fixed on the corresponding reel on the horizontal axis along the groove of the fixing link; the corresponding reel on the vertical axis and the corresponding reel on the horizontal axis are driven by independent motors.

[0009] Preferably, the rotation control module of the bionic eye camera includes: a motion control module for converting angle commands into motor pulse signals; an error compensation module for pulse compensation of horizontal steering errors; and a limit control module for limiting the rotation angle to prevent structural damage. Preferably, the piezoresistive thin-film sensor is installed at the mechanical claw joints of the first and second robotic arms to detect the grasping force and provide feedback on the grasping status.

[0010] Preferably, the lidar sensor, GPS sensor, and bionic eye camera are connected to the airborne router; the piezoresistive thin-film sensor and the bionic eye camera are both connected to the STM32 control board; the airborne industrial computer is connected to the airborne router to obtain data from the sensors in the local area network; the STM32 control board is connected to the airborne industrial computer via RS232, transmits the signal from the piezoresistive thin-film sensor, and obtains the rotation signal from the bionic eye camera to control the bionic eye camera to rotate.

[0011] The present invention also provides a method of using the above-mentioned device, comprising the following steps:

[0012] S1. Start the support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics, and perform equipment status self-check, equipment initialization and sensor calibration.

[0013] S2. The operator selects and loads the configuration file of the overall support scheme stored locally in the console, and chooses whether to import the support scheme completion status log file. If there is no configuration file, the equipment will check the completion status and generate a log file.

[0014] S3. Obtain the current position of the device through the GPS sensor, obtain the yaw angle of the fuselage with the assistance of the GPS sensor, query the configuration file and log file, and based on the query results, the control system automatically plans the path and navigates to the nearest steel support interface position that is "not installed"; and makes the orientation of the device body perpendicular to the axis of the support steel pipe interface, with the support steel pipe interface located in the center of the front of the device body, and the distance between the device body and the support steel pipe interface 1m to 2m; S4. Use a bionic eye camera to collect images of the target steel support interface, obtain the interface status, determine whether the interface where the steel support is not installed in the log has a steel support installed, if it has been installed, change the status of the interface in the log file, and return to S3 to execute;

[0015] S5. Collect image data of the storage area through a bionic eye camera, identify the available steel pipes in the storage area, and generate a robotic arm grasping scheme.

[0016] S6. Use the bionic eye camera again to collect images of the target steel support interface, obtain the interface status and position, and combine the robotic arm grasping scheme to plan the complete action sequence of the robotic arm: grasp the steel pipe - transport the steel pipe - accurately connect and install the steel pipe to the target interface;

[0017] S7. Based on the complete action sequence work plan generated by S6, control the robotic arm to perform the gripping, transporting, precise docking and installation operations of a single steel support;

[0018] S8. After completion, check the splicing status and modify the log file.

[0019] Preferably, in steps S4 and S6, a bionic eye camera is used to acquire images of the target steel support interface, specifically including the following steps:

[0020] A1. Based on the current location of the device obtained by the GPS sensor and the target interface location in the configuration file, calculate the relative position of the target interface with respect to the device.

[0021] A2. Based on the current yaw angle of the device, determine whether the target interface is within the maximum field of view of the two cameras. If so, adjust the vertical and horizontal rotation of the bionic eye to make the target interface as close as possible to the center of the two bionic eye cameras.

[0022] A3. Collect camera image data and import the image data into the YOLOv5 model. Identify the interface and search for any interfaces without steel supports. If none are found, return the result.

[0023] A4. Based on the annotation box generated by YOLOv5, expand the annotation box by 50% in all directions and crop the image of the area selected by the expanded annotation box for subsequent processes;

[0024] A5. Based on the type of steel material or coating color of the outer perimeter of the support steel pipe interface, use its RGB pixel value as a reference, and expand the RGB pigment value by 20 to the upper and lower range as the threshold for subsequent image binarization.

[0025] A6. Binarize the image according to the threshold of A5, so that the outer periphery of the support steel pipe interface appears white in the image and the rest appears black.

[0026] A7. Extract contours from the binarized image, select the largest contour to form the minimum bounding rectangle, obtain the positions of the straight lines on both sides of the minimum bounding rectangle, and select the corresponding straight lines as the projection of the support steel pipe interface edge in the image according to the link direction in the configuration file.

[0027] A8. Based on the pixel coordinates of the support steel pipe interface edge in the images of the two cameras, calculate the specific position of the key point of the support steel pipe interface relative to the support equipment, so that the optical axis of the camera passes through the intersection point A of the central axis of the support steel pipe and the rightmost point of the interface, and the key point B determined by the intersection point with the outer periphery of the interface. The specific process is as follows: establish equations for the image processing results of the two bionic cameras respectively through the following formulas, and solve them simultaneously.

[0028]

[0029] in The intrinsic parameters of the camera are determined through camera calibration. The rotation matrix R can be determined based on the rotation angle of the bionic eye. Since the camera is fixed at a fixed position of the bionic eye and the distance between the origins is known, the translation matrix T can be obtained by performing pose transformation using the rotation matrix R. CThe distance from the camera to the desired point is related to the actual distance between the camera and the point in question. This distance is obtained by solving a system of equations relating the two cameras. u and v are pixel coordinates, obtained through image processing. c x =u0,c y =v0 is related to the offset between the image coordinate system and the pixel coordinates. The center point of the image coordinate system is the midpoint of the pixel coordinates (u0, v0), f x =f / d x f y =f / d y The transformation between the image coordinate system and the pixel coordinate system is related to the mapping ratio. This transformation involves mapping the length coordinates to pixel units proportionally. The mapping ratio is the single-pixel length dx and dy on the x and y axes, respectively. f is related to the camera focal length and is the camera focal length coefficient. W Y W Z W The coordinates of the center of the outer periphery of the interface in the body coordinate system are consistent in the relationship between the two bionic cameras.

[0030] Preferably, the robotic arm grasping scheme is generated in S5, specifically including the following steps:

[0031] B1. Collect image data from the bionic camera and import the image data into the YOLOv5 model to identify the support steel pipe;

[0032] B2. Based on the generated annotation boxes, select the annotation boxes with a confidence level greater than 0.96 and whose aspect ratio differs from the ratio of the length to the diameter of the support steel pipe by less than or equal to 15% as the gripping objects, and select the two gripping points of the two robotic arms at 2 / 5 and 4 / 5 of the total length of the support steel pipe.

[0033] B3. After selecting the target object, the rotation angle of each joint is calculated by using inverse kinematics based on the target grasp point.

[0034] B4. Based on the rotation angle of each joint, generate the trajectory plan using a cubic polynomial trajectory planning model.

[0035] Preferably, the step S7, which involves controlling the robotic arm to perform the grasping, transporting, precise docking, and installation operations of a single steel support, specifically includes the following steps:

[0036] C1. Based on the complete action sequence work plan generated by S6, control the mechanical claw to reach the gripping position and perform gripping. The gripping status of the support steel pipe is judged by the piezoresistive thin film sensor.

[0037] C2. Adjust the horizontal rotation of the bionic eye. If the support steel pipe is connected to the left side of the interface, the interface will be located on the right side of the camera's field of view; otherwise, the interface will be located on the left side of the camera's field of view.

[0038] C3. Control the robotic arm to move the support steel pipe to the coaxial position of the interface, with a distance of 200-300mm. Collect image data, binarize the image according to the threshold generated by A5, collect the contour, extract the upper edge of the two contours with the largest area. If the difference between the positions of the two upper edges is less than 20 pixels, it is determined that the support steel pipe is coaxial with the interface.

[0039] C4. After the support steel pipe is aligned with the interface axis, to ensure the horizontal movement of the support steel pipe, it is divided into 30mm segments according to the distance from the interface to the support steel pipe. A cubic polynomial trajectory planning model is used, and the overall initial and final speeds are set to zero, and the initial and final speeds between segments are set to 2mm / s to ensure low speed when approaching docking. The robotic arm is controlled to move the support steel pipe toward the interface according to the plan. During this period, an image is collected every 0.2 seconds. According to step C3, it is determined whether the coaxial alignment has been achieved. At the same time, the image is imported into the YOLOv5 model to determine whether the docking is completed. If the axis alignment fails, the key point positions of the interface are obtained again according to steps A4 to A8, and trajectory planning is performed to make the axes coincide. Step C4 is executed again.

[0040] After the C5 and YOLOv5 models have completed the docking, check the completion status of the robotic arm trajectory planning path. If it has been completed, then this step is considered complete.

[0041] Beneficial effects

[0042] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. This invention uses two robotic arms to grasp, transport, and connect steel pipes by mimicking the gripping method of a human arm. This improves the stability of the steel pipes throughout the entire process and prevents them from tilting or slipping during transport. At the same time, it uses dual bionic cameras to expand the field of view and has two image acquisition points, minimizing the obstruction caused by the robotic arms or the surrounding environment.

[0044] 2. This invention uses a bionic eye camera, which enables the camera to rotate horizontally or vertically through a stepper motor. This not only expands the field of view for image acquisition under extreme conditions, but also ensures that the working area is in the center of the camera's field of view in most cases, reducing the impact of factors such as camera optical axis offset on visual positioning.

[0045] 3. This invention introduces the YOLO model for annotation in target recognition and uses traditional visual methods for localization based on the annotation results, thereby improving image processing speed and achieving higher accuracy.

[0046] 4. This invention not only designs a bionic eye camera and its image processing related content, but also designs equipment for the establishment of support structures in the case of subway cut-and-cover construction and a work process based on the vision of the bionic eye camera, which improves construction speed and safety. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 This is a structural schematic diagram of the support equipment for subway foundation pit construction based on binocular vision and human eye bionics according to the present invention.

[0049] Figure 2 This is a schematic diagram of the mechanical claw piezoresistive thin-film sensor installation of the present invention;

[0050] Figure 3 This is a schematic diagram of the multi-sensor arrangement of the present invention;

[0051] Figure 4 This is a schematic diagram of the bionic eye camera of the present invention;

[0052] Figure 5 This is a schematic diagram of bionic eye movement control.

[0053] Figure 6 This is a schematic diagram showing the relationship between the horizontal rotation angle and the length of the flexible cable;

[0054] Figure 7 This is a schematic diagram showing the relationship between the vertical rotation angle and the length of the flexible cable;

[0055] Figure 8 Schematic diagram of the support steel pipe interface;

[0056] In the diagram: 1. Walking mechanism; 2. Workbench; 3. First robotic arm; 3-7-1. Finger joint one; 3-7-2. Finger joint two; 3-7-3. Finger joint three; 3-7-4. Piezoresistive thin-film sensor one; 3-7-5. Piezoresistive thin-film sensor two; 4. Second robotic arm; 5. Sensor and control system; 5-1. GPS sensor one; 5-2. Industrial control computer components; 5-3. LiDAR; 5-4. GPS sensor two; 6-1. Bionic eye camera one; 6-2. Bionic eye camera two; 6-1-1. Eyeball; 6-1-2. Rotation axis; 6-1-3. Camera; 6-1-4. Fixed link. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example:

[0059] like Figures 1-8 As shown in this embodiment, a subway foundation pit excavation equipment based on multi-joint linkage and its usage method are described.

[0060] like Figure 1 As shown, a support splicing device for subway foundation pit construction based on binocular vision and human eye bionics includes: a walking mechanism 1 for moving the entire device; a workbench 2 mounted on the walking mechanism 1 for mounting a sensor system 5 and a first robotic arm 3 and a second robotic arm 4; the first robotic arm 3 and the second robotic arm 4 are used to collaboratively grasp, transport, and dock support steel pipes to improve the stability of support construction; the sensor system 5 includes a lidar 5-3, a GPS sensor 1 5-1, a GPS sensor 2 5-4, a piezoresistive thin-film sensor, and two bionic eye cameras 6; and a control system 7 including an industrial computer, an STM32 control board, and a driver, used to process data from the sensor system and control the precise horizontal and vertical rotation of the bionic eyes, and to detect and adjust the grasping, transporting, and docking status of the robotic arms.

[0061] The lidar sensor, GPS sensor, and bionic eye camera are connected to the airborne router; the piezoresistive thin-film sensor and bionic eye camera are both connected to the STM32 control board; the airborne industrial computer is connected to the airborne router to obtain data from the sensors in the local area network; the STM32 control board is connected to the airborne industrial computer via RS232, transmits the signal from the piezoresistive thin-film sensor, and obtains the rotation signal from the bionic eye camera to control the rotation of the bionic eye camera.

[0062] like Figure 3 As shown, GPS sensor 5-1 is installed at the front of the equipment body, and GPS sensor 5-4 is installed at the rear of the equipment body. By using the two GPS sensors together, the position of the aircraft can be obtained, as well as the yaw angle, pitch angle, and roll angle. By using the two bionic eye cameras 6-1 and 6-2, the specific position of the steel pipe interface relative to the support equipment can be obtained, and the field of view under extreme conditions can be expanded. The bionic eye cameras 6-1 and 6-2 use two image acquisition points. When one camera is blocked, the other camera can acquire data to guide the work. The industrial control computer component 5-2 is installed on the equipment body and includes an airborne router, an airborne industrial control computer, and an airborne touch screen.

[0063] like Figure 2 As shown, the robotic arm 1 and robotic arm 2 in this example adopt existing structures well known to those skilled in the art. The robotic claws at the front ends of robotic arm 1 and robotic arm 2 are provided with interconnected finger joints 1 3-7-1, 2 3-7-2, and 3 3-7-3. Piezoresistive thin-film sensors 1 3-7-4 and 2 3-7-5 are provided on the inner side of finger joints 1 3-7-1 and 2 3-7-2.

[0064] like Figure 5 As shown, the rotation control method of the bionic eye camera consists of three parts: motion control, error compensation, and limit control.

[0065] (1) Motion control

[0066] The bionic eye employs open-loop control. The industrial computer calculates the horizontal and vertical rotation angles based on requirements and converts these angle values ​​into corresponding control signals, which are transmitted to the STM32 control board via RS232. The STM32 then converts these control signals into pulse and direction signals, which are sent to the drivers. The drivers drive the vertical and horizontal motors to rotate, thus causing the bionic eye to rotate by the corresponding vertical and horizontal angles. The speed and angle of the bionic eye's rotation are controlled by adjusting the frequency and number of pulses.

[0067] In this embodiment, the STM32 control board is model STM32F103VET6.

[0068] (2) Error Compensation

[0069] To compensate for errors generated during the bionic eye's steering process, theoretical error analysis is used to compensate for errors in the horizontal steering phase. The measured error value of horizontal steering is extracted, and the corresponding number of compensation pulses is calculated. A steering judgment program, well-known to those skilled in the art, is written on an STM32 control board. When the steering phase is detected, the number of pulse signals is compensated, thereby improving the rotational accuracy of horizontal steering. The specific compensation formula is as follows:

[0070] N=k(ΔT + ΔE)

[0071] Where: k is the proportional relationship between the number of pulses and the angle; ΔT is the tension error of the flexible cable, which only exists during the camera turning phase and is obtained by looking up a table from the current angle. The table is measured after the bionic eye camera is assembled, and is zero at other times; ΔE is the elastic deformation error of the flexible cable, which is obtained by looking up a table from the current angle. The table is measured after the bionic eye camera is assembled.

[0072] (3) Limit control

[0073] To prevent damage to the mechanism caused by the bionic eye's vertical and horizontal rotation exceeding the allowable range, a maximum constraint angle, or limit, is set for the rotation. Considering that the bionic eye's rotation angle is directly proportional to the number of pulses, a limit pulse calculation and judgment program, well-known to those skilled in the art, is written into the STM32 microcontroller. Based on the information provided by the control signal, the number of limit pulses of the corresponding timer is incremented or decremented, and it is determined in real time whether the number of limit pulses has reached the limit value. Once exceeded, the timer will immediately stop outputting pulses, thereby stopping the bionic eye's forward rotation. At the same time, it does not affect the bionic eye's reverse rotation; only the corresponding control signal needs to be input to reverse the movement.

[0074] like Figure 4 As shown, the bionic eye camera in this embodiment is a dual-motor cable-driven bionic eye. The bionic eye camera includes: a camera 6-1-3, an eyeball structure 6-1-1, a rotating shaft 6-1-2, and a fixed connecting rod 6-1-4. The eyeball 6-1-1 is mounted and fixed on the vertical axis of the rotating shaft 6-1-2. The horizontal axis of the rotating shaft 6-1-2 is mounted on the fixed connecting rod 6-1-4, which is fixed to the worktable 2 by bolts. The camera 6-1-3 is mounted on the eyeball structure 6-1-1. The eyeball structure 6-1-1 is driven to rotate by a flexible cable. The flexible cable is fixed to the reel of the drive motor. The rotation of the motor causes the length of the flexible cable to change, thereby simulating the elongation and contraction of the extraocular muscles, thus realizing eyeball rotation. A groove for fixing the flexible cable is reserved on the vertical axis of the rotating shaft 6-1-2; a groove is reserved on the horizontal axis of the rotating shaft 6-1-2. The central hole allows the flexible cable to pass through along its vertical axis. Bearings are installed on both the vertical and horizontal axes to reduce friction. The eyeball 6-1-1 consists of a front eyeball and a rear eyeball. The front eyeball has a pre-drilled hole for the flexible cable. A nut is pre-embedded on the front eyeball and connected to the rear eyeball by threads. Both eyeballs have bearing mounting grooves so that they can be fixed on the vertical axis. The flexible cable that controls vertical rotation has one end fixed to the flexible cable fixing groove on the vertical axis, and the other end, away from the flexible cable fixing groove, is fixed to a corresponding reel on the vertical axis. The flexible cable that controls horizontal rotation has one end fixed to the pre-drilled hole in the front eyeball, and the other end, away from the pre-drilled hole, passes through the central hole in the horizontal axis and is fixed to the corresponding reel on the horizontal axis along the groove of the fixing link 6-1-4. The corresponding reels on the vertical and horizontal axes are driven by independent motors.

[0075] In this example, the flexible cable, reel, and motor are all existing products well known to those skilled in the art, and the connection methods between them are also existing connection methods well known to those skilled in the art.

[0076] like Figure 6As shown, O is the center of the eyeball; R is the sum of the eyeball radius r1 and the flexible cord radius r2, i.e., R = r1 + r2; M is the initial position point where the flexible cord connects to the eyeball; N is the final position point where the eyeball rotates vertically upwards; H is the arc. At any point on the horizon; assuming the eyeball rotates upwards from M to H, then the vertical rotation angle λ of the eyeball is ∠MOH. Using... The change in line length between them represents the change in the line length of the entire cord-1. Therefore, the rotation angle λ of the eyeball is linearly related to the change in the line length L1 of the cord-1.

[0077]

[0078] like Figure 7 As shown, A is the initial position point where the ligament 3 connects to the eyeball; B is the point where the ligament 3 is tangent to the eyeball; C is the arc. Any point on the line; D is the ending point where the chord 3 connects to the eyeball; D is the ending point where the eyeball rotates horizontally to the right; E is the turning point of the chord 3; F is the reference point, OF⊥EF; Assuming the eyeball rotates to the right from A to C, then the vertical rotation angle θ of the eyeball is ∠AOC; the reference angle ∠COF=γ、 ∠EFC=α The change in line length between C and E can be used to represent the change in line length of the three systems of the chord. It is known that the eyeball rotation has two states before and after the point of tangency, so it is divided into two stages.

[0079] Phase 1: When C is in During this period, the eye's rotation angle θ is linearly related to the change in the length L3 of the chordal 3.

[0080]

[0081] Phase Two: When C is in During this period, the rotation angle θ of the eyeball has a non-linear relationship with the change in the length L3 of the chordus 3, and the final relationship is as follows:

[0082]

[0083] The support steel pipe interface structure in this embodiment is as follows: Figure 8 As shown.

[0084] In this embodiment, the walking mechanism 1, workbench 2, first robotic arm 3, finger joint 1 3-7-1, finger joint 2 3-7-2, finger joint 3 3-7-3, piezoresistive thin-film sensor 1 3-7-4, piezoresistive thin-film sensor 2 3-7-5, second robotic arm 4, sensor system 5, GPS sensor 1 5-1, industrial control computer component 5-2, lidar 5-3, GPS sensor 2 5-4, bionic eye camera 1 6-1, bionic eye camera 2 6-2, eyeball 6-1-1, rotating axis 6-1-2, camera 6-1-3, fixed link 6-1-4, control system 7, and other structures not disclosed in detail all adopt existing products or structures well known to those skilled in the art, and the connection or control methods between them also adopt existing connection or control methods well known to those skilled in the art.

[0085] This embodiment discloses a method for using subway foundation pit excavation equipment based on multi-joint linkage. The single-step operation of gripping, transporting, and connecting support steel pipes based on the above equipment and control principles specifically includes the following steps:

[0086] S1. Using methods well known to those skilled in the art, start the support equipment in this example and perform equipment status self-check, equipment initialization, and sensor calibration;

[0087] S2. The operator selects and loads the configuration file of the overall support scheme stored locally in the console, and chooses whether to import the support scheme completion status log file. If there is no configuration file, the equipment will check the completion status and generate a log file.

[0088] S3. The equipment obtains its current position through GPS sensor 5-4 and obtains its fuselage yaw angle with the assistance of GPS sensor 5-1. The specific method of obtaining the position is an existing method known to those skilled in the art. The system queries the configuration file and log file. Based on the query results, the system automatically plans a path and navigates to the nearest steel support interface position that is in the "not installed" state. The fuselage is oriented perpendicular to the axis of the support steel pipe interface, the support steel pipe interface is located in the center of the front of the fuselage, and the distance between the fuselage and the support steel pipe interface is 1m to 2m.

[0089] S4. Use bionic eye camera 1 6-1 and bionic eye camera 2 6-2 to acquire images of the target steel support interface, obtain the interface status, determine whether the interface where no steel support is installed in the log has steel support installed, if it has been installed, change the interface status in the log file, and return to S3 to execute.

[0090] S5. Collect image data of the storage warehouse area known to those skilled in the art, identify usable steel pipes within the storage warehouse, and generate a robotic arm grasping plan; "generating a robotic arm grasping plan" specifically includes the following operations:

[0091] B1. Collect camera image data and import the image data into the YOLOv5 model to identify the support steel pipe;

[0092] B2. Based on the generated annotation boxes, select the annotation boxes with a confidence level of 0.96 or higher, and whose aspect ratio differs from the aspect ratio of the support steel pipe by no more than 15%, as the gripping objects. Select 2 / 5 and 4 / 5 of the total length of the support steel pipe as the two gripping points of the two robotic arms to prevent the steel pipe from accidentally rotating or slipping during single-point gripping.

[0093] B3. After the object to be grasped is selected, the rotation angle of each joint is inversely calculated based on the target grasping point using inverse kinematics models such as the DH parameter method, which are well known to those skilled in the art.

[0094] B4. Based on the rotation angle of each joint, generate the trajectory plan using a cubic polynomial trajectory planning model;

[0095] S6. Use bionic eye camera 6-1 and bionic eye camera 6-2 again to collect images of the target steel support interface, obtain the interface status and position, and combine the robotic arm grasping scheme to plan the complete action sequence of the robotic arm: grasp the steel pipe - transport the steel pipe - accurately connect and install the steel pipe to the target interface.

[0096] Steps S4 and S6, "Using bionic eye camera 1 6-1 and bionic eye camera 2 6-2 to acquire images of the target steel support interface and obtain the interface status," specifically include the following operations:

[0097] A1. Based on the current location obtained by GPS sensor 1 and GPS sensor 2, and the target interface location in the configuration file, calculate the relative position of the target interface with respect to the support equipment.

[0098] A2. Based on the current yaw angle of the support equipment, determine whether the target interface is within the maximum field of view of the two cameras. If so, adjust the vertical and horizontal rotation of the bionic eye to bring the target interface closer to the center of the two bionic eye cameras.

[0099] A3. Collect camera image data and import the image data into the YOLOv5 model. Identify the interface and search for any interfaces without steel supports. If none are found, return the result.

[0100] A4. Based on the annotation box generated by YOLOv5, expand the annotation box by 50% in all directions and crop the image of the area selected by the expanded annotation box for subsequent processes;

[0101] A5. Based on the type of steel material or coating color of the outer perimeter of the support steel pipe interface, use its RGB pixel value as a reference, and expand the RGB pigment value by 20 to the upper and lower range as the threshold for subsequent image binarization.

[0102] A6 binarizes the image based on the threshold of A5, making the outer periphery of the support steel pipe interface appear white in the image, while the rest appears black.

[0103] A7. Extract contours from the binarized image, select the largest contour to form the minimum bounding rectangle, obtain the positions of the straight lines on both sides of the minimum bounding rectangle, and select the corresponding straight lines as the projection of the support steel pipe interface edge in the image according to the link direction in the configuration file.

[0104] A8. Based on the pixel coordinates of the support pipe interface edge in the images of the two cameras, calculate the specific position of the key points of the support pipe interface relative to the support equipment, such as... Figure 8 As shown, the arrow points in the direction of the camera's optical axis, and the dotted line represents the central axis of the support steel pipe. The intersection point A, where the camera's optical axis line passes through the central axis of the support steel pipe and the rightmost point of the interface, and the intersection point with the outer perimeter of the interface, is the determined key point B. The specific process is as follows: Equations are established for the image processing results of the two bionic cameras using the following formulas, and then the equations are solved simultaneously.

[0105]

[0106] in The intrinsic parameters of the camera are determined through camera calibration. The rotation matrix R can be determined based on the rotation angle of the bionic eye. Since the camera is fixed at a fixed position of the bionic eye and the distance between the origins is known, the translation matrix T can be obtained by performing pose transformation using the rotation matrix R. C The equations for the two cameras are solved by combining the equations for the two cameras. u and v are pixel coordinates obtained from image processing. c x =u0,c y =v0 is related to the offset between the image coordinate system and the pixel coordinates. The center point of the image coordinate system is the midpoint of the pixel coordinates (u0, v0), f x =f / d x f y =f / d y The transformation between the image coordinate system and the pixel coordinate system is related to the mapping ratio. This transformation involves mapping the length coordinates to pixel units proportionally. The mapping ratio is the single-pixel length dx and dy on the x and y axes, respectively. f is related to the camera focal length and is the camera focal length coefficient. W Y W Z W The coordinates of the center of the outer periphery of the interface in the body coordinate system are consistent in the relationship between the two bionic cameras;

[0107] S7. Based on the complete action sequence work plan generated in S6, control the robotic arm to perform the gripping, transporting, precise docking, and installation operations of a single steel support; specifically including the following operations:

[0108] C1. Based on the generated gripping trajectory plan, control the mechanical claw to reach the gripping position and perform gripping. The gripping completion status of the support steel pipe is judged by the piezoresistive thin film sensor 1 3-7-4 and the piezoresistive thin film sensor 2 3-7-5.

[0109] C2. Adjust the horizontal rotation of the bionic eye. If the support steel pipe is connected to the left side of the interface, the interface will be located on the right side of the camera's field of view; otherwise, it will be located on the left side.

[0110] C3. Control the robotic arm to move the support steel pipe to the coaxial position of the interface, with a distance of 200-300mm. Collect image data, binarize the image according to the threshold generated by A5, collect the contour, extract the upper edge of the two contours with the largest area. If the difference between the positions of the two upper edges is less than 20 pixels, it is determined that the support steel pipe is coaxial with the interface.

[0111] C4. After the support steel pipe is aligned with the interface axis, to ensure the horizontal movement of the support steel pipe, it is divided into 30mm segments according to the distance from the interface to the support steel pipe. A cubic polynomial trajectory planning model is used, and the overall initial and final speeds are set to zero, and the initial and final speeds between segments are set to 2mm / s to ensure low speed when approaching docking. The robotic arm is controlled to move the support steel pipe toward the interface according to the plan. During this period, an image is collected every 0.2 seconds. According to step C3, it is determined whether the coaxial alignment has been achieved. At the same time, the image is imported into the YOLOv5 model to determine whether the docking is completed. If the axis alignment fails, the key point positions of the interface are obtained again according to steps A4 to A8, and trajectory planning is performed to make the axes coincide. Step C4 is executed again.

[0112] After the C5 and YOLOv5 models have completed the docking, check the completion status of the robotic arm trajectory planning path. If it has been completed, then this step is considered complete.

[0113] S8. After completion, check the splicing status and modify the log file.

[0114] The YOLOv5 model in this embodiment is a prior art model well known to those skilled in the art.

[0115] In this specification, the apparatus disclosed in the embodiments corresponds to the methods disclosed in the embodiments, and the relevant parts can be referred to in the method section.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A support and splicing device for subway foundation pit construction based on binocular vision and human eye bionics, characterized in that, include: Walking mechanism (1), used for the entire mobile device; The workbench (2) installed on the walking mechanism (1) is used to install the sensor system (5) and the first robotic arm (3) and the second robotic arm (4); the first robotic arm (3) and the second robotic arm (4) are used to collaboratively grasp, transport and dock the support steel pipe to improve the stability of the support construction; the sensor system (5) includes a lidar (5-3), a GPS sensor, a piezoresistive thin film sensor and two bionic eye cameras (6); the control system (7) includes an industrial computer, an STM32 control board and a driver, used to process the data of the sensor system and control the precise horizontal and vertical rotation of the bionic eye as well as to detect and adjust the grasping, transporting and docking status of the robotic arm.

2. The support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 1, characterized in that, The bionic eye camera is a dual-motor cable-driven bionic eye. The bionic eye camera includes: a camera (6-1-3), an eyeball structure (6-1-1), a rotating shaft (6-1-2), and a fixed connecting rod (6-1-4). The eyeball (6-1-1) is mounted and fixed on the vertical axis of the rotating shaft (6-1-2). The horizontal axis of the rotating shaft (6-1-2) is mounted on the fixed connecting rod (6-1-4). The fixed connecting rod (6-1-4) is fixed to the worktable (2) by bolts. The camera (6-1-3) is mounted on the eyeball structure (6-1-1). The eyeball structure (6-1-1) is driven to rotate by a flexible cable. The flexible cable is fixed to the pulley of the drive motor. The rotation of the motor causes the length of the flexible cable to change, thereby simulating the elongation and contraction of the extraocular muscles, thus realizing the rotation of the eyeball.

3. The support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 2, characterized in that, The vertical axis of the rotating shaft (6-1-2) has a pre-reserved groove for fixing a flexible cable; the horizontal axis of the rotating shaft (6-1-2) has a pre-reserved central hole for the flexible cable to pass through along its vertical axis; bearings are installed on both the vertical and horizontal axes to reduce friction; the eyeball (6-1-1) consists of an anterior eyeball and a posterior eyeball; the anterior eyeball has a pre-reserved hole for installing a flexible cable, and a nut is pre-embedded on the anterior eyeball to connect with the posterior eyeball via a thread; both eyeballs have bearing mounting grooves so that the two eyeballs can be installed and fixed on the vertical axis. On the shaft; a flexible cable controlling vertical rotation, one end of which is fixed to the flexible cable fixing groove on the vertical shaft, and the other end away from the flexible cable fixing groove is fixed to a corresponding reel on the vertical shaft; a flexible cable controlling horizontal rotation, one end of which is fixed to the pre-reserved flexible cable mounting hole in the anterior eye hemisphere, and the other end away from the pre-reserved flexible cable mounting hole in the hemisphere passes through the pre-reserved shaft hole in the horizontal shaft and is arranged and fixed to the corresponding reel on the horizontal shaft along the groove of the fixing link (6-1-4); the corresponding reel on the vertical shaft and the corresponding reel on the horizontal shaft are driven by independent motors.

4. The support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 2, characterized in that, The rotation control module of the bionic eye camera includes: a motion control module for converting angle commands into motor pulse signals; an error compensation module for pulse compensation of horizontal steering errors; and a limit control module for limiting the rotation angle to prevent structural damage.

5. The support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 1, characterized in that, The piezoresistive thin-film sensor is installed at the joint of the mechanical claw of the first robotic arm (3) and the second robotic arm (4) to detect the grasping force and provide feedback on the grasping status.

6. The support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 1, characterized in that, The lidar sensor, GPS sensor, and bionic eye camera are connected to the airborne router; the piezoresistive thin-film sensor and bionic eye camera are both connected to the STM32 control board; the airborne industrial computer is connected to the airborne router to obtain data from the sensors in the local area network; the STM32 control board is connected to the airborne industrial computer via RS232, transmits the signal from the piezoresistive thin-film sensor, and obtains the rotation signal from the bionic eye camera to control the rotation of the bionic eye camera.

7. A method for using a support and splicing device for subway foundation pit construction based on binocular vision and human eye bionics, as described in any one of claims 1-6, characterized in that... Includes the following steps: S1. Start the support and splicing equipment for subway foundation pit construction based on binocular vision and human eye bionics, and perform equipment status self-check, equipment initialization and sensor calibration. S2. The operator selects and loads the configuration file of the overall support scheme stored locally in the console, and chooses whether to import the support scheme completion status log file. If there is no configuration file, the equipment will check the completion status and generate a log file. S3. The system obtains the current location of the device using two GPS sensors and assists in obtaining the fuselage yaw angle. It then queries the configuration file and log files. Based on the query results, the control system automatically plans a path and navigates to the nearest steel support interface location that is currently in an "inactive" state. The equipment body should be oriented perpendicular to the axis of the support steel pipe interface, with the support steel pipe interface located in the center of the front of the equipment body, and the equipment body and the support steel pipe interface should be 1m to 2m apart. S4. Use a bionic eye camera to capture images of the target steel support interface, obtain the interface status, determine whether the interface without steel support in the log has steel support installed, if it has been installed, change the interface status in the log file, and return to S3 to execute. S5. Collect image data of the storage area through a bionic eye camera, identify the available steel pipes in the storage area, and generate a robotic arm grasping scheme. S6. Use the bionic eye camera again to collect images of the target steel support interface, obtain the interface status and position, and combine the robotic arm grasping scheme to plan the complete action sequence of the robotic arm: grasp the steel pipe - transport the steel pipe - accurately connect and install the steel pipe to the target interface; S7. Based on the complete action sequence work plan generated by S6, control the robotic arm to perform the gripping, transporting, precise docking and installation operations of a single steel support; S8. After completion, check the splicing status and modify the log file.

8. The method of using a support splicing device for subway foundation pit construction based on binocular vision and human eye bionics as described in claim 7, characterized in that, Steps S4 and S6 involve using a bionic eye camera to acquire images of the target steel support interface, specifically including the following steps: A1. Based on the current location of the device obtained by the GPS sensor and the target interface location in the configuration file, calculate the relative position of the target interface with respect to the device. A2. Based on the current yaw angle of the device, determine whether the target interface is within the maximum field of view of the two cameras. If it is, adjust the vertical and horizontal rotation of the bionic eye to bring the target interface closer to the center of the two bionic eye cameras. A3. Collect camera image data and import the image data into the YOLOv5 model. Identify the interface and search for any interfaces without steel supports. If none are found, return the result. A4. Based on the annotation box generated by YOLOv5, expand the annotation box by 50% in all directions and crop the image of the area selected by the expanded annotation box for subsequent processes; A5. Based on the type of steel material or coating color of the outer perimeter of the support steel pipe interface, use its RGB pixel value as a reference, and expand the RGB pigment value by 20 to the upper and lower range as the threshold for subsequent image binarization. A6. Binarize the image according to the threshold of A5, so that the outer periphery of the support steel pipe interface appears white in the image and the rest appears black. A7. Extract contours from the binarized image, select the largest contour to form the minimum bounding rectangle, obtain the positions of the straight lines on both sides of the minimum bounding rectangle, and select the corresponding straight lines as the projection of the support steel pipe interface edge in the image according to the link direction in the configuration file. A8. Based on the pixel coordinates of the support steel pipe interface edge in the images of the two cameras, calculate the specific position of the key point of the support steel pipe interface relative to the support equipment, so that the optical axis of the camera passes through the intersection point A of the central axis of the support steel pipe and the rightmost point of the interface, and the key point B determined by the intersection point with the outer periphery of the interface. The specific process is as follows: establish equations for the image processing results of the two bionic cameras respectively through the following formulas, and solve them simultaneously. in The intrinsic parameters of the camera are determined through camera calibration. The rotation matrix R can be determined based on the rotation angle of the bionic eye. Since the camera is fixed at a fixed position of the bionic eye and the distance between the origins is known, the translation matrix T can be obtained by performing pose transformation using the rotation matrix R. C The distance from the camera to the desired point is related to the actual distance between the two cameras, and is obtained by solving the equations relating the two cameras simultaneously; u and v are pixel coordinates, obtained through image processing; c x =u0,c y =v0 is related to the offset between the image coordinate system and the pixel coordinates; the center point of the image coordinate system is the midpoint of the pixel coordinates (u0, v0); f x =f / d x f y =f / d y The transformation between the image coordinate system and the pixel coordinate system is related to the mapping ratio. This transformation involves mapping the length coordinates to pixel units proportionally, with the mapping ratio being the single-pixel lengths dx and dy on the x and y axes, respectively. f is related to the camera focal length and is the camera focal length coefficient. Z C The equations relating the two cameras are solved by combining the equations, where u and v are pixel coordinates, and X is obtained through image processing. W Y W Z W The coordinates of the center of the outer periphery of the interface in the body coordinate system are consistent in the relationship between the two bionic cameras.

9. The method of using a support splicing device for subway foundation pit construction based on binocular vision and human eye bionics, as described in claim 7, is characterized in that... The process of generating a robotic arm grasping scheme in S5 includes the following steps: B1. Collect image data from the bionic camera and import the image data into the YOLOv5 model to identify the support steel pipe; B2. Based on the generated annotation boxes, select the annotation boxes with a confidence level greater than 0.96 and whose aspect ratio differs from the ratio of the length to the diameter of the support steel pipe by less than or equal to 15% as the gripping objects, and select the two gripping points of the two robotic arms at 2 / 5 and 4 / 5 of the total length of the support steel pipe. B3. After selecting the object to be grasped, the rotation angle of each joint is calculated by using inverse kinematics based on the target grasping point. B4. Based on the rotation angle of each joint, generate the trajectory plan using a cubic polynomial trajectory planning model.

10. The method of using a support splicing device for subway foundation pit construction based on binocular vision bionics, as described in claim 7, is characterized in that... Step S7 involves controlling the robotic arm to perform the gripping, transporting, precise docking, and installation of a single steel support, specifically including the following steps: C1. Based on the complete action sequence work plan generated by S6, control the mechanical claw to reach the gripping position and perform gripping. The gripping status of the support steel pipe is judged by the piezoresistive thin film sensor. C2. Adjust the horizontal rotation of the bionic eye. If the support steel pipe is connected to the left side of the interface, the interface will be located on the right side of the camera's field of view; otherwise, the interface will be located on the left side of the camera's field of view. C3. Control the robotic arm to move the support steel pipe to the coaxial position of the interface, with a distance of 200-300mm. Collect image data, binarize the image according to the threshold generated by A5, collect the contour, extract the upper edge of the two contours with the largest area. If the difference between the positions of the two upper edges is less than 20 pixels, it is determined that the support steel pipe is coaxial with the interface. C4. After the support steel pipe is aligned with the interface axis, to ensure the horizontal movement of the support steel pipe, it is divided into 30mm segments according to the distance from the interface to the support steel pipe. A cubic polynomial trajectory planning model is used, and the overall initial and final speeds are set to zero, and the initial and final speeds between segments are set to 2mm / s to ensure low speed when approaching docking. The robotic arm is controlled to move the support steel pipe toward the interface according to the plan. During this period, an image is collected every 0.2 seconds. According to step C3, it is determined whether the coaxial alignment has been achieved. At the same time, the image is imported into the YOLOv5 model to determine whether the docking is completed. If the axis alignment fails, the key point positions of the interface are obtained again according to steps A4 to A8, and trajectory planning is performed to make the axes coincide. Step C4 is executed again. After the C5 and YOLOv5 models have completed the docking, check the completion status of the robotic arm trajectory planning path. If it has been completed, then this step is considered complete.

Images