First-view-angle teleoperation concrete construction system based on virtual reality

Abstract

The invention relates to a first visual angle teleoperation concrete construction system based on virtual reality, and belongs to the technical field of robot teleoperation and AR. The system comprises a concrete spreader subsystem, an integrated annular light source target, an electronic compass and a motion control module, and the enhanced teleoperation subsystem is integrated with a first visual angle motion resolving module, a teleoperation handle and AR glasses. Resolving relative poses of a tool coordinate system and a camera coordinate system by fusing annular light source target visual positioning, double electronic compass global absolute orientation data and joint angle data; meanwhile, the AR glasses render a coordinate system and state information by taking the center of the annular light source target as an original point; establishing an orientation alignment relation between coordinate systems of the two electronic compasses based on global absolute orientation data of the two electronic compasses; and mapping the first visual angle operation instruction into a target pose in a base coordinate system, and driving and executing the first visual angle operation instruction. The teleoperation of visual space mapping is realized through the first visual angle resolving module, the AR and the teleoperation, so that the operation safety is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of robot teleoperation and augmented reality (AR) interaction technology, and in particular to a first-view augmented teleoperation system for concrete placing boom construction. Background Technology

[0002] Concrete placing booms are key equipment in building construction. Currently, the end-point control in this field mainly relies on the traditional mode of "control console operation + human assistance": the operator makes judgments remotely by relying on a two-dimensional control panel and limited visual feedback, while another assistant is required to go into the dangerous work area to provide feedback through a walkie-talkie or to directly perform contact-based manual traction.

[0003] The traditional approach exposes support personnel to the hazardous environment of heavy concrete placing booms and flowing concrete at close range, facing threats such as collisions and splashes, resulting in poor safety. CN118547891B discloses a method to mitigate collision risks by adding a flexible structure to the end effector, but this is only a passive protection and does not change the inherent dangers of contact control; at the same time, the approach also poses a risk of disconnect between operation and perception, as the operator cannot intuitively obtain end effector depth and attitude information.

[0004] In the more general field of robot teleoperation technology, existing methods also have inherent limitations. Traditional methods mainly rely on a fixed camera's third-party observation perspective and joystick-based joint mapping control. With the former, it is difficult for the operator to perceive the depth and posture of the end effector in three-dimensional space from a two-dimensional image, and insufficient spatial feedback leads to difficulties in fine manipulation; with the latter, there is no intuitive spatial correspondence between the operation command and the actual movement direction of the end effector, requiring the operator to perform coordinate transformations through imagination, resulting in a high cognitive load and a certain rate of error.

[0005] In summary, there is an urgent need in this field for an innovative control system solution that can fundamentally transform the existing interaction paradigm and enable contactless, intuitive, and safe remote operation of concrete placing booms. Summary of the Invention

[0006] To address the shortcomings of existing technologies, such as the inability to physically isolate operators from hazardous environments and the lack of intuitive spatial mapping between control commands and end-effector movements, this invention provides a first-view enhanced teleoperation system and method for concrete construction. The technical solution of this invention is as follows: A first-view enhanced teleoperation system for concrete construction includes a concrete placing machine subsystem and an enhanced teleoperation subsystem.

[0007] Furthermore, the concrete placing boom subsystem includes a power module 1, a placing boom body, a motion control module, a data acquisition module, and a wireless data transmission module 1; the power module 1 is used to supply power to the concrete placing boom subsystem; the placing boom body is used to perform actual work tasks; the motion control module is connected to the placing boom body and is used to adjust the joint angle of the robotic arm and the posture of the placing tube according to the drive signal; the wireless data transmission module 1 is used to transmit data between the two subsystems; the enhanced remote operation subsystem includes a remote operation handle, a safety helmet, and AR glasses; the safety helmet is used to integrate sensors and a processing module; the AR glasses are used to realize visual computing and 3D rendering.

[0008] Furthermore, the data acquisition module includes: an electronic compass 1, a joint encoder, and a ring light source target. The electronic compass 1 is used to provide global absolute orientation data of the fabric laying machine base; the ring light source target is set on the end hose for visual positioning marking.

[0009] Furthermore, the remote control handle includes: a power module 2, directional buttons, a safety button, and a wireless data transmission module 2; the power module 2 is used to power the remote control handle; the directional buttons are used to output remote control commands on the XYZ axes in the first viewpoint coordinate system; the safety button is used to ensure the output of any command with the operator's permission; and the wireless data transmission module 2 is used to send the remote control commands to the safety helmet.

[0010] Furthermore, the safety helmet includes: a power module 3, a power supply module, a first-view motion calculation module, a wireless data transmission module 3, and a wired data transmission module 1 for the safety helmet; the power module 3 is used to supply power to the integrated components inside the safety helmet; the power supply module is used to transmit power from the power module 2 to the AR glasses; the first-view motion calculation module is used to convert the relative pose between coordinate systems into the target pose in the base coordinate system, and then calculate the drive signals of each joint and the attitude control signals of the end effector based on the target pose; the wireless data transmission module 3 is used to receive the status information of the concrete placing machine subsystem and send the calculated motion commands to the concrete placing machine subsystem; the wired data transmission module 1 for the safety helmet is used to send rendering information to the AR glasses.

[0011] Furthermore, the AR glasses include: a miniature camera, an optical engine, an electronic compass 2, an AR data processing module, and a wired data transmission module 2 for the AR glasses; the miniature camera is used to capture environmental images; the optical engine is used to display augmented reality information; the electronic compass 2 is used to provide global absolute orientation data for the AR glasses; the AR data processing module is used to receive the relative pose results and system status data calculated by the first-view motion calculation module, and generate image rendering frames containing augmented reality information such as target coordinate system, text labels, and status icons; the wired data transmission module 2 for the AR glasses is used to receive rendering information from the safety helmet.

[0012] Specifically, the augmented reality solution of the AR glasses uses a waveguide scheme. The optical engine includes a display module, a waveguide, and a coupler. The display module generates the original virtual image. The waveguide uses the principle of total internal reflection to send the image to the human eye. The coupler guides light into and out of the waveguide to complete the input and output of the image.

[0013] Furthermore, the annular light source target is composed of a series of short, closely arranged LED light sources. These light sources are evenly distributed along the circumference formed by two symmetrical support screws, and their high brightness and good continuity ensure that a sufficient number of feature light points can be observed from any angle from the camera's perspective. The lower surface of the annular light source target must be perpendicular to the axis of the flexible tube. This is a geometric prerequisite for the subsequent visual algorithm to accurately calculate the z-axis of the tool coordinate system. The target as a whole is stably positioned at a certain height above the discharge port, thereby preventing concrete splashes from below from contaminating the light source.

[0014] The core data streams of the system include motion control stream, AR rendering stream, and operation state stream.

[0015] Furthermore, the motion control flow process is as follows: S1: The wireless data transmission module 3 continuously receives system status feedback from the concrete placing machine subsystem through the wireless data transmission module 1, including global absolute orientation data provided by the electronic compass of the placing machine base and joint angle data provided by the encoders of each joint. S2: The wireless data transmission module 3 receives motion commands from the remote control handle through the wireless data transmission module 2; at the same time, it receives the ring light source target image captured by the miniature camera transmitted from the AR glasses and the global absolute orientation data of the AR glasses provided by the built-in electronic compass through the wired data transmission module 1 of the safety helmet. S3: The first-person perspective motion calculation module collects the above multi-source information and performs fusion processing: based on the ring light source target image, camera intrinsic parameters, and the known size of the ring light source target, the relative pose of the tool coordinate system with respect to the camera coordinate system is calculated; based on joint angle data and global absolute orientation data of the base, the theoretical pose of the tool coordinate system with respect to the base coordinate system is calculated through the robot's forward kinematics; and based on the two global absolute orientation data, the orientation alignment relationship between the AR glasses coordinate system and the cloth machine base coordinate system is established. S4: Wireless data transmission module 3 collects the remote operation commands transmitted by wireless data transmission module 2. The first-view motion calculation module maps the remote operation commands into the target pose in the tool coordinate system under the coordinate system of the fabric laying machine base, based on the orientation alignment relationship. Then, it calculates the control signals, including joint drive signals and attitude control signals, through inverse kinematics. S5: The control command is sent to the wireless data transmission module 1 via the wireless data transmission module 3, and the motion control module drives the robotic arm and end effector to move according to the control signal.

[0016] Furthermore, the AR rendering stream transmission process is as follows: the first-view motion calculation module sends the calculated pose of the tool coordinate system relative to the AR glasses coordinate system and the real-time status data of the system to the AR glasses through the wired data transmission module 1 of the safety helmet (3); the AR data processing module in the AR glasses receives the above data and generates a rendering frame including a virtual coordinate system with the center of the ring light source target as the origin and a status information panel; the rendering frame is sent to the optical engine, which converts the image information into light rays, and combines them with the light rays of the real scene through waveguides and couplers, and finally forms an augmented reality fused image on the display module.

[0017] Furthermore, the operation status stream transmission process is as follows: the first-view motion calculation module determines the current system operation status in real time based on a preset safety distance; the determination result is used as an operation status command and sent to the AR glasses for display via wired data transmission 1 of the safety helmet; the AR glasses render and display the received operation status information to the operator in real time; the operator controls the opening and closing of the safety button according to the operation status information, and closes the safety button when the status information is "ready" or "in motion"; when the status information is "emergency stop" or "collaborative calibration", the safety button is opened, and the motion control module also stops working due to physical disconnection, thereby realizing closed-loop management of interaction permissions.

[0018] The motion control flow, AR rendering flow, and operation state flow are executed in parallel and continuously in a loop, ensuring a precise operation loop, a status display loop, and a safe interaction loop, thereby jointly achieving a smooth, fast, and safe teleoperation experience for the system.

[0019] The coordinate system transformation relationship involves the following core coordinate systems: the fabric placing machine base coordinate system {B}, the coordinate system of each joint of the robotic arm {J_i}, the virtual coordinate system {D} with the center of the ring light source target as the origin, and the camera coordinate system {C}.

[0020] The global absolute orientation angle α1 of the fabric machine base and the global absolute orientation angle α2 of the AR glasses are obtained based on the data from the electronic compass.

[0021] The translation operator from the coordinate system of joint i-1 to joint i is used To represent, the rotation operator is used To represent, we use the transformation operator. To express.

[0022] S1: Based on the angle parameters of each joint of the robotic arm transmitted by the concrete placing boom subsystem, and the parameters of each robotic arm, the translation operators between the coordinate systems of each joint can be obtained. ; S2: This leads to the translation operator of the tool coordinate system relative to the base coordinate system {B}. ; S3: Based on the global orientation angles of the fabric laying machine base and the AR glasses, the rotation operators of the fabric laying machine base coordinate system {B} and the camera coordinate system {C} can be calculated. ; S4: The orientation of the tool coordinate system {T} is determined by the camera coordinate system {C}, thereby solving for the rotation operator between the tool coordinate system {T} and the fabric placing machine base coordinate system {B}. ; S5: By translation operator and rotation operator Then, the transformation operator between the two coordinate systems can be calculated. ; Furthermore, in the first-view motion calculation module, the visual detection and pose calculation process of the miniature camera for the ring-shaped light source target is as follows: S1: The miniature camera continuously acquires scene images containing the ring light source target, and preprocesses the images, including color space-based region segmentation and edge enhancement processing, to highlight the contour features of the target marker; S2: Perform contour extraction and analysis on the preprocessed image. Through contour tracking and shape matching algorithms, identify and separate the complete image contours of the two ring light source targets. Based on the contours, calculate the geometric features of the rectangular contours of each ring light source target, including the center image coordinates, width, and pixel distance between the centers of the two rectangles. S3: Based on the fixed left-right positional relationship of the two ring-shaped light source targets, establish the correspondence between image coordinates and physical coordinates. By determining the horizontal orientation of the two identified rectangular contours in the image, they are distinguished as the left and right light strips, thus completing the matching; S4: Based on the rectangular contour geometric features calculated in step S2, the previously calibrated camera intrinsic parameters, and the correspondence established in step S3, the rotation matrix and translation vector from the target coordinate system to the camera coordinate system are calculated using the PnP algorithm. The known fixed physical distance between the two ring-shaped light source targets provides an absolute scale reference for monocular vision, enabling stable and reliable calculation of the tool coordinate system's three-dimensional coordinates and three-axis orientation in the camera coordinate system, thus completing the pose calculation of the tool coordinate system relative to the camera coordinate system.

[0023] Furthermore, the core of the rendering process for the AR glasses lies in the optical engine's processing and synthesis of real-time data: S1: The wired transmission module 2 collects the end pose and state data calculated by the first-view motion calculation module, and the AR data processing module generates an image data frame containing target coordinate system, text, and icon graphic elements based on the data. S2: The image data is sent to the optical engine of the AR glasses, and the display module of the optical engine generates original light carrying virtual image information based on the image data; S3: The original light beam enters the waveguide through the input coupler, and is propagated and expanded inside the waveguide through total internal reflection; S4: The light beam, after being guided and expanded by the waveguide, leaves the waveguide through the output coupler and finally combines with the real ambient light passing through the waveguide, and is projected together into the operator's pupil, thus forming a stable and accurate superposition of augmented reality information in the real field of vision.

[0024] Furthermore, the AR glasses are configured such that, after the first-view motion calculation module calculates the pose relationship, the AR glasses, based on the calculated pose relationship between the AR glasses and the ring light source target, render a three-dimensional coordinate axis with directional arrows, with the center of the ring light source target as the origin, to provide the operator with an intuitive spatial motion reference; simultaneously, in the side area of ​​the field of view, the real-time status parameters of the system are overlaid and displayed in the form of text or graphic panels, including the relative pose of the end effector with respect to the AR glasses coordinate system, real-time motion speed, and system operation status such as "ready", "in motion", "emergency stop" and "cooperative calibration". Beneficial effects

[0025] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: 1. It realizes intuitive spatial mapping teleoperation. Through AR glasses, a virtual coordinate system fixed to the ring light source target is presented in the operator's field of vision. The direction seen is the control direction. It fundamentally solves the problem of unintuitive spatial mapping between commands and end actions in traditional teleoperation, and significantly improves control intuition and positioning accuracy.

[0026] 2. A non-contact remote safe operation mode has been constructed, in which operators can obtain an immersive first-person perspective through AR glasses in a safe area without the need for personnel to enter the dangerous area to assist in the operation. This achieves physical isolation between personnel and dangerous sources such as heavy robotic arms and flowing concrete, greatly reducing safety risks.

[0027] 3. By integrating information from multiple sensors to achieve stable closed-loop control, the system combines visual target positioning, dual electronic compass orientation perception, and joint encoder data to achieve high-precision end-effector pose calculation and status feedback. Information is also intuitively overlaid through an AR interface, enhancing the anti-interference capability and control stability in complex construction environments. Attached Figure Description

[0028] Figure 1 This is a system configuration diagram of the present invention; Figure 2 This is a system flowchart of the present invention; Figure 3 This is a data transmission diagram of the present invention; Figure 4 Diagram showing the installation location of the ring light source target; Figure 5 This is a schematic diagram of the coordinate transformation relationship of the present invention, wherein {B} is the base coordinate system, {S} is the world coordinate system, {J1} is the first joint coordinate system, {J2} is the second joint coordinate system, {J3} is the third joint coordinate system, {J4} is the fourth joint coordinate system, {T} is the tool coordinate system, {G} is the target coordinate system, {A} is the target coordinate system, {C} is the camera coordinate system, and {D} is the virtual coordinate system; Figure 6 This is a system operation logic flowchart of Embodiment 1 of the present invention; Figure 7 This is a system operation logic flowchart of Embodiment 2 of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and key points of this invention clearer, the invention will be further described below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the invention.

[0030] In this invention, the data acquisition module of the system mainly includes three types of functional units: a sensor unit for sensing environmental and status information, a marker unit as a visual positioning reference, and a display output unit for information presentation, and is not limited to the sensor unit.

[0031] like Figure 1 The diagram shown illustrates the overall structure of the first-view AR-enhanced teleoperation system for concrete construction described in this invention. The system consists of two main parts: a concrete placing machine subsystem and an enhanced teleoperation subsystem, connected by a data transmission module.

[0032] The concrete placing boom subsystem is deployed at the construction site, and its core components include: Concrete placing boom body (1): The main structure of the robot that performs the concrete placing task, including: Motion control module: connected to the concrete placing boom body (1), used to adjust the angle of each joint of the robotic arm and the posture of the end placing tube according to the received drive signal; Data acquisition module: integrates an electronic compass 1 to provide global absolute orientation data of the concrete placing boom base, a joint encoder to collect the angle of each joint, and a ring light source target set on the end flexible tube as a visual positioning mark; Wireless data transmission module 1: used to perform bidirectional data communication with the enhanced teleoperation subsystem; Power supply module 1: used to supply power to the entire concrete placing boom subsystem.

[0033] The enhanced teleoperation subsystem is worn and operated by the operator, and its core components include: Remote control handle (2): includes directional buttons, safety buttons and wireless data transmission module 2, used to send remote operation commands defined in the first-view coordinate system to the system; Safety helmet (3): internally integrates first-view motion calculation module, wireless data transmission module 3 and wired data transmission module 1 of safety helmet (3); AR glasses (4): includes a miniature camera for capturing environmental images, an optical engine for displaying augmented reality information, an electronic compass 2 for providing global absolute orientation data of AR glasses (4) and an AR data processing module.

[0034] like Figure 2 The diagram shown illustrates the core operation flow of the system of this invention. The system's workflow mainly includes the following steps: S1: Start the concrete placing machine subsystem. The motion control module controls the robotic arm to extend to the ready position, and at the same time activates the electronic compass 1, the ring light source target, the joint encoder and the wireless data transmission module 1.

[0035] S2: The operator wears the enhanced remote operation subsystem. The miniature camera of the AR glasses (4) begins to capture an image containing a target with a ring light source; at the same time, the safety helmet (3) begins to receive base global absolute orientation data and joint angle data from the wireless data transmission module 1 of the concrete placing boom subsystem via the wireless data transmission module 3;

[0036] S3: The first-person motion calculation module synchronously processes visual images, dual electronic compass orientation data, and joint angle data to calculate the pose of the tool coordinate system at the end of the robotic arm relative to the camera coordinate system, and establishes the orientation alignment relationship between the camera coordinate system and the cloth laying machine base coordinate system.

[0037] S4: The operator issues motion commands via the remote control handle (2). Based on the orientation alignment relationship, the first-view motion calculation module maps the commands in the first-view coordinate system to the target pose in the tool coordinate system under the base coordinate system, and then solves the drive signals and end-effector attitude control signals of each joint through robot inverse kinematics;

[0038] S5: AR glasses (4) Based on the calculated pose of the tool coordinate system relative to the camera coordinate system, with the center of the ring light source target as the origin, a virtual three-dimensional coordinate system is rendered in real time in the operator's field of vision, and system status information is superimposed and displayed.

[0039] S6: Control commands are sent to the motion control module of the concrete placing boom subsystem via wireless network to drive the robotic arm and end effector to move; at the same time, the robot status is fed back in real time, and the calculation and display are updated to form a closed loop of "perception-calculation-control-feedback".

[0040] like Figure 3 The diagram shows the main data transmission path within the system of this invention. After the system starts, the wireless data transmission module 1 sends the electronic compass data and joint encoder data of the fabric laying machine to the safety helmet (3) via the wireless transmission module 3. Then, based on the remote operation command sent by the wireless data transmission module 2, it generates the robotic arm joint drive signal and the fabric tube attitude control signal to be sent to the wireless data transmission module 1. The wired data transmission module 1 of the safety helmet (3) receives the image information and electronic compass data from the wired data transmission module 2 of the AR glasses (4) and forwards them to the wireless data transmission module 3. Based on the first-view motion calculation module inside the safety helmet, the operation status information is output and then sent to the wired data transmission module 2 of the AR glasses (4) via the wireless data transmission module 3. At the same time, the rendering information calculated by the AR processing module is sent to the wired data transmission module 2 of the AR glasses (4). After that, the AR glasses display the content on the display module based on the rendering information and the operation status information.

[0041] like Figure 4The diagram illustrates the coordinate system transformation relationships of this invention. The coordinate systems include the world coordinate system {S}, the fabric placing machine base coordinate system {B}, and the coordinate systems of each joint of the robotic arm {...}. The coordinate system includes the target coordinate system {A}, the tool coordinate system {T} at the end of the robotic arm, the camera coordinate system {C}, and the virtual coordinate system {D} with the center of the ring light source target as the origin (i is the joint number, i≥1).

[0042] like Figure 5 The diagram shown is a schematic representation of the structure, installation position, and dimensional relationship of the ring light source target in the system of the present invention.

[0043] The ring-shaped light source target (5) is the core component for the system to achieve high-precision visual positioning. The target is composed of a series of high-brightness light sources arranged uniformly and closely along the circumference to form a complete luminous ring. To securely install it on the periphery of the flexible concrete fabric hose (7) at the end, an openable ring-shaped mounting bracket is used for support and fixation.

[0044] The bracket (6) adopts a split design, specifically consisting of two symmetrical semi-circular shells connected by screws. This design allows the entire target assembly to be directly installed or disassembled on-site without moving or disassembling the hose (7) itself, greatly improving the convenience of installation and maintenance. During installation, the inner surface of the target (5) is firmly fixed to the outer periphery of the mounting bracket (6).

[0045] During installation, two key spatial relationships must be ensured: first, the lower surface of the target must be perpendicular to the axis of the hose to provide a stable, directly facing observation plane for the vision system; second, the target must be suspended at a certain distance above the discharge port to avoid concrete splashing and ensure a good field of view.

[0046] like Figure 6 The diagram shows the system flow of the present invention under standard operating conditions where the safety helmet (3) and AR glasses (4) are worn accurately and the miniature camera completely captures images of the ring-shaped light source target. S1: The concrete placing boom subsystem performs a power-on self-test. The motion control module controls the robotic arm to extend to the ready-to-work position, activating the electronic compass, ring light target, and wireless data transmission module 1. The global absolute orientation data of the placing boom base and the angle data of each joint begin to be continuously transmitted through the wireless data transmission module 1.

[0047] S2: The operator wears the enhanced remote operation subsystem. The wireless data transmission module 3 inside the safety helmet (3) begins to receive the global absolute orientation data of the base of the concrete placing machine subsystem, the global absolute orientation angle α1 of the placing machine base, and the global absolute orientation angle α2 and joint angle data of the AR glasses (4); at the same time, the AR glasses (4) are powered on, and its built-in miniature camera begins to continuously capture the scene image of the ring light source target, and its built-in electronic compass begins to generate the global absolute orientation data of the AR glasses (4);

[0048] S3: The first-person motion calculation module performs calculations based on data synchronously received from three channels by the wireless data transmission module 3. The wired data transmission module 1 of the safety helmet (3) acquires the ring light source target image collected by the miniature camera and the global absolute orientation data of the AR glasses (4); The wireless data transmission module 1 acquires the global absolute orientation data of the base and the joint angle data from the concrete placing boom subsystem. The remote operation command is received from the remote control handle (2) via the wireless data transmission module 2; S4: The first-person motion solution module performs the following core calculations based on the above multi-source information: Define the translation operator from joint i-1 to joint i as follows: The rotation operator is The transformation operator is ;

[0049] Target coordinate system pose calculation: Based on the ring light source target image, camera intrinsic parameters and the known size of the ring light source target, combined with the global absolute orientation data of the AR glasses (4), the pose of the tool coordinate system relative to the AR glasses (4) coordinate system is calculated. Motion pose calculation: The translation operator is calculated based on the coordinate system {B} of the placing boom base; the relative position translation operator of the tool end coordinate system {T} origin is used. It is obtained by successively multiplying the translation operators of each joint and the tool end, and its expression is:

[0050] Rotation operator calculation: based on global absolute orientation angle , The rotation operator of the tool end coordinate system {T} relative to the base coordinate system {B} Its expression is:

[0051] Where c α = cos α represents the cosine operation, and s α = sin α represents the sine operation; Based on the joint angle data obtained from S3 and the global absolute orientation data of the base, the first-view motion calculation module calculates the theoretical pose of the tool coordinate system relative to the base coordinate system through the robot's forward kinematics. S5: The AR data processing module receives the pose of the tool coordinate system relative to the AR glasses (4) coordinate system calculated by the first-view motion calculation module. The AR glasses (4) renders the virtual coordinate system stably and superimposes it on the operator's real field of vision with the center of the ring light source target as the origin. At the same time, the relative pose, motion speed and "ready" status of the end effector are updated on the graphics panel on the side of the field of vision. S6: The operator observes the coordinate system in the AR interface, presses the direction button corresponding to the positive X-axis on the remote control handle, and sends the further instructions defined in the first-view coordinate system through the wireless data transmission module 2. The first-view motion calculation module calculates the position of the tool coordinate system T' relative to the AR glasses (4) coordinate system based on the orientation alignment relationship established in step S3, where a is the displacement of the original coordinate system T along the desired direction, and the original T coordinate system becomes the coordinate system T' through a; Map the first-view command to the target position in the tool coordinate system under the cloth placing machine base coordinate system, according to We calculate the angle parameters of each joint of the robotic arm after movement, and then calculate the drive signals of each joint and the attitude control signals of the end effector. The expression of the end effector operator after movement is as follows:

[0052] S7: Control commands are sent to the robot motion control module via the wireless data transmission module 3 to drive the robotic arm and end hose to move precisely. During the movement, steps S2 to S5 are executed in a high-speed loop: the real-time status of the concrete placing machine subsystem is fed back to the remote operation terminal, the first-view motion calculation module updates the calculation results accordingly, and the AR glasses (4) update the coordinate system and status information display in real time, thus forming a precise operation closed loop of "perception-calculation-control-feedback".

[0053] like Figure 7 As shown, this invention demonstrates how, in the case where the operator's head tilt is too large, causing severe perspective distortion of the visual marker and the miniature camera to fully capture the ring-shaped light source target system, the robustness of pose calculation and the continuity of system operation are ensured by using proactive human-computer interaction guidance as the core strategy.

[0054] S1: The operator's head tilt causes perspective distortion in the ring-shaped target image captured by the miniature camera, transforming its regular elongated outline into an irregular trapezoid in the image. The AR data processing module calculates the aspect ratio deviation based on the image information acquired by the miniature camera, and this geometric deformation index serves as the basis for evaluating the deformation threshold.

[0055] S2: When the geometric deformation index exceeds the preset deformation threshold, the AR data processing module determines that the current view is unreliable and non-ideal, and generates a visual prompt instruction.

[0056] S3: The visual cue is sent to the optical engine of the AR glasses (4), rendering the clear text "Collaborative Calibration" and an intuitive warning icon in the upper left corner of the operator's field of vision. Based on this cue, the operator actively adjusts their head posture to reduce the tilt angle between themselves and the ring light source target.

[0057] S4: The system continuously monitors visual quality assessment indicators. When the operator adjusts the head posture to the optimal range, so that the geometric deformation index returns to within an acceptable threshold, the AR data processing module determines that the viewing angle has been optimized. The AR glasses (4) then clear the original prompt information and generate a "ready" status prompt, switching to S2 of Example 1.

[0058] The processes described in S1 to S4 constitute a high-speed "detection-prompt-correction-recovery" monitoring closed loop. This closed loop executes in parallel with the main control flow (S2 to S6) described in Example 1, continuously ensuring the observation quality from the first-person perspective, thereby ensuring the intuitive spatial mapping teleoperation of the entire system.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A first-view enhanced teleoperation system for concrete construction, characterized in that, This includes a concrete placing boom subsystem and an enhanced remote operation subsystem; The concrete placing boom subsystem includes a power module 1, a placing boom body (1), a motion control module, a data acquisition module, and a wireless data transmission module 1; Power module 1 is used to supply power to the concrete placing boom subsystem; placing boom body (1) is used to perform actual operation tasks; The motion control module is connected to the fabric laying machine body (1) and is used to adjust the joint angle of the robotic arm and the posture of the fabric tube according to the drive signal; Wireless data transmission module 1 is used for data transmission between the two subsystems; The enhanced teleoperation subsystem includes a teleoperation handle (2), a safety helmet (3), and AR glasses (4); the safety helmet (3) is used to integrate the processing module and the data transmission module; the AR glasses (4) are used to realize visual computing and three-dimensional rendering.

2. The system according to claim 1, characterized in that, The data acquisition module includes: an electronic compass 1, a joint encoder, and a ring light source target. The electronic compass 1 is used to provide global absolute orientation data of the fabric laying machine base; the ring light source target is set on the end hose for visual positioning marking.

3. The system according to claim 1, characterized in that, The remote control handle (2) includes: a power module 2, directional buttons, a safety button, and a wireless data transmission module 2; the power module 2 is used to power the remote control handle (2); the directional buttons are used to output remote operation commands on the XYZ axes in the first viewpoint coordinate system; the safety button is used to ensure the output of any command with the operator's permission; the wireless data transmission module 2 is used to send the remote operation commands to the safety helmet (3).

4. The system according to claim 1, characterized in that, The safety helmet (3) includes: a power module 3, a power supply module, a first-view motion calculation module, a wireless data transmission module 3, and a wired data transmission module 1 for the safety helmet (3); the power module 3 is used to supply power to the integrated components inside the safety helmet (3); the power supply module is used to transmit the power from the power module 3 to the AR glasses (4); the first-view motion calculation module is used to convert the relative pose between coordinate systems into the target pose in the base coordinate system, and then calculate the drive signals of each joint and the attitude control signals of the end effector based on the target pose; the wireless data transmission module 3 is used to receive the status information of the concrete placing machine subsystem and send the calculated motion commands to the concrete placing machine subsystem; the wired data transmission module 1 for the safety helmet (3) is used to send rendering information to the AR glasses.

5. The system according to claim 1, characterized in that, The AR glasses (4) include: a power supply module, a miniature camera, an optical engine, an electronic compass 2, an AR data processing module, and a wired data transmission module 2 for the AR glasses (4); the miniature camera is used to capture environmental images; the optical engine is used to display augmented reality information, display module, waveguide and coupler, and the display module generates original virtual images; the waveguide uses the principle of total internal reflection to send images to the human eye; the coupler guides light into and out of the waveguide to complete the input and output of images; the electronic compass 2 is used to provide global absolute orientation data for the AR glasses (4); the AR data processing module is used to receive the relative pose results and system status data calculated by the first-view motion calculation module, and generate image rendering frames containing augmented reality information such as target coordinate system, text labels, and status icons; the wired data transmission module 2 for the AR glasses (4) is used to receive rendering information from the safety helmet (3).

6. The system according to claim 1, characterized in that, The system implementation logic includes the following steps: S1: Start the system, prepare the concrete placing machine subsystem, control the robotic arm to unfold using the motion control module, and activate the electronic compasses 1 and 2, the ring light source target and the data transmission module at the same time. S2: The operator wears the enhanced remote operation subsystem, and the miniature camera inside the AR glasses (4) begins to capture the image of the ring light source target; the wireless data transmission module 3 inside the safety helmet (3) begins to receive the base global absolute orientation data and joint angle data from the wireless transmission module 1 of the concrete placing machine subsystem; S3: The first-person motion resolution module synchronously performs the following processing: Based on the ring light source target image, the pre-stored camera intrinsic parameters and the known size of the ring light source target, and combined with the global absolute orientation data of the AR glasses (4), the pose of the tool coordinate system relative to the camera coordinate system is calculated; based on the joint angle data collected by the joint encoder, the theoretical pose of the tool coordinate system relative to the base coordinate system is calculated through the robot's forward kinematics; based on the global absolute orientation data of the AR glasses (4) and the fabric machine base, the orientation alignment relationship between the camera coordinate system and the fabric machine base coordinate system is established. S4: The operator issues a motion command defined in the first-view coordinate system through the remote control handle; the first-view motion calculation module calculates the orientation alignment relationship and the pose of the tool coordinate system relative to the camera coordinate system based on step S3, and maps the motion command to the target pose of the tool coordinate system in the cloth laying machine base coordinate system. S5: Based on the target pose, the drive signals of each joint and the attitude control signals of the end effector are calculated by solving the robot inverse kinematics, and the signals are sent to the motion control module of the concrete placing machine subsystem. S6: The AR glasses (4) based on the pose of the tool coordinate system relative to the camera coordinate system calculated in step S3, render a virtual coordinate system with the center of the ring light source target as the origin and aligned with the tool coordinate system in the operator's field of vision, and overlay and display system status information in real time. S7: The motion control module drives the robotic arm and end effector to move, completing actions consistent with the operator's first-person perspective commands, forming a closed-loop control.

7. The ring-shaped light source target according to claim 2, characterized in that, The ring light source target (5) is composed of high-brightness light sources that are uniformly and closely arranged along the circumference, and is fixed around the periphery of the end hose (7) by an openable ring mounting bracket (6); the mounting bracket is composed of two symmetrical semi-circular shells connected by screws, and the inner surface of the target is fixed to the outer periphery of the bracket; after installation, the lower surface of the target is perpendicular to the axial direction of the hose, and its whole is located above the discharge port at a certain height.

8. The safety helmet (3) according to claim 4, characterized in that, The first-view motion calculation module is configured as follows: S1: Receive the ring light source target image captured by the miniature camera, the global absolute orientation data provided by the AR glasses (4) and the electronic compass of the fabric machine base respectively, the motion command issued by the remote control handle and the joint angle data collected by the joint encoder; S2: Based on the target image of the ring light source, the known size and the camera intrinsic parameters of the micro camera, and combined with the global absolute orientation data of the AR glasses (4), the pose of the tool coordinate system relative to the AR glasses (4) coordinate system is calculated; S3: Based on the global absolute orientation data of the AR glasses (4) and the fabric machine base, establish the orientation alignment relationship between their coordinate systems; S4: Based on the orientation alignment relationship, the motion command issued by the remote control handle in the first view coordinate system is mapped to the target pose in the tool coordinate system in the cloth laying machine base coordinate system; S5: Based on the target pose, the drive signals of each joint and the attitude control signals of the end effector are calculated by solving the robot inverse kinematics and sent to the motion control module.

9. The AR glasses (4) according to claim 5, characterized in that, The AR glasses (4) are configured as follows: Based on the relative pose of the tool coordinate system and the camera coordinate system calculated by the first perspective motion calculation module, a virtual three-dimensional coordinate system is rendered in the display module with the center of the ring light source target as the origin. Meanwhile, the current operating status, including "Ready", "In Motion", "Emergency Stop" or "Collaborative Calibration", is superimposed on the left side of the display module, and the relative pose and real-time motion speed of the end effector are superimposed on the right side of the display module.

Images