A method for detecting the pose of a segmented primary mirror of an astronomical telescope and a method and system for grabbing the segmented primary mirror

Abstract

This invention discloses a method and system for detecting the pose of a sub-mirror in a modular astronomical telescope, belonging to the field of visual inspection application technology. The pose detection method includes: acquiring a reference point cloud and a candidate region point cloud before and after lifting the target sub-mirror; performing a difference comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to solve for the initial detection pose after lifting, including the initial center coordinates and the initial rotation matrix; estimating the observation center coordinates of the candidate region point cloud; comparing whether the center deviation and attitude deviation are both within the allowable range; if so, outputting the initial detection pose; otherwise, fusing the observation center coordinates and the initial center coordinates, and reconstructing the final pose output. This invention can accurately estimate the three-dimensional center position and attitude of the target sub-mirror without increasing or minimizing modifications to the telescope back structure, thereby improving the automation level, work efficiency, and operational safety of modular primary mirror maintenance.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection application technology, specifically relating to a method for detecting the pose of a sub-mirror in a spliced ​​astronomical telescope, as well as a method and system for grasping the sub-mirror. Background Technology

[0002] Large-aperture astronomical telescopes are critical infrastructure for precise deep-space observations. With increasing demands for higher angular resolution and greater light-gathering power, the continuous increase in primary mirror aperture has become a trend. However, the overall ultra-large mirror surface is limited by engineering bottlenecks in manufacturing, transportation, and assembly. Therefore, modular primary mirrors combined with active optical control have become the mainstream approach: decomposing the primary mirror into independently installable and calibrated sub-mirror units, improving the feasibility and economy of aperture expansion while ensuring surface accuracy. Practices exemplified by LAMOST (Large Sky Area Multi-Object Fiber Spectroscopic Telescope) have validated its value in large field-of-view and high-efficiency observations.

[0003] During the long-term operation of a modular telescope, the primary mirror requires periodic maintenance, including routine cleaning, recoating after reflective film aging, surface orientation adjustment, and replacement of faulty sub-mirrors. Due to the large number of sub-mirrors, their substantial size and weight, and their high value, coupled with the stringent requirements for assembly and repeatability of the modular primary mirror, maintenance often involves the disassembly, handling, and repositioning of sub-mirrors. To meet maintenance space and safety requirements, sub-mirrors typically need to be slightly lifted along the normal direction using a support mechanism or lifting device to facilitate subsequent manual or robotic grasping, hoisting, and replacement operations. Inaccurate lifting status recognition or spatial orientation assessment can easily lead to grasping and alignment failures, reduced maintenance efficiency, and even safety risks such as sub-mirror surface damage and support structure collisions. Therefore, reliable detection of the slight lifting status of individual sub-mirrors and high-precision acquisition of their center position and attitude parameters during maintenance are key technical aspects supporting automated maintenance and safe operation of modular primary mirrors.

[0004] In existing technologies, to determine whether the sub-mirror has been raised to its correct position and its approximate travel distance, sensors such as encoders, limit triggers, and proximity switches are often installed on the back of the sub-mirror or its supporting structure, or rearward / lateral cameras are used to monitor local mechanisms. These solutions typically require adding sensors and wiring in the narrow space behind the mirror, resulting in high system modification and maintenance costs. Furthermore, most solutions can only provide single-dimensional displacement or discrete state quantities, making it difficult to directly obtain complete spatial attitude information of the raised sub-mirror. The robustness and accuracy of state determination remain limited under conditions of gaps in the supporting structure, accumulated mechanical errors from long-term service, or local occlusion. Existing sensor / rearward monitoring solutions struggle to simultaneously meet the requirements of high-precision attitude estimation and robot grasping alignment without modifying the mirror back structure. Meanwhile, with the development of 3D vision measurement technology, research has attempted to deploy structured light, ToF sensors, or binocular / multi-view depth cameras in front of the primary mirror to achieve macroscopic surface reconstruction and overall deformation monitoring of the primary mirror through point cloud reconstruction and surface fitting. These methods improve measurement flexibility and deployment convenience without requiring contact with the mirror back. However, most solutions still focus on full mirror shape reconstruction or macroscopic deformation assessment, lacking sufficient adaptability to fine-grained maintenance scenarios. In situations where significant manual markings are lacking, sub-mirror boundaries are susceptible to reflection and occlusion, and the lifting height variation is relatively limited, issues such as unstable candidate sub-mirror recognition, fluctuations in edge region detection, and insufficient pose calculation accuracy may arise. These problems make it difficult to directly meet the comprehensive requirements of real-time performance, stability, and high accuracy for automatic sub-mirror capture during maintenance. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a method and system for detecting the position and attitude of sub-mirrors in a modular astronomical telescope, as well as a method and system for grasping sub-mirrors. This method can accurately estimate the three-dimensional center position and attitude of the target sub-mirror without increasing or minimizing modifications to the back structure of the telescope, thereby improving the automation level, work efficiency, and operational safety of the maintenance of modular primary mirrors.

[0006] This invention provides the following technical solution:

[0007] In a first aspect, a method for detecting the pose of a sub-mirror in a modular astronomical telescope is provided, wherein the astronomical telescope is equipped with several sub-mirrors, and the target sub-mirror is lifted individually during maintenance, including: Step S1: Before lifting, acquire the reference point cloud and prior data of the target sub-mirror, including: prior center coordinates. Reference normal vector, prior profile parameters, and reference edge direction vector; Step S2: After lifting, obtain the current point cloud of the target sub-mirror, and perform spatial clipping on the current point cloud based on prior data to obtain the candidate region point cloud of the target sub-mirror, and estimate the current normal vector and current reference edge direction vector of the candidate region point cloud. Step S3: Perform a difference comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combine the prior data, the current normal vector, and the current reference edge direction vector to solve the initial detection pose of the target sub-mirror after lifting. Including initial center coordinates and the initial rotation matrix ; Step S4: Estimate the observation center coordinates of the candidate region point cloud Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are fused. and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector, the final pose of the target sub-mirror is reconstructed and output by combining it with the current reference edge direction vector.

[0008] Optionally, the reference point cloud in step S1 and the current point cloud in step S2 are both located in the telescope reference coordinate system with the sub-mirror mounting base as the geometric reference.

[0009] Optionally, in step S2, the point set of the candidate region point cloud is represented as: ; in, For target sub-mirror Candidate region point cloud, The first candidate region in the point cloud The three-dimensional coordinate vector of a point for The projection of the reference point cloud onto the reference plane. For target sub-mirror The prior contour region. The reference normal vector of the target sub-mirror, with superscript Indicates transpose. This is the preset effective working surface thickness threshold along the reference normal.

[0010] Optionally, in step S3, the step of performing a differential comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction specifically involves: After performing point cloud matching between the reference point cloud and the candidate region point cloud, the height residual along the reference normal for each pair of matched points is calculated, and the median of all height residuals is defined as the lifting height of the target sub-mirror. ; ; ; in, This represents the median. for and The height of the residual, The first point cloud as the reference point cloud The three-dimensional coordinate vector of a point The first candidate region in the point cloud The three-dimensional coordinate vector of a point This represents the number of matching point pairs between the baseline point cloud and the candidate region point cloud. The reference normal vector of the target sub-mirror, with superscript This indicates transpose.

[0011] Optionally, in step S3, the initial detection pose after the target sub-mirror is lifted is determined. : Including initial center coordinates and the initial rotation matrix Specifically: Based on the lifting height of the target sub-mirror, the prior center coordinates are... Updated to initial center coordinates along the reference direction ; ,in, The lifting height of the target sub-mirror. This is the reference normal vector of the target sub-mirror; Calculate the pitch angle of the current normal vector relative to the reference normal vector. and roll angle ; ; ; in, For the current normal vector, As the reference normal vector, , and These are the current normal vectors in the telescope's reference coordinate system. , and Projection on axis , and These are the reference normal vectors in the telescope's reference coordinate system. , and Projection on the axis; Calculate the yaw angle of the current reference edge direction vector relative to the base reference edge direction vector. ; ; in, The reference edge direction vector is used as the reference. This is the current reference edge direction vector; The pitch, roll, and yaw angles are converted into relative rotation matrices between the candidate region point cloud and the reference point cloud according to the following formula. And based on the prior pose of the target sub-mirror, the initial rotation matrix is ​​obtained. ; ; in, This is the small-angle attitude increment vector. Indicates the antisymmetric matrix operator, It is the identity matrix. The prior pose is determined based on the reference normal vector and the reference reference edge direction vector.

[0012] Optionally, in step S4, the coordinates of the observation center of the estimated candidate region point cloud are... The specific formula is: ; in, The first candidate region in the point cloud The three-dimensional coordinate vector of a point The number of points in the point cloud of the candidate region.

[0013] Optionally, in step S4, the coordinates of the fused observation center and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector and combining it with the current reference edge direction vector, the final pose of the target sub-mirror is reconstructed and output, specifically as follows: The coordinates of the observation center are merged using a weighted fusion method. and initial center coordinates The corrected final center coordinates are obtained. ; Final center coordinates As a constraint, the current normal vector is adjusted through rigid body uniformity to obtain the updated normal vector. ; Using the final center coordinates Updated normal vector and the current reference edge direction vector By performing projection and cross product operations, a new local coordinate system is established that is fixedly connected to the target sub-mirror, and the corresponding rotation matrix is ​​output as the corrected attitude matrix of the target sub-mirror. The corrected attitude matrix and final center coordinates The final pose of the target sub-mirror is uniformly represented as... .

[0014] Optionally, it also includes: step S5, performing a confidence score on the final pose of the target sub-mirror; if the score is lower than a set threshold, the final pose obtained in step S4 is not allowed to be output, and the process returns to step S2 to re-acquire the current point cloud; if the score is higher than the set threshold, the final pose obtained in step S4 is allowed to be output. Credibility score of the final pose of the target sub-mirror for: ; in, For the first Weight of each evaluation item The residuals are used for local fitting of the point cloud of the candidate region. The percentage of valid points in the candidate region's point cloud relative to the current point cloud. This is an index for the consistency of the boundary direction of the target sub-mirrors. The standard deviation of the residuals of all matching point pairs in the baseline point cloud and the candidate region point cloud; and They are respectively and The normalized scaling parameter, , , and Evaluation items , , and The corresponding weights.

[0015] Secondly, a method for retrieving sub-mirrors from a modular astronomical telescope is provided, including: Receive the target sub-scope replacement command and drive the target sub-scope to be raised into position; Perform the method for detecting the pose of a segmented telescope sub-mirror as described in any one of the first aspects to obtain the final pose of the output target sub-mirror; Perform hand-eye calibration on the grasping robot and map the final pose of the output target sub-mirror to the end target pose of the grasping robot. This drives the grabbing robot to grab the target sub-mirror.

[0016] Thirdly, a modular astronomical telescope sub-mirror grasping system is provided, including: Sub-mirror support structure, used to support all sub-mirrors configured in the astronomical telescope and to lift the target sub-mirror after receiving a lifting command; A depth camera is used to acquire the initial reference point cloud and the initial current point cloud of the target sub-mirror before and after lifting. The grasping robot is used to grasp the target sub-mirror according to grasping instructions; The prior data acquisition module is used to perform hand-eye calibration on the grasping robot, and to acquire the transformation matrix of the depth camera to the telescope reference coordinate system. Based on the transformation matrix, the initial reference point cloud is transformed to obtain the reference point cloud and then the prior data of the target sub-mirror is acquired. The vision processing and control system includes a point cloud processing module, an initial pose evaluation module, a final pose determination module, an end-target pose determination module, and a control module. The point cloud processing module is used to perform coordinate transformation on the initial current point cloud according to the transformation matrix of the camera coordinates to the telescope reference coordinate system to obtain the current point cloud, and to perform spatial clipping on the current point cloud according to the prior data to obtain the candidate region point cloud of the target sub-mirror, and to estimate the current normal vector and the current reference edge direction vector of the candidate region point cloud. The initial pose evaluation module is used to perform differential comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combined with prior data, the current normal vector, and the current reference edge direction vector, it solves the initial detection pose of the target sub-mirror after lifting. : Including initial center coordinates and the initial rotation matrix ; The final pose determination module is used to estimate the coordinates of the observation center of the candidate region point cloud. Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are weighted and fused. and initial center coordinates The corrected final center coordinates are obtained. The final pose of the target sub-mirror is reconstructed and output by combining the current normal vector and the current reference edge direction vector. The end-effector pose determination module is used to map the final pose of the output target sub-mirror to the end-effector pose of the grasping robot based on the hand-eye calibration results of the grasping robot. ; The control module is used to receive target sub-mirror replacement commands, generate lifting commands for the target sub-mirror, and determine the target position and orientation based on the end-effector's pose. Generate fetching instructions.

[0017] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention introduces forward depth vision detection and geometric prior constraints of sub-mirror array into the maintenance scenario of the sub-mirror of the spliced ​​astronomical telescope, which solves the problem in the prior art that it relies on the back sensor or human experience and is difficult to complete the lifting status determination and accurate pose calculation without modifying the back structure. This enables the sub-mirror maintenance process to realize lifting detection and pose output under a unified coordinate system, effectively improving the level of maintenance automation, the success rate of subsequent grabbing, and the safety of operation. Furthermore, this invention addresses the issues of small attitude changes and susceptibility to noise during micro-lifting by proposing a differential small-angle solution based on prior data to estimate the initial rotation matrix, thereby improving attitude angle accuracy and stability. Simultaneously, considering the coupling effect between attitude deviation and center offset in micro-lifting operations, this invention introduces center offset coupling correction when the initial detected pose offset or attitude deviation is large. This involves fusing the observed center coordinates and the initial center coordinates to obtain the corrected final center coordinates. After updating the current normal vector and combining it with the current reference edge direction vector, the final pose of the target sub-mirror is reconstructed and output. This approach balances real-time performance and sub-mirror pose detection accuracy, meeting the high-precision attitude requirements of the subsequent grasping process, and ultimately improving the automation level, operational efficiency, and operational safety of the spliced ​​primary mirror maintenance operation.

[0018] (2) The present invention introduces a confidence score for the final pose, which can comprehensively evaluate the fitting residual, effective point quality and boundary direction consistency, etc. When the confidence is insufficient, it automatically triggers resampling to ensure that the output is a high confidence pose that can be used for robot grasping, thereby improving the safety and success rate of the operation. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method for detecting the position and orientation of the sub-mirrors of the spliced ​​astronomical telescope according to the present invention; Figure 2 This is a maintenance structure diagram of the astronomical telescope to which this invention is addressed; Figure 3 This is a schematic diagram of the multi-coordinate system transformation of the present invention; Figure 4 This is a schematic diagram of the lifting height estimation of the present invention; Figure 5 This is a schematic diagram illustrating the calculation of pitch and roll angles according to the present invention; Figure 6 This is a schematic diagram of the yaw angle calculation of the present invention. Figure 6 (a) in the diagram is a schematic diagram showing the position of the reference edge direction vector of the target sub-mirror. Figure 6 (b) in the figure is a schematic diagram showing the position of the current reference edge direction vector of the target sub-mirror relative to the reference reference edge direction vector after lifting; Figure 7This is a schematic diagram of the center offset coupling correction process of the present invention; Figure 8 This is a flowchart illustrating the specific implementation of the method for grasping the sub-mirrors of the spliced ​​astronomical telescope of the present invention. Figure 9 This is a structural block diagram of the spliced ​​astronomical telescope sub-mirror grasping system of the present invention; Figure 10 This is a flowchart of the multi-coordinate unification and prior data acquisition in Embodiment 3 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be used to limit the scope of protection of the present invention. It should be noted that the term "comprising" and any variations thereof in the specification, claims and the above-mentioned drawings of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or devices.

[0021] Example 1: The astronomical telescope addressed in this application is equipped with several sub-mirrors, such as... Figure 2 As shown, several sub-mirrors form a sub-mirror array, which is supported by a sub-mirror support structure. When maintenance of the astronomical telescope is required, the target sub-mirror can be lifted individually through the sub-mirror support structure. During the pose detection of the sub-mirrors, a depth camera capable of acquiring the depth information of the target sub-mirrors is required. When grasping the sub-mirrors, an automatic grasping robot is required. The structures of the sub-mirror support structure and the grasping robot can refer to existing technologies.

[0022] like Figure 1 As shown, a method for detecting the pose of sub-mirrors in a modular astronomical telescope includes the following steps: Step S1: Before lifting, acquire the reference point cloud and prior data of the target sub-mirror, including: prior center coordinates. Reference normal vector, prior profile parameters, and reference edge direction vector.

[0023] Step S2: After lifting, obtain the current point cloud of the target sub-mirror, and perform spatial clipping on the current point cloud based on prior data to obtain the candidate region point cloud of the target sub-mirror, and estimate the current normal vector and current reference edge direction vector of the candidate region point cloud. Step S3: Perform a difference comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combine the prior data, the current normal vector, and the current reference edge direction vector to solve the initial detection pose of the target sub-mirror after lifting. : Including initial center coordinates and the initial rotation matrix ; Step S4: Estimate the observation center coordinates of the candidate region point cloud Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are fused. and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector, the final pose of the target sub-mirror is reconstructed and output by combining it with the current reference edge direction vector.

[0024] Before performing step S1, it is necessary to define a coordinate system and calibrate the extrinsic parameters of the depth camera, such as... Figure 10 As shown.

[0025] Define a coordinate system, specifically: establish a telescope reference coordinate system using the sub-mirror mounting base as the geometric reference. ,like Figure 3 As shown, its origin can be set at the sub-mirror array assembly reference point or the geometric center of the sub-mirror array. The axis is in the opposite direction of gravity. The axis is along the structural reference direction selected by the reference plane. The axes are determined by the right-hand rule; a camera coordinate system is established with the optical center of the depth camera as the reference point. .

[0026] The extrinsic parameter calibration of the depth camera involves: placing a calibration target in front of the sub-mirror array or using measurable structural reference features as constraints; acquiring point cloud data from multiple perspectives and extracting target feature points, feature planes, or structural boundary features; and combining this with the target's position in the telescope's reference coordinate system. Given the known geometric position in the image, the camera extrinsic parameters are solved by minimizing the point-to-point or point-to-plane error. A robust estimation strategy is adopted to suppress the influence of outliers caused by specular reflection, occlusion, and depth holes, thereby obtaining the transformation matrix that transforms the camera coordinate system to the telescope reference coordinate system.

[0027] Step S1 specifically includes: S11: Take the reference point cloud of the target sub-mirror before lifting.

[0028] The reference point cloud is located in the telescope's reference coordinate system and is obtained by coordinate transformation of the initial reference point cloud acquired by the depth camera. The reference point cloud can be obtained by fusing and denoising multiple frames of depth data.

[0029] S12: Obtain prior data for the target sub-mirror, including: prior center coordinates Reference normal vector, prior profile parameters, and reference edge direction vector.

[0030] Based on the telescope's structural design data and offline measurement results during the assembly and adjustment phase, a unique number was assigned to each sub-mirror in the sub-mirror array, and the prior center coordinates were recorded. The reference normal vector, prior contour parameters, and reference reference edge direction vector are included. The prior contour parameters include: a set of hexagonal vertices, a bounding box, or an equivalent contour model.

[0031] It is worth noting that in step S1 of this implementation, a consistency verification and drift monitoring mechanism can be introduced. Specifically, several unlifted sub-mirrors are selected to collect depth data and convert it to the telescope reference coordinate system. In the middle, the prior center coordinates of the corresponding number The system compares the reference normal vectors and calculates consistency indicators such as center offset, normal angle, and contour alignment error. When the consistency indicators are within the allowable range, the prior data of the target sub-mirror is output and fixed. When the consistency indicators exceed the threshold, recalibration and / or updating of the prior data are triggered, thereby avoiding the use of incorrect reference poses caused by coordinate drift or prior mismatch for grasping.

[0032] Step S2, specifically: S21: After lifting, acquire the initial current point cloud of the target sub-mirror and transform it to the telescope reference coordinate system. In this process, the current point cloud of the target point cloud is obtained.

[0033] S22: Perform preprocessing on the current point cloud, such as outlier removal, downsampling, and local smoothing, and perform spatial clipping based on the prior data of the target sub-mirror to obtain the candidate region point cloud of the target sub-mirror. If necessary, further remove the edge rings near the splicing seams, chamfers, and areas with high incidence of deep voids, and retain only the high-confidence working surface point cloud set as the candidate region point cloud. The spatial clipping method refers to the existing technology.

[0034] The point set representation of the candidate region point cloud is as follows: ; in, For target sub-mirror Candidate region point cloud, The first candidate region in the point cloud The three-dimensional coordinate vector of a point for The projection of the reference point cloud onto the reference plane. For target sub-mirror The prior contour region. The reference normal vector of the target sub-mirror, with superscript Indicates transpose. This is the preset effective working surface thickness threshold along the reference normal.

[0035] S23: Estimate the current normal vector and current reference edge direction vector of the candidate region point cloud.

[0036] The current normal vector of the point cloud in the candidate region is estimated by fitting local surfaces. The current reference edge direction vector of the candidate region point cloud is estimated through edge detection and prior contour parameters. The current normal vector of the target sub-mirror is also calculated. and reference normal vector It is obtained based on the effective working surface of the candidate region point cloud and the reference point cloud.

[0037] To specifically improve the accuracy of attitude angle calculation under micro-lift conditions, this invention employs a differential small-angle calculation strategy from the reference point cloud to the candidate region point cloud to obtain the initial detection pose. Therefore, step S3 specifically includes: S31: After performing point cloud matching between the reference point cloud and the candidate region point cloud, calculate the height residual along the reference normal for each pair of matched points, and define the median of all height residuals as the lifting height of the target sub-mirror. ; ; ; in, This represents the median. for and The height of the residual, The first point cloud as the reference point cloud The three-dimensional coordinate vector of a point The first candidate region in the point cloud The three-dimensional coordinate vector of a point This represents the number of matching point pairs between the baseline point cloud and the candidate region point cloud. The reference normal vector of the target sub-mirror, with superscript This indicates transpose.

[0038] Step S31: Place the reference point cloud and the candidate region point cloud in the telescope reference coordinate system. Differential comparisons are performed, and the height residuals and their statistics are calculated along the reference normal direction to characterize the small displacement changes caused by the lift. Using the median, a robust statistic, can improve the robustness to outliers and local voids.

[0039] S32: Based on the lifting height of the target sub-mirror, the prior center coordinates are... Updated to initial center coordinates along the reference direction : ,in, The lifting height of the target sub-mirror. This is the reference normal vector of the target sub-mirror.

[0040] like Figure 4 As shown, the prior center coordinates of the target sub-mirror and its reference normal vector The reference geometric datum for target sub-mirror pose detection, initial center coordinates and the current normal vector This reflects the actual condition after lifting. Due to the lifting height... In the telescope reference coordinate system Lower edge reference normal (i.e., reference normal vector) The direction is obtained, which can be directly used for the prior center coordinates. Normals are updated, and differential references are provided for subsequent attitude increment calculations.

[0041] S33: Calculate the pitch angle of the current normal vector relative to the reference normal vector. and roll angle ; ; ; in, For the current normal vector, The reference normal vector, superscript Indicates transpose. , and These are the current normal vectors in the telescope's reference coordinate system. , and Projection on axis , and These are the reference normal vectors in the telescope's reference coordinate system. , and Projection on the axis.

[0042] like Figure 5 As shown, the current normal vector Relative to the reference normal vector The deflection can be decomposed into pitch angles within two mutually orthogonal reference profiles. and roll angle By using the current normal vector Relative to the reference normal vector Small-angle deflection is decomposed into pitch angle and roll angle It can obtain an accurate description of the attitude change of the sub-mirror while maintaining the stability of the solution under micro-lifting conditions.

[0043] like Figure 6 As shown, S34: Calculate the yaw angle of the current reference edge direction vector relative to the base reference edge direction vector. : ; in, The reference edge direction vector is used as the reference. This is the current reference edge direction vector.

[0044] like Figure 6 (a) and Figure 6 As shown in (b) above, the reference edge direction vector before lifting is in the top view plane. Using prior central coordinates Established as a reference, the current reference edge direction vector With initial center coordinates Extracted as a reference. This is done by using the current reference edge direction vector. relative to the reference edge direction vector By comparing them within the same reference frame, the included angle can be obtained. As a yaw angle output, it supplements the degree of freedom of rotation around the normal direction, which is difficult to uniquely determine by relying solely on normal information.

[0045] S35: Convert the pitch angle, roll angle, and yaw angle into a relative rotation matrix between the candidate region point cloud and the reference point cloud according to the following formula. And based on the prior pose of the target sub-mirror, the initial rotation matrix is ​​obtained. ; ; in, This is the small-angle attitude increment vector. Indicates the antisymmetric matrix operator, It is the identity matrix. The prior pose is determined based on the reference normal vector and the reference reference edge direction vector.

[0046] Since the attitude change is small in micro-lift scenarios, a small-angle attitude increment vector is used. Approximation can reduce the initial rotation matrix Noise amplification effect when solving directly.

[0047] That is, the initial detection pose Represented as: .

[0048] Step S4 specifically includes: S41: Estimate the observation center coordinates of the candidate region point cloud : ; in, The first candidate region in the point cloud The three-dimensional coordinate vector of a point The number of points in the point cloud of the candidate region.

[0049] S42: Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then center offset coupling correction is performed, the final pose of the target sub-mirror is reconstructed and output.

[0050] Center offset coupling correction is: fused observation center coordinates and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector, the final pose of the target sub-mirror is reconstructed and output by combining it with the current reference edge direction vector.

[0051] Before proceeding to step S42, it is necessary to preset the center deviation threshold and attitude deviation threshold. If the observed center coordinates with initial center coordinates If the deviation is greater than or equal to a preset center deviation threshold, then center offset coupling correction is performed. If it is less than the preset center deviation threshold, then it is determined whether the pose deviation of the candidate region point cloud relative to the reference point cloud is less than the pose deviation threshold. If so, then the initial detection pose is adjusted. The final pose of the target sub-mirror is output; otherwise, a center offset coupling correction is performed. This hierarchical strategy can balance real-time performance and accuracy.

[0052] The pose deviation of the candidate region point cloud relative to the reference point cloud is the relative rotation matrix of the candidate region point cloud relative to the reference point cloud. The corresponding pitch angle, roll angle, and yaw angle.

[0053] like Figure 7As shown, center offset coupling correction is performed, the final pose of the target sub-mirror is reconstructed and output, specifically including: The coordinates of the observation center are merged using a weighted fusion method. and initial center coordinates The corrected final center coordinates are obtained. : ,in, To optimize weighting, when the point cloud quality of the candidate region is high, the weighting of the observation center coordinates should be appropriately increased. The weight of the initial center coordinates is increased when there is significant local occlusion or voids. The weights are adjusted to achieve coupling correction between center offset and attitude estimation.

[0054] Final center coordinates As a constraint, the current normal vector is adjusted through rigid body uniformity to obtain the updated normal vector. Using the final center coordinates Updated normal vector and the current reference edge direction vector By performing projection and cross product operations, a new local coordinate system is established that is fixedly connected to the target sub-mirror, and the corresponding rotation matrix is ​​output as the corrected attitude matrix of the target sub-mirror. The corrected attitude matrix and final center coordinates The final pose of the target sub-mirror is uniformly represented as... , .

[0055] The candidate region point cloud is rotated (i.e., rigidly uniformized) to satisfy the final center coordinates. The constraints are then applied to obtain the updated normal vector. , the current reference edge direction vector Projected onto the updated normal vector Within the orthogonal working plane, the principal direction within the plane is obtained. Then, the normal vector is constructed and updated according to the right-hand rule. In the main direction of the dough Orthogonal auxiliary directions The updated normal vector , in-plane main direction and auxiliary directions The corrected attitude matrix is ​​constructed using column vectors. Through this center offset coupling correction process, the final center coordinates can be... With the corrected attitude matrix It satisfies self-consistency constraints to avoid inconsistencies where the center position corresponds to one geometric state, while the attitude angle corresponds to another geometric state.

[0056] In some other embodiments, a method for detecting the pose of a sub-mirror of a spliced ​​astronomical telescope further includes step S5: scoring the confidence level of the final pose of the target sub-mirror; if the score is lower than a set threshold, the final pose obtained in step S4 is not allowed to be output, and the process returns to step S2 to re-acquire the current point cloud until the score is higher than the set threshold; if the score is higher than the set threshold, the final pose obtained in step S4 is allowed to be output. Credibility score of the final pose of the target sub-mirror for: ; in, For the first Weight of each evaluation item The residuals are used for local fitting of the point cloud of the candidate region. The percentage of valid points in the candidate region's point cloud relative to the current point cloud. This is an index for the consistency of the boundary direction of the target sub-mirrors. The standard deviation of the residuals of all matching point pairs in the baseline point cloud and the candidate region point cloud; and They are respectively and The normalized scaling parameter, , , and Evaluation items , , and The weight.

[0057] Example 2: A method for retrieving sub-mirrors from a modular astronomical telescope is provided, including: Receive the target sub-scope replacement command and drive the target sub-scope to be raised into position; Perform any of the spliced ​​astronomical telescope sub-mirror pose detection methods in Example 1 to obtain the final pose of the output target sub-mirror; Perform hand-eye calibration on the grasping robot and map the final pose of the output target sub-mirror to the end target pose of the grasping robot. This drives the grabbing robot to grab the target sub-mirror.

[0058] Hand-eye calibration of the grasping robot aims to establish the hand-eye relationship between the depth camera and the robot's end effector. This involves constructing a hand-eye calibration sample set, solving the rigid transformation between the depth camera and the robot's end effector using global nonlinear least squares optimization, and thresholding and fixing the hand-eye calibration error. This establishes a complete coordinate transformation chain, ensuring accuracy within the telescope's reference coordinate system. The final pose of the target sub-mirror generated in the process can be stably mapped to the end-effector pose that the robot can execute.

[0059] For a more detailed explanation of the above-mentioned sub-mirror pose detection method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0060] like Figure 8 As shown, the specific implementation steps of a method for retrieving sub-mirrors from a modular astronomical telescope are as follows: Step E1: Receive the target sub-mirror replacement instruction and read the target sub-mirror number and prior data; Step E2: Drive the sub-mirror support structure to lift the target sub-mirror, and after lifting it into place, perform initial current point cloud acquisition, preprocessing and coordinate transformation to obtain the current point cloud of the target sub-mirror; Step E3: Based on prior data, perform spatial cropping on the current point cloud to obtain the candidate region point cloud of the target sub-mirror; Step E4: After matching the reference point cloud and the candidate region point cloud, calculate the height residual of each pair of matching points along the reference normal. If the height residual exceeds the lift detection threshold, a robust statistic is used to define the lift height. Otherwise, return to step E2 to re-acquire the initial current point cloud. Step E5: Compare the candidate region point cloud with the reference point cloud in the same coordinate system using differential calculation to solve for the initial detection pose of the target sub-mirror after lifting. : Including initial center coordinates and the initial rotation matrix ; Step E6: Estimate the observation center coordinates of the candidate region point cloud Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then a center offset coupling correction is performed, specifically: fusing the observation center coordinates. and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector, combine it with the current reference edge direction vector to reconstruct the final pose of the target sub-mirror and output it. Step E7: Calculate the confidence score of the final pose of the target sub-mirror. If it is greater than or equal to the set threshold, the final pose obtained in step E6 is allowed to be output. Otherwise, return to step E2 to re-acquire the initial current point cloud after the target sub-mirror is raised into place, until the confidence score of the final pose calculated after resampling is greater than the set threshold. Step E8: Map the final pose of the output target sub-mirror to the end-effector pose of the grasping robot. ; Step E9: Drive the grasping robot to grasp the target sub-mirror.

[0061] Example 3: like Figure 9 As shown, a modular astronomical telescope sub-mirror grasping system includes: a sub-mirror support structure, a depth camera, a grasping robot, a prior data acquisition module, and a vision processing and control system, as detailed below: The vision processing and control system includes a point cloud processing module, an initial pose evaluation module, a final pose determination module, an end-target pose determination module, and a control module.

[0062] The sub-mirror support structure is used to support all the sub-mirrors configured in the astronomical telescope and to lift the target sub-mirror after receiving a lifting command; the sub-mirror support structure can adopt existing technology.

[0063] A depth camera is used to acquire the initial reference point cloud of the target sub-mirror before lifting and the initial current point cloud of the target sub-mirror after lifting.

[0064] A grasping robot is used to grasp a target sub-mirror according to grasping instructions. The structure of the grasping robot is based on existing technology.

[0065] The prior data acquisition module is used for hand-eye calibration of the grasping robot, and to acquire the transformation matrix from the depth camera's camera coordinates to the telescope's reference coordinate system. Based on this transformation matrix, it performs coordinate transformation on the initial reference point cloud to obtain the reference point cloud, and then acquires the prior data for the target sub-mirror. The specific process is as follows: Figure 10 As shown.

[0066] The vision processing and control system includes a point cloud processing module, an initial pose evaluation module, a final pose determination module, an end-target pose determination module, and a control module.

[0067] The point cloud processing module is used to perform coordinate transformation on the initial current point cloud according to the transformation matrix of the camera coordinate system to the telescope reference coordinate system to obtain the current point cloud, and to perform spatial clipping on the current point cloud according to the prior data to obtain the candidate region point cloud of the target sub-mirror, and to estimate the current normal vector and the current reference edge direction vector of the candidate region point cloud.

[0068] The initial pose evaluation module is used to perform differential comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combined with prior data, the current normal vector, and the current reference edge direction vector, it solves the initial detection pose of the target sub-mirror after lifting. : Including initial center coordinates and the initial rotation matrix .

[0069] The final pose determination module is used to estimate the coordinates of the observation center of the candidate region point cloud. Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are weighted and fused. and initial center coordinates The corrected final center coordinates are obtained. The final pose of the target sub-mirror is reconstructed and output by combining the current normal vector and the current reference edge direction vector.

[0070] The end-effector pose determination module is used to map the final pose of the output target sub-mirror to the end-effector pose of the grasping robot based on the hand-eye calibration results of the grasping robot. .

[0071] The control module is used to receive target sub-mirror replacement commands, generate lifting commands for the target sub-mirror, and determine the target position and orientation based on the end-effector's pose. Generate fetching instructions.

[0072] For more detailed information on the processes of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Those skilled in the art will clearly understand that the technologies in the embodiments of this invention can be implemented using software and necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments of this invention.

[0074] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope, wherein the astronomical telescope is equipped with several sub-mirrors, and the target sub-mirror is lifted individually during maintenance, characterized in that... include: Step S1: Before lifting, acquire the reference point cloud and prior data of the target sub-mirror, including: prior center coordinates. Reference normal vector, prior profile parameters, and reference edge direction vector; Step S2: After lifting, obtain the current point cloud of the target sub-mirror, and perform spatial clipping on the current point cloud based on prior data to obtain the candidate region point cloud of the target sub-mirror, and estimate the current normal vector and current reference edge direction vector of the candidate region point cloud. Step S3: Perform a difference comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combine the prior data, the current normal vector, and the current reference edge direction vector to solve the initial detection pose of the target sub-mirror after lifting. Including initial center coordinates and the initial rotation matrix ; Step S4: Estimate the observation center coordinates of the candidate region point cloud Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are fused. and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector, the final pose of the target sub-mirror is reconstructed and output by combining it with the current reference edge direction vector.

2. The method for detecting the position and attitude of sub-mirrors in a modular astronomical telescope according to claim 1, characterized in that, The reference point cloud in step S1 and the current point cloud in step S2 are both located in the telescope reference coordinate system with the sub-mirror mounting base as the geometric reference.

3. The method for detecting the position and attitude of sub-mirrors in a modular astronomical telescope according to claim 1, characterized in that, In step S2, the point set of the candidate region point cloud is represented as follows: ； in, For target sub-mirror Candidate region point cloud, The first candidate region in the point cloud The three-dimensional coordinate vector of a point for The projection of the reference point cloud onto the reference plane. For target sub-mirror The prior contour region. The reference normal vector of the target sub-mirror, with superscript Indicates transpose. This is the preset effective working surface thickness threshold along the reference normal.

4. The method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope according to claim 1, characterized in that, In step S3, the step of performing a differential comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction specifically involves: After performing point cloud matching between the reference point cloud and the candidate region point cloud, the height residual along the reference normal for each pair of matched points is calculated, and the median of all height residuals is defined as the lifting height of the target sub-mirror. ; ； ； in, This represents the median. for and The height of the residual, The first point cloud as the reference point cloud The three-dimensional coordinate vector of a point The first candidate region in the point cloud The three-dimensional coordinate vector of a point This represents the number of matching point pairs between the baseline point cloud and the candidate region point cloud. The reference normal vector of the target sub-mirror, with superscript This indicates transpose.

5. The method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope according to claim 4, characterized in that, In step S3, the initial detection pose of the target sub-mirror after lifting is solved. : Including initial center coordinates and the initial rotation matrix Specifically: Based on the lifting height of the target sub-mirror, the prior center coordinates are... Updated to initial center coordinates along the reference direction ; ,in, The lifting height of the target sub-mirror. This is the reference normal vector of the target sub-mirror; Calculate the pitch angle of the current normal vector relative to the reference normal vector. and roll angle ; ； ； in, For the current normal vector, As the reference normal vector, , and These are the current normal vectors in the telescope's reference coordinate system. , and Projection on axis , and These are the reference normal vectors in the telescope's reference coordinate system. , and Projection on the axis; Calculate the yaw angle of the current reference edge direction vector relative to the base reference edge direction vector. ; ； in, The reference edge direction vector is used as the reference. This is the current reference edge direction vector; The pitch, roll, and yaw angles are converted into relative rotation matrices between the candidate region point cloud and the reference point cloud according to the following formula. And based on the prior pose of the target sub-mirror, the initial rotation matrix is ​​obtained. ; ； in, This is the small-angle attitude increment vector. Indicates the antisymmetric matrix operator, It is the identity matrix. The prior pose is determined based on the reference normal vector and the reference reference edge direction vector.

6. The method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope according to claim 1, characterized in that, In step S4, the coordinates of the observation center of the estimated candidate region point cloud are... The specific formula is: ； in, The first candidate region in the point cloud The three-dimensional coordinate vector of a point The number of points in the point cloud of the candidate region.

7. The method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope according to claim 1, characterized in that, In step S4, the coordinates of the fused observation center and initial center coordinates The corrected final center coordinates are obtained. After updating the current normal vector and combining it with the current reference edge direction vector, the final pose of the target sub-mirror is reconstructed and output, specifically as follows: The coordinates of the observation center are merged using a weighted fusion method. and initial center coordinates The corrected final center coordinates are obtained. ; Final center coordinates As a constraint, the current normal vector is adjusted through rigid body uniformity to obtain the updated normal vector. ; Using the final center coordinates Updated normal vector and the current reference edge direction vector By performing projection and cross product operations, a new local coordinate system is established that is fixedly connected to the target sub-mirror, and the corresponding rotation matrix is ​​output as the corrected attitude matrix of the target sub-mirror. The corrected attitude matrix and final center coordinates The final pose of the target sub-mirror is uniformly represented as... .

8. The method for detecting the position and attitude of a sub-mirror in a modular astronomical telescope according to claim 1, characterized in that, It also includes: step S5, which scores the confidence level of the final pose of the target sub-mirror. If the score is lower than the set threshold, the final pose obtained in step S4 is not allowed to be output, and the process returns to step S2 to reacquire the current point cloud. If the score is higher than the set threshold, the final pose obtained in step S4 is allowed to be output. Credibility score of the final pose of the target sub-mirror for: ； in, For the first Weight of each evaluation item The residuals are used for local fitting of the point cloud of the candidate region. The percentage of valid points in the candidate region's point cloud relative to the current point cloud. This is an index for the consistency of the boundary direction of the target sub-mirrors. The standard deviation of the residuals of all matching point pairs in the baseline point cloud and the candidate region point cloud; and They are respectively and The normalized scaling parameter, , , and Evaluation items , , and The corresponding weights.

9. A method for grasping sub-mirrors of a modular astronomical telescope, characterized in that, include: Receive the target sub-scope replacement command and drive the target sub-scope to be raised into position; The method for detecting the pose of a sub-mirror of a spliced ​​astronomical telescope according to any one of claims 1-8 is used to obtain the final pose of the target sub-mirror. Perform hand-eye calibration on the grasping robot and map the final pose of the output target sub-mirror to the end target pose of the grasping robot. This drives the grabbing robot to grab the target sub-mirror.

10. A modular astronomical telescope sub-mirror grasping system, characterized in that, include: Sub-mirror support structure, used to support all sub-mirrors configured in the astronomical telescope and to lift the target sub-mirror after receiving a lifting command; A depth camera is used to acquire the initial reference point cloud and the initial current point cloud of the target sub-mirror before and after lifting. The grasping robot is used to grasp the target sub-mirror according to grasping instructions; The prior data acquisition module is used to perform hand-eye calibration on the grasping robot, and to acquire the transformation matrix of the depth camera to the telescope reference coordinate system. Based on the transformation matrix, the initial reference point cloud is transformed to obtain the reference point cloud and then the prior data of the target sub-mirror is acquired. The vision processing and control system includes a point cloud processing module, an initial pose evaluation module, a final pose determination module, an end-target pose determination module, and a control module. The point cloud processing module is used to perform coordinate transformation on the initial current point cloud according to the transformation matrix of the camera coordinates to the telescope reference coordinate system to obtain the current point cloud, and to perform spatial clipping on the current point cloud according to the prior data to obtain the candidate region point cloud of the target sub-mirror, and to estimate the current normal vector and the current reference edge direction vector of the candidate region point cloud. The initial pose evaluation module is used to perform differential comparison between the candidate region point cloud and the reference point cloud in the same coordinate system to obtain the lifting height of the target sub-mirror in the reference normal direction. Combined with prior data, the current normal vector, and the current reference edge direction vector, it solves the initial detection pose of the target sub-mirror after lifting. : Including initial center coordinates and the initial rotation matrix ; The final pose determination module is used to estimate the coordinates of the observation center of the candidate region point cloud. Compare the coordinates of the observation center with initial center coordinates If both the deviation of the candidate region point cloud and the pose deviation of the candidate region point cloud relative to the reference point cloud are within the allowable range, then the initial detection pose will be... If the final pose of the target sub-mirror is not output, then the coordinates of the observation center are weighted and fused. and initial center coordinates The corrected final center coordinates are obtained. The final pose of the target sub-mirror is reconstructed and output by combining the current normal vector and the current reference edge direction vector. The end-effector pose determination module is used to map the final pose of the output target sub-mirror to the end-effector pose of the grasping robot based on the hand-eye calibration results of the grasping robot. ; The control module is used to receive target sub-mirror replacement commands, generate lifting commands for the target sub-mirror, and determine the target position and orientation based on the end-effector's pose. Generate fetching instructions.

Images