Visualization method for global automatic deviation detection of digital array 3D projection

Abstract

The present invention discloses a method for visualizing global automatic deviation detection in digital area array 3D projections, comprising: establishing a positioning model of a robotic arm and a digital area array 3D projection; establishing a relative positional relationship between the target position of the robotic arm and the digital area array 3D projection positioning model and the object being detected, thereby constructing a global measurement field; planning a path for the robotic arm based on the detection range of the object being detected and the global measurement field to obtain a measurement target path; and obtaining a measurement model of the object being detected based on the measurement target path, and performing a comparative analysis with a theoretical model to achieve global automatic deviation visualization detection in 3D projections. Compared to previous detection methods, the present invention is more autonomous, more accurate, and more comprehensive, enabling global visualization of 3D projections, automated deviation detection, improved detection efficiency, and reduced personnel detection time.

Description

Technical Field

[0001] The present invention belongs to the field of 3D projection visualization, relates to qualitative and quantitative evaluation of pits, gaps, and step differences, and in particular to a global automatic deviation detection visualization method for digital array 3D projection. Background Art

[0002] Discontinuous features such as pits or cracks may appear on the surfaces of aircraft, car bodies, precision machinery parts, glass, and sheet metal. Detecting these pits and cracks requires resolutions ranging from microns to millimeters. Digital area array 3D projection scans the surface of the object being inspected, precisely locating pits and gaps on the surface and improving measurement accuracy.

[0003] Currently, 3D laser projection technology is widely used in the inspection field as a high-precision, high-positioning quality manufacturing technology. Because surface pits and assembly deviations of the inspected objects are difficult to measure with high precision, before the application of 3D laser projection positioning technology, traditional assembly positioning technology mostly used manual measuring tools such as vernier calipers, micrometers, and dial indicators to locate and assemble the inspected objects. This meant that the deviation position was determined by the eyes and experience of designers and inspectors, and the accuracy often did not meet the standard. Furthermore, the measurement was time-consuming and, due to the limitations of the operator's proficiency, the deviation data read contained large errors. Later, a handheld gap gun was used to detect gaps. However, the gap gun was unable to locate the entire surface, and manual large-area inspection was too cumbersome. Inspections were prone to deviations, leading to frequent operations and low efficiency.

[0004] Therefore, a global automatic deviation detection and visualization method for digital array 3D projection is urgently needed to address the shortcomings of the existing technology. Summary of the Invention

[0005] The purpose of the present invention is to propose a global automatic deviation detection and visualization method for digital array 3D projection to solve the problem of automatic detection and visualization of object surface information.

[0006] To achieve the above object, the present invention provides a digital array 3D projection global automatic deviation detection visualization method, comprising the following steps:

[0007] Establish a 3D projection positioning model of the robotic arm and digital array;

[0008] Establishing the relative position relationship between the target position of the manipulator and the digital array 3D projection positioning model and the detected object to construct a global measurement field;

[0009] Performing path planning on the robotic arm according to the detection range of the detected object and the global measurement field to obtain a measurement target path;

[0010] Based on the measurement target path, a measurement model of the object to be detected is obtained, and compared and analyzed with the theoretical model to achieve global automatic deviation visualization detection of 3D projection.

[0011] Optionally, establish a 3D projection positioning model of the robotic arm and the digital array, including:

[0012] Calibrate the calibration plate in the world coordinate system to obtain the relative pose between the digital array 3D projection and the calibration plate, and then solve the optical center of the digital array 3D projection;

[0013] Establish a base coordinate system for the robotic arm, use the base coordinate system of the robotic arm as the working coordinate system, solve the end coordinate system of the robotic arm through the base coordinate system, solve the relative position relationship between the end coordinate system of the robotic arm and the optical center of the digital area array 3D projection, and then establish a positioning model of the robotic arm and the digital area array 3D projection.

[0014] Optionally, establish the base coordinate system of the robot arm, including:

[0015] Calculate the relative pose relationship between the laser tracker and the calibration plate of the 3D projection of the calibration digital array, place the robotic arm at zero position, control the angles of each joint, and measure the coordinate value of the flange fixing point at the end of the robotic arm through the laser tracker target ball at every preset angle;

[0016] Keeping the target axis angle unchanged, rotate the first axis, measure the first coordinate value of the flange fixed point at the end of the robot arm, and then obtain the fitting circle;

[0017] Determine the center of the circle according to the fitted circle, and draw a straight line passing through the center of the circle and perpendicular to the plane where the fitted circle is located as the Z axis of the base coordinate system;

[0018] Repositioning the robotic arm to zero position, keeping the target axis angle unchanged, rotating the second axis, measuring the second coordinate value of the flange fixing point at the end of the robotic arm, and then obtaining the fitting plane;

[0019] Fitting the robotic arm base and measuring the plane where the robotic arm base is located;

[0020] According to the Z axis of the base coordinate system and the plane where the robot arm base is located, a normal line of the fitting plane is obtained, and the normal line is used as the Y axis of the base coordinate system;

[0021] The X axis of the base coordinate system is determined according to the coordinate system rule.

[0022] Optionally, the relative position relationship between the end coordinate system of the robotic arm and the optical center of the digital area array 3D projection is:

[0023]

[0024] Among them, T o is the transformation matrix from the digital array 3D projection coordinate system to the robot end coordinate system, E is the 3×3 unit matrix, P0=[X o Y o Z o ] T is a 3×1 position matrix.

[0025] Optionally, establishing a relative positional relationship between the target position of the robotic arm and the digital area array 3D projection positioning model and the detected object, and constructing a global measurement field includes:

[0026] According to the relative posture relationship between the calibration plate and the laser tracker, a laser tracker target ball is placed at each reference point on the surface of the object to be detected to calibrate the parameters of the range to be detected on the surface of the object to be detected;

[0027] Based on the detection range of the object to be detected, the robotic arm is placed at the optimal measurement position of the digital area array 3D projection, and the robotic arm drives the digital area array 3D projection to scan the detection range, and the number of positions required for the robotic arm is calculated according to the detection range, and then the target position of the robotic arm is obtained according to the optimal measurement position of the digital area array 3D projection;

[0028] The target position of the manipulator is calibrated row by row to obtain the relative position relationship between the target position of the manipulator, the digital area array 3D projection positioning model, and the detected object, thereby constructing the global measurement field.

[0029] Optionally, performing path planning on the robotic arm according to the scanning range of the detected object and the global measurement field to obtain a measurement target path includes:

[0030] acquiring the relative position relationship according to the global measurement field;

[0031] Performing path planning for a single station of the robotic arm based on the relative position relationship to achieve automatic detection of pits, grooves, and curved surfaces, and moving the robotic arm to an initial zero position;

[0032] Based on the detection range of the object to be detected, the surface of the object to be detected is scanned and detected by the digital area array 3D projection driven by the rotation of the robotic arm joint, and a global planning is performed on several of the stations with the robotic arm stiffness and the optimal measurement distance of the digital area array 3D projection as the target to obtain the target path.

[0033] Optionally, path planning is performed on the robotic arm at a single station to achieve automatic detection of pits, grooves, and curved surfaces, including:

[0034] When the object to be inspected has the pit, the robotic arm drives the digital area array 3D projection to perform surround detection above the pit, obtain the shape and depth of the pit, and realize automatic detection of the pit;

[0035] When the object to be detected has the groove, the outer contour of the groove is scanned, then the scanning transition is made from the outer contour to the inner side of the groove, and then the scanning returns to the starting point of the outer contour scanning to complete the automatic detection of the groove;

[0036] When the object to be inspected has the curved surface, a target curve of the curved surface is extracted, coordinate points are selected on the target curve, a function curve is fitted, and then the digital area array 3D projection is driven by the robotic arm to perform scanning and inspection according to the function curve.

[0037] Optionally, based on the measurement target path, a measurement model of the object to be inspected is obtained, and compared and analyzed with a theoretical model to achieve 3D projection global automatic deviation detection visualization, including:

[0038] Based on the measurement target path, the robotic arm drives the digital area array 3D projection to move, and the digital area array 3D projection scans to obtain point cloud data of the surface of the object being inspected;

[0039] Sequentially stitching the point cloud data to obtain a measurement model of the detected object;

[0040] If the theoretical model is known, the coordinate system of the theoretical model and the measurement model are aligned, and the point cloud data is processed and spliced, and then compared and analyzed with the theoretical model to obtain deviation pair data, obtain the detection result of the detected object, and use color index to realize 3D projection global automatic deviation detection visualization, wherein the point cloud data includes ordered point cloud and unordered point cloud;

[0041] If the theoretical model is unknown, the image obtained by the digital area array 3D projection scanning is subjected to noise reduction and enhancement, and the images scanned by a single station and the images scanned by several stations are sequentially spliced ​​to obtain the surface features of the object to be detected, and then an automated deviation detection 3D projection model is established to realize global automatic deviation detection visualization of 3D projection.

[0042] Optionally, processing and splicing the point cloud data includes:

[0043] Using a direct observation method, noise points in the ordered point cloud are deleted to obtain a noise-reduced ordered point cloud;

[0044] A double filtering method is used to perform noise reduction and smoothing on the disordered point cloud to obtain a noise-reduced disordered point cloud;

[0045] The ordered point cloud after noise reduction and the disordered point cloud after noise reduction are simplified, and the point cloud data obtained by scanning the digital area array 3D projection along the measurement target path by the robotic arm are sequentially spliced.

[0046] The present invention has the following beneficial effects:

[0047] Compared with the existing technology, the present invention realizes the fusion of robotic arm and digital area array 3D projection when detecting the integrity of the object surface, optimizes the steps of autonomous measurement required by personnel for detecting objects, and establishes an integrated dynamic projection model with global scanning of the measured object by digital area array 3D projection and autonomous movement of the robotic arm; based on the coordinate transformation matrix and the relative posture solution algorithm, compared with previous detection methods, it is more autonomous, more accurate and more comprehensive, and can realize global visualization of 3D projection, automatic deviation detection, improve detection efficiency, and reduce personnel detection time; for object surface detection, the present invention integrates robotic arm and digital area array 3D projection system to realize all-round global visualization of digital area array 3D projection, providing a reference for the construction of digital twin dynamic evolution model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0049] Figure 1 Schematic diagram of the process of the digital area array 3D projection global automatic deviation detection visualization method according to an embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the 3D projection calibration model of the robotic arm and digital area array proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] To address the shortcomings of existing technologies, large-scale inspection and deviation display can promptly check and correct dimensional deviations, reducing quality defects and increasing product inspection rates. To detect the actual surface conditions of the object being measured, digital area array 3D projection is combined with a robotic arm. The movement of the robotic arm drives the digital area array 3D projection to scan and inspect the object being measured, enabling automated scanning of the object being measured. This reduces detection errors, shortens detection time, and improves detection efficiency. Furthermore, the actual surface conditions of the object being measured can be monitored in real time, enabling global automatic deviation detection visualization using 3D projection.

[0054] To address the inability of digital area array 3D projection to achieve global positioning, automated layout, and global visualization, a detection method combining a robotic arm and digital area array 3D projection is proposed. First, a positioning model of the robotic arm and the digital area array 3D projection is established. Once the positioning model is established, a laser tracker is used to determine the relative positional relationship between the robotic arm, the digital area array 3D projection positioning model, and the object being inspected, completing the construction of the global measurement field. The robotic arm's path is planned according to the inspection range of the object, which was determined during the measurement field construction. Since the measurement model coordinate system is already generated during the measurement field construction, it is aligned with the theoretical model coordinate system. As the robotic arm moves along the planned measurement target path, the point cloud scanned by the digital area array 3D projection is sequentially spliced ​​to form a complete measurement model of the inspection area. 3D deviation comparison analysis is performed with the theoretical model, directly displaying the deviation data. A mapping relationship between the color display of the deviation detection data and the color index is determined. The deviation comparison chart allows for intuitive visualization of the deviation percentage on the surface of the inspected object, thus achieving global automatic deviation detection visualization.

[0055] like Figure 1 As shown, the digital array 3D projection global automatic deviation detection visualization method of the present invention includes the following steps:

[0056] Establish a 3D projection positioning model of the robotic arm and digital array;

[0057] Establish the relative position relationship between the target position of the robotic arm and the digital array 3D projection positioning model and the object to be detected, and construct a global measurement field;

[0058] The robot arm is path-planned based on the detection range of the object to be detected and the global measurement field to obtain the measurement target path;

[0059] Based on the measurement target path, the measurement model of the object being inspected is obtained and compared with the theoretical model to achieve global automatic deviation visualization detection of 3D projection.

[0060] Furthermore, a 3D projection positioning model of the robotic arm and the digital array is established, including:

[0061] Calibrate the calibration plate in the world coordinate system to obtain the relative pose between the digital array 3D projection and the calibration plate, and then solve the optical center of the digital array 3D projection;

[0062] Establish the base coordinate system of the robotic arm and use the base coordinate system of the robotic arm as the working coordinate system. Based on the working coordinate system, the end coordinate system of the robotic arm is solved, and then the relative position relationship between the end coordinate system of the robotic arm and the optical center of the digital area array 3D projection is solved, thereby establishing a positioning model of the robotic arm and the digital area array 3D projection.

[0063] Specifically, the 3D projection detection system plays a vital role in the global measurement field. A complete detection system can synchronize the surface status of the object being measured in real time and display the 3D information of the surface of the object being measured more intuitively. The digital area array 3D projection is fixed to the end of the robotic arm, and there is a calibration plate under the world coordinate system {W}. This calibration plate can be used to simultaneously solve the relative posture relationship between it, the digital area array 3D projection and the laser tracker. The calibration plate is calibrated using the optical projection positioning principle of the digital area array 3D projection, and the relative posture between the digital area array 3D projection and the calibration plate can be obtained, and then the optical center of the digital area array 3D projection, that is, the coordinate origin of the digital area array 3D projection coordinate system {P}, can be solved. For the positioning of the robotic arm, a laser tracker is used to calibrate the parameters of its base coordinate system. After establishing the base coordinate system, the base coordinate system of the manipulator is used as the working coordinate system. The end coordinate system of the manipulator is solved based on the working coordinate system, and then the relative position relationship between the end coordinate system of the manipulator and the optical center of the digital array 3D projection is solved to establish the positioning model of the manipulator and the digital array 3D projection, such as Figure 2 shown.

[0064] Furthermore, the base coordinate system of the robotic arm is established, including:

[0065] Obtain the relative position relationship between the laser tracker and the calibration plate of the calibration digital array 3D projection, and place the robotic arm at zero position to control the angles of each joint;

[0066] Keeping the target axis angle unchanged, rotate the first axis, measure the first coordinate value of the flange fixed point at the end of the robot arm, and then obtain the fitting circle;

[0067] Determine the center of the circle based on the fitted circle, and use the straight line passing through the center of the circle and perpendicular to the plane where the fitted circle is located as the Z axis of the base coordinate system;

[0068] Reposition the robot arm to zero position, keep the target axis angle unchanged, rotate the second axis, measure the second coordinate value of the flange fixed point at the end of the robot arm, and then obtain the fitting plane;

[0069] Fit the robotic arm base and measure the plane where the robotic arm base is located;

[0070] According to the Z axis of the base coordinate system and the plane where the robot arm base is located, the normal of the fitting plane is obtained, the normal is used as the Y axis of the base coordinate system, and the X axis of the base coordinate system is determined according to the coordinate system rule.

[0071] Specifically, for the positioning of the robotic arm, a laser tracker is used to calibrate the parameters of its base coordinate system. Place the robot arm at zero position, with the angles of each joint being 0°, -90°, 0°, -90°, 0°, and 0° respectively; keep the angles of other axes unchanged, rotate axis 1, and measure the coordinate values ​​of a fixed point on the end flange of the robot arm using the laser tracker target ball at certain angles, and fit a circle C1 based on these points; determine the center of the circle based on the fitted circle, and use the straight line l1 passing through the center of the circle and perpendicular to the plane where the circle C1 is located as the Z axis of the base coordinate system; place the robot arm at zero position again, keep the angles of other axes unchanged, rotate axis 2, and measure the coordinate values ​​of the fixed point on the end flange of the robot arm using the laser tracker target ball at certain angles, and fit a plane M based on these points; place the laser tracker target ball on the base of the robot arm, fit the robot arm base on the computer software, and measure the plane B where the base is located; use the intersection of the Z axis of the base coordinate system and the plane where the base is located as the origin of the base coordinate system, and make the normal of plane M through the coordinate origin as the Y axis, and finally determine the X axis according to the coordinate system rule. Where {B} is the base coordinate system of the manipulator, located on the plane of the manipulator's mounting base. It serves as the reference for the torsion angle coordinate systems of the other links of the manipulator and is used to describe the pose of the manipulator joints in the base coordinate system. {M} is the manipulator's end coordinate system, which is theoretically invariant relative to the optical center of the 3D projection of the digital area array fixed to the end of the manipulator.

[0072] Furthermore, the relative position relationship between the end coordinate system of the robot arm and the optical center of the digital area array 3D is:

[0073]

[0074] Among them, T o is the transformation matrix from the digital array 3D projection coordinate system to the robot end coordinate system, E is the 3×3 unit matrix, P0=[X o Y o Z o ] T is a 3×1 position matrix.

[0075] It can be seen from this that the changes in the position of the digital area array 3D projection coordinate system caused by the movement of the robotic arm can be solved in real time. In this way, the robotic arm and the digital area array 3D projection can be regarded as a relatively moving overall model, thereby establishing a positioning model between the robotic arm and the digital area array 3D projection.

[0076] Furthermore, the relative position relationship between the target position of the robot arm and the digital array 3D projection positioning model and the detected object is established to construct a global measurement field, including:

[0077] Determine the relative position relationship between the calibration plate and the laser tracker for calibrating the digital area array 3D projection, and place the laser tracker target ball at each reference point on the surface of the object to be inspected to calibrate the parameters of the projection range on the surface of the object to be inspected;

[0078] Based on the detection range of the object to be detected, a robotic arm is placed at the optimal measurement position of the digital area array 3D projection. The robotic arm drives the digital area array 3D projection to scan the detection range. The number of target positions required for the robotic arm is calculated based on the detection range, and then the target positions of the robotic arm are planned based on the optimal measurement position of the digital area array 3D projection.

[0079] The target position of the robotic arm is calibrated row by row to obtain the relative position relationship between the target position of the robotic arm and the digital array 3D projection positioning model and the object to be detected, and then a global measurement field is constructed.

[0080] Specifically, in order to achieve real-time detection of gaps and pits on the surface of the object being inspected, it is necessary to build a high-precision measurement field of the global coordinate system. Since the positioning model of the robotic arm and the digital area array 3D projection has been established, it is also necessary to establish the relative relationship between the positioning model and the object being inspected. Solve the relative posture relationship between the calibration plate of the digital area array 3D projection and the laser tracker, place the laser tracker target ball at each reference point on the surface of the object being inspected, calibrate the parameters of the area to be projected on the surface of the object being inspected, and then the laser tracker receives the laser reflected back by the target ball to obtain the coordinate system {Q} of the object being inspected. Place the robotic arm with the digital area array 3D projection installed at the optimal measurement position of the digital area array 3D projection. The placement position must ensure that the robotic arm can drive the digital area array 3D projection to scan all the surface information of the object being inspected that needs to be detected at this station. The scanning task of this station can be ended only after the scanning is completed. Since the measurement area of ​​a single robotic arm station is limited, and the robotic arm driven by the digital area array 3D projection cannot complete the measurement of the entire inspection range of the object to be inspected at a single station, it is necessary to divide the inspection range into areas according to the rigidity of the robotic arm, and use the robotic arm to drive the digital area array 3D projection to scan and inspect the divided areas one by one. The target station planning of the robotic arm is completed based on the optimal measurement position of the digital area array 3D projection, and then all the robotic arm stations required for the object to be inspected are calibrated row by row using a laser tracker. This determines the relative positional relationship between the target station of the robotic arm and the digital area array 3D projection positioning model and the object to be inspected. This forms a full coverage scan of the object to be inspected by the digital area array 3D projection, realizing real-time detection of the global coordinate system.

[0081] Furthermore, the path of the robot arm is regulated according to the scanning range of the object to be detected and the global measurement field to obtain the measurement target path, including:

[0082] Obtain relative position relationship based on the global measurement field;

[0083] Based on the relative position relationship, the robot arm performs path planning for a single station, realizes automatic detection of pits, grooves, and curved surfaces, and moves the robot arm to the initial zero position;

[0084] Based on the inspection range of the object to be inspected, the surface of the object to be inspected is scanned and inspected by driving the digital area array 3D projection through the rotation of the robotic arm joints. Based on the stiffness of the robotic arm and the optimal measurement distance of the digital area array 3D projection, a global planning is performed on several stations to obtain the measurement target path.

[0085] Specifically, after the global measurement field is constructed, that is, after the global coordinate system is unified with high precision, the relative position relationship between the digital array 3D projection and the object to be inspected can be solved in real time. In order to realize the automated detection of gaps and pits on the surface of the object to be inspected by the digital array 3D projection, the trajectory of the robot arm is first planned for a single station. Since the surface of the object to be inspected is not entirely flat and is limited by the measurement range and accuracy of the digital array 3D projection itself, it is necessary to plan the scanning path of the robot arm single station to realize automatic detection of pits, grooves, and curved surfaces. At this time, the robot arm can be moved to the initial zero position, and the surface of the object to be inspected can be scanned and inspected by rotating the joints to complete the scanning task of this station. Subsequently, multiple stations are globally planned: the robot arm station planning first meets the accessibility requirements of scanning all features on the surface of the object to be inspected. The robot arm station planning is carried out with the robot arm stiffness and the optimal measurement distance of the digital array 3D projection as the target. Due to the large size of the object to be inspected and the complex environment of the measurement field itself, a single station cannot cover the entire object to be inspected. Using the laser tracker's scanned range of the object, the surface is divided into several areas, each scanned separately until the entire object is scanned. After the robotic arm planning for the entire object is complete, the robotic arm, with the digital area array 3D projection fixed, is placed in the first station. The robotic arm then moves, driving the digital area array 3D projection to complete the measurement task for the entire measurement field according to the plan.

[0086] Furthermore, the robot arm performs path planning for a single station to achieve automatic detection of pits, grooves, and curved surfaces, including:

[0087] When there is a pit on the object being inspected, the robot arm drives the digital area array 3D projection to perform surround detection above the pit, obtain the shape and depth of the pit, and realize automatic detection of the pit;

[0088] When there is a groove on the object being inspected, the outer contour of the groove is scanned, then the outer contour is transitioned to the inner side of the groove for scanning, and then the scan returns to the starting point of the outer contour scan to complete the automatic detection of the groove.

[0089] When the object to be inspected has a curved surface, the target curve of the surface is extracted, and coordinate points are selected on the target curve to fit the function curve. The digital area array 3D projection is then driven by a robotic arm to perform scanning and inspection according to the function curve.

[0090] Specifically, for situations where there may be pits on the surface to be measured, a cone surround measurement method is adopted, that is, when a pit is detected, the robotic arm drives the digital surface array 3D projection to perform a surround detection along the shape of the pit above the pit. At this time, the field of view will form a cone, and the specific shape and depth of the pit can be detected; for situations where there may be grooves on the surface of the object to be measured, a method of scanning the intersection of two planes is adopted, that is, first completing the scan of the outer contour, then transitioning from the outer contour to scanning the inside of the groove, and finally returning to the starting point of the outer contour scan to complete the detection task of the groove; for situations where there may be curved surface features on the surface of the object to be measured, a measurement method based on the normal direction of the surface is adopted, that is, extracting a curve from the curved surface, and selecting an appropriate number of coordinate points on the curve to fit an approximate function curve using the interpolation method, and the robotic arm can drive the digital surface array 3D projection to perform scanning and detection according to the fitted curve.

[0091] Furthermore, based on the measurement target path, the measurement model of the object being inspected is obtained and compared with the theoretical model to achieve global automatic deviation detection visualization in 3D projection, including:

[0092] Based on the measurement target path, the robotic arm drives the digital area array 3D projection to move, and the digital area array 3D projection scans to obtain point cloud data on the surface of the object being inspected;

[0093] The point cloud data are stitched together in sequence to obtain the measurement model of the detected object;

[0094] If the theoretical model is known, the coordinate systems of the theoretical model and the measurement model are aligned, and the point cloud data is processed and spliced. Then, a comparison analysis is performed with the theoretical model to obtain deviation comparison data and the deviation detection result of the detected object. The color index is used to realize the global automatic deviation detection visualization of 3D projection. The point cloud data includes ordered point clouds and unordered point clouds.

[0095] If the theoretical model is unknown, the images obtained by digital area array 3D projection scanning are subjected to noise reduction and enhancement, and the images scanned by a single station and the images scanned by several stations are stitched in turn to obtain the surface features of the object being inspected. Then, an automated deviation detection 3D projection model is established to realize the visualization of global automatic deviation detection in 3D projection.

[0096] Furthermore, the point cloud data is processed and spliced, including:

[0097] The direct observation method is used to delete the noise points in the ordered point cloud to obtain the denoised ordered point cloud;

[0098] The double filtering method is used to reduce noise and smooth the disordered point cloud to obtain the disordered point cloud after noise reduction;

[0099] The ordered point cloud after noise reduction and the disordered point cloud after noise reduction are simplified, and the point cloud data obtained by scanning the digital area array 3D projection along the measurement target path by the robotic arm are spliced ​​in sequence.

[0100] Specifically, since the laser tracker has already measured the coordinate system of the area to be inspected by the object being measured by establishing the entire measurement field, if the theoretical geometric model of the object being inspected is also known through 3D modeling, the known theoretical model is aligned with the measured model of the area to be inspected, obtained by scanning and stitching the object using digital area array 3D projection, for subsequent deviation detection. The point cloud obtained by digital area array 3D projection scanning is processed and stitched together: First, noise reduction is performed on the point cloud data obtained by digital area array 3D projection scanning. For ordered point clouds, direct observation is first used to directly import the scanned raw point cloud data into the processing software. Noise points that can be directly distinguished by the human eye, such as isolated points, can be manually deleted within the relevant software. For unordered point clouds, a dual filtering method is used to reduce noise and smooth the point cloud data, indirectly eliminating or reducing the impact of noise on the accuracy of point cloud stitching. The denoised point cloud data is then streamlined while maintaining the surface details of the object being inspected, reducing data redundancy and accelerating point cloud stitching and 3D imaging. After noise reduction and streamlining, the point clouds scanned by the robotic arm along the planned path can be stitched together. Since the robotic arm's measurement path planning is complete and the coordinate systems of the theoretical and measured models are aligned, a complete measurement model of the object being inspected can be gradually obtained as the robotic arm drives the 3D projection of the digital area array. A 3D deviation comparison analysis is performed between the measured and theoretical models to obtain deviation comparison data, thereby determining the detection results of pits and gaps on the object's surface. To address the issue of visualizing the detection results, a color indexing approach is employed. The color index is established using the RGB triplet color model commonly used in computer systems. By determining the mapping relationship between the color display of the deviation detection data and the color index, the deviation data can be directly displayed in the deviation comparison chart. For pits, the length, width, and height deviation data are displayed, while for gap step deviation data, the gap and step deviation data are directly displayed. Furthermore, the color distribution in the deviation comparison chart allows for intuitive visualization of the deviation data and position information on the object's surface, achieving global automatic deviation detection visualization.

[0101] If there is no theoretical model, the images scanned by the digital area array 3D projection are subjected to noise reduction and enhancement, and then the images scanned by a single station are spliced ​​first, and then the images scanned by each station are processed and spliced ​​in turn. Then, the curvature, morphology and other characteristics of the detected object can be obtained, thereby establishing an automated deviation detection 3D projection model and realizing the visualization of 3D projection automatic deviation detection.

[0102] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A global automatic deviation detection and visualization method for digital array 3D projection, characterized by: The specific steps include: Establish a 3D projection positioning model of the robotic arm and digital array; Establishing the relative position relationship between the target position of the manipulator and the digital array 3D projection positioning model and the detected object to construct a global measurement field; Establishing the relative position relationship between the target position of the manipulator and the digital array 3D projection positioning model and the detected object, and constructing the global measurement field includes: According to the relative posture relationship between the calibration plate and the laser tracker, a laser tracker target ball is placed at each reference point on the surface of the object to be detected to calibrate the parameters of the range to be detected on the surface of the object to be detected; Based on the detection range of the object to be detected, the robotic arm is placed at the optimal measurement position of the digital area array 3D projection, and the robotic arm drives the digital area array 3D projection to scan the detection range, and the number of positions required for the robotic arm is calculated according to the detection range, and then the target position of the robotic arm is obtained according to the optimal measurement position of the digital area array 3D projection; Calibrate the target position of the robotic arm row by row to obtain the relative positional relationship between the target position of the robotic arm, the digital area array 3D projection positioning model, and the object to be detected, thereby constructing the global measurement field; plan the path of the robotic arm according to the detection range of the object to be detected and the global measurement field to obtain a measurement target path; The path of the manipulator is planned according to the scanning range of the detected object and the global measurement field to obtain a measurement target path, including: acquiring the relative position relationship according to the global measurement field; Performing path planning for a single station of the robotic arm based on the relative position relationship to achieve automatic detection of pits, grooves, and curved surfaces, and moving the robotic arm to an initial zero position; Based on the detection range of the object to be detected, the surface of the object to be detected is scanned and detected by driving the digital area array 3D projection through the rotation of the robot arm joint, and a global planning is performed on several of the stations based on the stiffness of the robot arm and the optimal measurement distance of the digital area array 3D projection to obtain the target path; Based on the measurement target path, a measurement model of the object to be inspected is obtained, and compared and analyzed with a theoretical model to achieve global automatic deviation visualization detection of 3D projection; Based on the measurement target path, a measurement model of the object to be inspected is obtained and compared with the theoretical model to achieve 3D projection global automatic deviation detection visualization, including: Based on the measurement target path, the robotic arm drives the digital area array 3D projection to move, and the digital area array 3D projection scans to obtain point cloud data of the surface of the object being inspected; Sequentially stitching the point cloud data to obtain a measurement model of the detected object; If the theoretical model is known, the coordinate system of the theoretical model and the measurement model are aligned, and the point cloud data is processed and spliced, and then compared and analyzed with the theoretical model to obtain deviation pair data, obtain the detection result of the detected object, and use color index to realize 3D projection global automatic deviation detection visualization, wherein the point cloud data includes ordered point cloud and unordered point cloud; If the theoretical model is unknown, the image obtained by the digital area array 3D projection scanning is subjected to noise reduction and enhancement, and the images scanned by a single station and the images scanned by several stations are sequentially spliced ​​to obtain the surface features of the object to be detected, and then an automated deviation detection 3D projection model is established to realize global automatic deviation detection visualization of 3D projection.

2. The digital array 3D projection global automatic deviation detection visualization method according to claim 1, characterized in that: Establish a 3D projection positioning model of the robotic arm and digital array, including: Calibrate the calibration plate in the world coordinate system to obtain the relative pose between the digital array 3D projection and the calibration plate, and then solve the optical center of the digital array 3D projection; Establish a base coordinate system for the robotic arm, use the base coordinate system of the robotic arm as the working coordinate system, solve the end coordinate system of the robotic arm through the base coordinate system, solve the relative position relationship between the end coordinate system of the robotic arm and the optical center of the digital area array 3D projection, and then establish a positioning model of the robotic arm and the digital area array 3D projection.

3. The method for global automatic deviation detection and visualization of digital array 3D projection according to claim 2, characterized in that: Establish the base coordinate system of the robotic arm, including: Calculate the relative pose relationship between the laser tracker and the calibration plate of the 3D projection of the calibration digital array, place the robotic arm at zero position, control the angles of each joint, and measure the coordinate value of the flange fixing point at the end of the robotic arm through the laser tracker target ball at every preset angle; Keeping the target axis angle unchanged, rotate the first axis, measure the first coordinate value of the flange fixed point at the end of the robot arm, and then obtain the fitting circle; Determine the center of the circle according to the fitted circle, and draw a straight line passing through the center of the circle and perpendicular to the plane where the fitted circle is located as the Z axis of the base coordinate system; Repositioning the robotic arm to zero position, keeping the target axis angle unchanged, rotating the second axis, measuring the second coordinate value of the flange fixing point at the end of the robotic arm, and then obtaining the fitting plane; Fitting the robotic arm base and measuring the plane where the robotic arm base is located; According to the Z axis of the base coordinate system and the plane where the robot arm base is located, a normal line of the fitting plane is obtained, and the normal line is used as the Y axis of the base coordinate system; The X axis of the base coordinate system is determined according to the coordinate system rule.

4. The method for global automatic deviation detection and visualization of digital area array 3D projection according to claim 2, characterized in that: The relative position relationship between the end coordinate system of the robotic arm and the optical center of the digital area array 3D projection is: Among them, T o is the transformation matrix from the digital array 3D projection coordinate system to the robot end coordinate system, E is the 3×3 unit matrix, P0=[X o Y o Z o ] T is a 3×1 position matrix.

5. The digital array 3D projection global automatic deviation detection visualization method according to claim 1, characterized in that: Perform path planning for a single station of the robotic arm to achieve automatic detection of pits, grooves, and curved surfaces, including: When the object to be inspected has the pit, the robotic arm drives the digital area array 3D projection to perform surround detection above the pit, obtain the shape and depth of the pit, and realize automatic detection of the pit; When the object to be detected has the groove, the outer contour of the groove is scanned, then the scanning transition is made from the outer contour to the inner side of the groove, and then the scanning returns to the starting point of the outer contour scanning to complete the automatic detection of the groove; When the object to be inspected has the curved surface, a target curve of the curved surface is extracted, coordinate points are selected on the target curve, a function curve is fitted, and then the digital area array 3D projection is driven by the robotic arm to perform scanning and inspection according to the function curve.

6. The digital array 3D projection global automatic deviation detection visualization method according to claim 1, characterized in that: Processing and splicing the point cloud data includes: Using a direct observation method, noise points in the ordered point cloud are deleted to obtain a noise-reduced ordered point cloud; A double filtering method is used to perform noise reduction and smoothing on the disordered point cloud to obtain a noise-reduced disordered point cloud; The ordered point cloud after noise reduction and the disordered point cloud after noise reduction are simplified, and the point cloud data obtained by scanning the digital area array 3D projection along the measurement target path by the robotic arm are sequentially spliced.

Images