Workpiece measurement method, device and system and readable storage medium

Abstract

The invention relates to a workpiece measuring method, device and system and a readable storage medium. The method is applied to a workpiece measurement system and comprises the following steps: when a mobile robot moves a depth sensor to a preset sensor detection area, detecting initial point cloud data of a to-be-measured workpiece through the depth sensor; converting the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; converting the first point cloud data from a robot coordinate system to a processing coordinate system corresponding to a workpiece table to obtain second point cloud data; converting the second point cloud data from the processing coordinate system to a workpiece coordinate system corresponding to the to-be-measured workpiece to obtain target point cloud data; and detecting position information of a plurality of contour points in the to-be-measured workpiece from the target point cloud data. The method can accurately measure the contour points in the workpiece.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing technology, and particularly to a workpiece measurement method, device, system and readable storage medium. Background Art

[0002] Workpiece measurement technology is widely used in fields such as vehicle manufacturing, aerospace, and precision instruments. For example, in vehicle manufacturing, through workpiece measurement technology, the geometric shape and dimensional deviation of workpieces in a vehicle can be detected to ensure the assembly quality and appearance quality of the vehicle body.

[0003] In traditional technologies, workpiece measurement methods mainly include: using contact measurement to accurately obtain the shape data of the actual workpiece surface, and by comparing the shape data of the actual workpiece surface with the workpiece theoretical model or fixed positions for feature point comparison, numerical parameters of the feature points are obtained. Further, based on the numerical parameters of the feature points, it can be determined whether the workpiece passes the inspection.

[0004] However, the current workpiece measurement methods have the problem of insufficient accuracy. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide an accurate workpiece measurement method, device, system, computer device, computer-readable storage medium and computer program product.

[0006] In a first aspect, the present application provides a workpiece measurement method, which is applied to a workpiece measurement system. The workpiece measurement system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected to the depth sensor; the method includes:

[0007] When the mobile robot moves the depth sensor to a preset sensor detection area, the depth sensor is used to detect the initial point cloud data corresponding to the workpiece to be measured, where the workpiece to be measured is placed on a workpiece table;

[0008] The initial point cloud data is converted from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data;

[0009] The first point cloud data is converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain the second point cloud data;

[0010] The second point cloud data is converted from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data;

[0011] From the target point cloud data, the position information of multiple contour points in the workpiece to be measured is detected.

[0012] In one embodiment, converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot includes:

[0013] Obtaining a first pose transformation relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot;

[0014] According to the first pose transformation relationship, converting the initial point cloud data from the sensor coordinate system to the robot coordinate system;

[0015] Converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table includes:

[0016] Obtaining a second pose transformation relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table;

[0017] According to the second pose transformation relationship, converting the first point cloud data from the robot coordinate system to the machining coordinate system.

[0018] In one embodiment, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured includes:

[0019] Detecting the workpiece type of the workpiece to be measured;

[0020] Based on the workpiece type, generating a third pose transformation relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured;

[0021] According to the third pose transformation relationship, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system.

[0022] In one embodiment, before converting the initial point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, the method further includes:

[0023] Detecting whether all the contour points of the workpiece to be measured are included in the first point cloud data;

[0024] In the case where all the contour points of the workpiece to be measured are not included in the first point cloud data, sending a movement instruction to the mobile robot, where the movement instruction carries updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information;

[0025] Returning to the step of detecting the initial point cloud data of the workpiece to be measured through the depth sensor until it is detected that all the contour points of the workpiece to be measured are included in the first point cloud data.

[0026] In one embodiment, detecting the position information of multiple contour points in the workpiece to be measured from the target point cloud data includes:

[0027] For each contour point in the workpiece to be measured, obtain the theoretical position information of the contour point and the preset contour point detection area corresponding to the contour point;

[0028] Detect the target area point cloud data matching the preset contour point detection area from the target point cloud data;

[0029] Perform plane fitting on the target area point cloud data according to the theoretical position information of the contour point to obtain the target plane;

[0030] Project the theoretical position information of the contour point onto the target plane to obtain the position information of the contour point.

[0031] In one embodiment, performing plane fitting on the target area point cloud according to the theoretical position information of the contour point to obtain the target plane includes:

[0032] Query reference points in the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset first distance threshold, and use the distance between the reference point and the contour point as the reference distance;

[0033] Query the target point cloud set in the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset ratio of the reference distance;

[0034] Perform plane fitting on the target point cloud set to obtain the target plane.

[0035] In one embodiment, the mobile robot includes at least a first mobile robot and a second mobile robot, and an end effector of the first mobile robot is connected to a depth sensor. The method further includes:

[0036] When the first mobile robot moves the depth sensor to the preset sensor detection area, obtain the initial point cloud data of the workpiece to be measured detected by the depth sensor;

[0037] Convert the initial point cloud data from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot to obtain the third point cloud data;

[0038] Convert the third point cloud data from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain the fourth point cloud data;

[0039] Convert the third point cloud data from the second robot coordinate system to the machining coordinate system to obtain the fifth point cloud data;

[0040] Convert the fifth point cloud data from the machining coordinate system to the workpiece coordinate system to obtain the sixth point cloud data;

[0041] Detect the position information of multiple contour points on the workpiece to be measured from the sixth point cloud data.

[0042] In a second aspect, the present application also provides a workpiece measurement device, which is applied to a workpiece measurement system. The workpiece measurement system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected to the depth sensor. The device includes:

[0043] An initial surface point cloud acquisition module, configured to detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor when the mobile robot moves the depth sensor to a preset sensor detection area, where the workpiece to be measured is placed on a workpiece table;

[0044] A first pose conversion module, configured to convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data;

[0045] A second pose conversion module, configured to convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data;

[0046] A third pose conversion module, configured to convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data;

[0047] A contour point measurement module, configured to detect the position information of multiple contour points on the workpiece to be measured from the target point cloud data.

[0048] In a third aspect, the present application also provides a workpiece measurement system, which at least includes:

[0049] A depth sensor, which is configured to detect the point cloud data of the workpiece to be measured;

[0050] A mobile robot, and the end effector of the mobile robot is connected to the depth sensor;

[0051] A controller, and the controller is configured to:

[0052] When the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on a workpiece table;

[0053] Convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data;

[0054] Convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data;

[0055] Convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, obtaining the target point cloud data;

[0056] Detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0057] In a fourth aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0058] When the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on the workpiece table;

[0059] Convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, obtaining the first point cloud data;

[0060] Convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, obtaining the second point cloud data;

[0061] Convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, obtaining the target point cloud data;

[0062] Detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0063] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0064] When the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on the workpiece table;

[0065] Convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, obtaining the first point cloud data;

[0066] Convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, obtaining the second point cloud data;

[0067] Convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, obtaining the target point cloud data;

[0068] Detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0069] In a sixth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0070] When the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on the workpiece table;

[0071] Convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data;

[0072] Convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain the second point cloud data;

[0073] Convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data;

[0074] Detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0075] For the above workpiece measurement method, device, system, computer device, computer-readable storage medium, and computer program product, currently, with the method of measuring workpieces by contact, the workpiece area measured each time is limited, and it is impossible to comprehensively measure the workpiece to be measured. Moreover, for complex workpieces, the probe cannot reach them, and it is also impossible to accurately measure the workpiece to be measured. At the same time, when comparing the shape data of the actual workpiece surface with the workpiece theoretical model, since it is necessary to perform fitting and matching on the shape data of the workpiece surface and the workpiece theoretical model, therefore, in the case where the workpiece to be measured is a simple workpiece with a featureless surface and there is a large deviation between the actual workpiece and the theoretical workpiece model, the matching effect is poor, which easily leads to a large deviation in the measurement result; while the present application is applied to a workpiece measurement system. Throughout the process, all areas of the workpiece can be completely measured by the depth sensor at one time, and the point cloud data detected by the depth sensor is more accurate. Therefore, by using the coordinate system conversion between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece table, and the workpiece coordinate system of the workpiece to be measured to extract contour points, it is possible to completely and accurately reflect the overall shape of the workpiece without comparing feature points with the workpiece theoretical model. Therefore, it is more accurate to measure the workpiece by adopting the above solution. Description of the Drawings

[0076] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0077] Figure 1 It is an application environment diagram of the workpiece measurement method in an embodiment;

[0078] Figure 2 It is a schematic flowchart of the workpiece measurement method in an embodiment;

[0079] Figure 3 It is a schematic flowchart of the workpiece measurement method in another embodiment;

[0080] Figure 4 It is a schematic diagram of the actual architecture of the workpiece measurement system in a specific application embodiment;

[0081] Figure 5 It is a structural block diagram of the workpiece measurement device in an embodiment;

[0082] Figure 6 It is an internal structure diagram of a computer device in an embodiment. Specific Embodiments

[0083] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0084] Workpiece measurement technology is widely used in fields such as vehicle manufacturing, aerospace, and precision instruments. For example, in vehicle manufacturing, through workpiece measurement technology, the geometric shape and dimensional deviation of workpieces in the vehicle can be detected to ensure the assembly quality and appearance quality of the vehicle body.

[0085] In traditional technologies, the workpiece measurement method mainly includes: using contact measurement to accurately obtain the shape data of the actual workpiece surface, and by comparing the shape data of the actual workpiece surface with the workpiece theoretical model or fixed position for feature points, obtaining the numerical parameters of the feature points. Further, based on the numerical parameters of the feature points, it can be determined whether the workpiece passes the inspection.

[0086] Specifically, a three-dimensional model is used for data conversion to construct a hypothetical measurement surface, and in cooperation with a coordinate measuring machine, a rod-shaped probe is used for progressive contact detection measurement along the normal direction of the hypothetical measurement surface; or a workpiece to be measured is fixed in a pre-set workpiece coordinate system by using a workpiece fixture for a specific configuration, and equipment such as a micrometer, a go / no-go gauge or a laser rangefinder is used to perform fixed or mobile measurement on the workpiece to obtain measurement data on the surface of the workpiece; or the data obtained by three-dimensional scanning is fitted and compared with the theoretical three-dimensional model, and the contour point measurement data of the workpiece is obtained by constructing the difference between the scanned three-dimensional point cloud and the theoretical surface model.

[0087] However, in the prior art, the method of using a coordinate rod-shaped probe for progressive contact detection measurement along the normal direction of a hypothetical measurement surface needs to be used in cooperation with a coordinate measuring machine, the hardware equipment is expensive, and only a part of the workpiece measurement data can be obtained by one detection measurement. For multi-point dense measurement, a complex program needs to be written for assistance, and there is a situation where the probe cannot reach for complex workpieces, so the workpiece cannot be accurately measured; and the method of detecting and measuring the surface of a workpiece by using a specific fixture and measuring tool needs to fix the workpiece in a fixed workpiece coordinate system by using a specific fixture and perform measurement by using a specific measuring tool. The measurement effect on individual workpieces to be measured with a large deviation from the theoretical contour is poor, and only a pass / fail result can be provided, and accurate quantitative measurement indicators cannot be given. At the same time, the hardware equipment is only adapted to a single workpiece; and the method of obtaining the difference on the actual workpiece surface by fitting and comparing the data obtained by three-dimensional scanning with the theoretical three-dimensional model needs to perform fitting and matching on the point cloud data and the theoretical surface model. The matching effect on workpieces to be measured with a simple featureless surface, a large deviation between the physical object and the theoretical workpiece model is poor, which easily leads to the deviation of the obtained measurement points from the theoretical values. At the same time, it cannot accurately feedback the situation where the overall workpiece is offset due to the deformation of the reference point for fixing the workpiece itself.

[0088] Therefore, the present application proposes a workpiece measurement method, which is applied to a workpiece measurement system. The workpiece measurement system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected to the depth sensor. The method includes: when the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on a workpiece table; converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data; converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data; converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; detecting the position information of multiple contour points in the workpiece to be measured from the target point cloud data. During the whole process, all areas of the workpiece can be completely measured by the depth sensor at one time, and the point cloud data detected by the depth sensor is more accurate. Therefore, by using the coordinate system conversion between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece table, and the workpiece coordinate system of the workpiece to be measured to extract contour points, the overall morphology of the workpiece can be completely and accurately reflected. In the case where the workpiece to be measured is a simple workpiece with a featureless surface and there is a large deviation between the physical workpiece and the theoretical workpiece model, since feature points are not used for measurement but coordinate system conversion is adopted, accurate target point cloud data can be obtained without comparing feature points with the workpiece theoretical model and no deviation will occur. Therefore, the workpiece measurement is more accurate by adopting the above solution.

[0089] The workpiece measurement method provided by the embodiments of the present application can be applied to an application environment such as Figure 1 shown in FIG. That is, in the workpiece measurement system 1000. The workpiece measurement system 1000 at least includes a depth sensor 102, a mobile robot 104, and a controller 108. The mobile robot 104 is connected to the depth sensor 102. The depth sensor is used to detect the point cloud data of the workpiece 110 to be measured, and the workpiece to be measured is placed on the workpiece table 106. Figure 1 Taking the example that the workpiece table 106 can be fixed on the base of the mobile robot 104, however, the workpiece table 106 can also be placed outside the mobile robot 104, which is not limited herein.

[0090] When the controller 108 controls the mobile robot 104 to move the depth sensor 102 to a preset sensor detection area, the initial point cloud data corresponding to the workpiece 110 to be measured is detected by the depth sensor 102; the initial point cloud data is converted from the sensor coordinate system corresponding to the depth sensor 102 to the robot coordinate system corresponding to the mobile robot 104 to obtain the first point cloud data; the initial point cloud data is converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table 106 to obtain the second point cloud data; the second point cloud data is converted from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece 110 to be measured to obtain the target point cloud data; the position information of multiple contour points in the workpiece 110 to be measured is detected from the target point cloud data.

[0091] In an exemplary embodiment, as Figure 2 shown, a workpiece measurement method is provided, which is applied to a workpiece measurement system 1000. The system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected to the depth sensor; the method includes:

[0092] S100, when the mobile robot moves the depth sensor to a preset sensor detection area, the initial point cloud data corresponding to the workpiece to be measured is detected by the depth sensor.

[0093] Among them, the workpiece to be measured is placed on the workpiece table. The end effector of the mobile robot is connected to the depth sensor. The workpiece table can be fixed on the base of the mobile robot or placed outside the mobile robot, which is not limited here. The depth sensor can be a sensor for detecting point cloud information, such as a three-dimensional scanning camera, etc.

[0094] Specifically, the workpiece measurement system further includes a controller, and the controller controls the mobile robot to drive the depth sensor along a predetermined trajectory to a preset detection area through the end effector. Generally, the preset detection area refers to an area where all contour point information on the workpiece to be measured can be detected. For example, if there is a contour point on the workpiece to be measured, the end effector of the mobile robot will drive the depth sensor according to the programmed robot program to run to the spatial position where this contour point can be detected, and these predetermined positions are taught in the mobile robot in advance, that is, the mobile robot will run to this position every time.

[0095] After the mobile robot moves the depth sensor to the preset sensor detection area, the controller sends a workpiece trigger signal to the depth sensor. After receiving the workpiece trigger signal, the depth sensor detects the workpiece to be measured, obtains the three-dimensional scanning data of the surface topography of the workpiece to be measured within the field of view, that is, the initial point cloud data, and feeds the initial point cloud data back to the controller. In practical applications, the surface topography of the workpiece to be measured can be a free-form surface, a plane, or other geometric three-dimensional scanning space structures.

[0096] Furthermore, using the depth sensor in cooperation with the mobile robot can perform large-range measurement and analysis of workpiece contour points. Moreover, multiple contour point data of the workpiece to be measured can be obtained simultaneously in one scan. And because the robot has flexible movements, therefore, in the case where the workpiece has a complex spatial structure, the depth sensor can also achieve complete measurement of the workpiece, improving the measurement accuracy and efficiency of the contour points in the workpiece.

[0097] In an exemplary embodiment, the controller is communicatively connected to the mobile robot and the depth sensor respectively through connection cables. The controller sends a movement trajectory instruction to the mobile robot through the connection cable. After receiving the movement trajectory instruction, the mobile robot moves the depth sensor to the preset sensor detection area; after the controller confirms that the depth sensor has reached the preset sensor detection area, it sends a workpiece trigger signal to the depth sensor through the connection cable. After receiving the workpiece trigger signal, the depth sensor detects the initial point cloud data of the workpiece to be measured.

[0098] S200, Convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data.

[0099] Among them, the robot coordinate system is actually the base coordinate system corresponding to the mobile robot.

[0100] Specifically, the depth sensor is connected to the end effector of the mobile robot. That is to say, the relationship between the depth sensor and the mobile robot is a "hand-eye" relationship. Therefore, the initial point cloud data in the sensor coordinate system corresponding to the depth sensor can be mapped to the base coordinate system corresponding to the mobile robot to obtain the first point cloud data of the workpiece to be measured in the robot coordinate system.

[0101] S300, Convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain the second point cloud data.

[0102] Among them, the workpiece table is used to fix and position the workpiece to ensure its stability and accuracy during the machining or measurement process. Different workpieces require different workpiece tables to adapt to their shapes and sizes.

[0103] Specifically, the workpiece table can be installed on the base of the mobile robot or at a position outside the mobile robot. In order to obtain more accurate point cloud data of the workpiece to be measured, it is necessary to convert the initial point cloud data from the robot coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured. Therefore, the initial point cloud data can be first converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain the second point cloud data in the machining coordinate system.

[0104] In addition, compared with the method of detecting and measuring the workpiece surface through specific fixed jigs and measuring tools in the prior art, the form of the workpiece table for fixing the workpiece in this application can be various, and it can be well compatible with different shapes of workpiece types. Moreover, due to the non-contact characteristic of the scanning probe, it can still maintain a good measurement effect for the individual workpieces to be measured with a large deviation from the theoretical contour, and can give quantitative measurement indicators.

[0105] S400. Convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data.

[0106] Specifically, the workpiece to be measured is placed on the machining table. After obtaining the second point cloud data in the machining coordinate system, it is also necessary to convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data in the workpiece coordinate system.

[0107] S500. Detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0108] Specifically, through the extraction and measurement algorithm of surface contour points, detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data. Further, based on the position information of all contour points, generate a measurement report and an analysis report of the contour points in the workpiece to be measured. In some other embodiments, this application is not limited to detecting the position information of contour points, and can also detect the position information of other features, such as holes and corners.

[0109] In an exemplary embodiment, based on the position information of all contour points, generate a measurement report and an analysis report of the contour points in the workpiece to be measured, including: obtaining the theoretical position information of all contour points and the allowable position error tolerance value set by the customer or default in the software, and based on the position information of all contour points, the theoretical position information of all contour points, and the allowable position error tolerance value set by the customer or default in the software, evaluate whether each contour point of the workpiece to be measured is qualified, and output the evaluation results of all contour points to the user.

[0110] Among them, based on the position information of all contour points, the theoretical position information of all contour points, and the allowable position error tolerance value set by the customer or default in the software, it is evaluated whether each contour point of the workpiece to be measured is qualified, including: for each contour point, obtaining the position error value between the position information and the theoretical position information, and judging whether the position error value is greater than the allowable position error tolerance value. If the position error value is greater than the allowable position error tolerance value, each contour point of the workpiece to be measured is unqualified; if the position error value is less than or equal to the allowable position error tolerance value, each contour point of the workpiece to be measured is qualified.

[0111] In the above workpiece measurement method, currently, when using the contact measurement method for workpieces, the workpiece area measured each time is limited, and it is impossible to comprehensively measure the workpiece to be measured. Moreover, for complex workpieces, the probe cannot reach them, and it is also impossible to accurately measure the workpiece to be measured. At the same time, when comparing the shape data of the actual workpiece surface with the workpiece theoretical model, since it is necessary to perform fitting and matching on the shape data of the workpiece surface and the workpiece theoretical model, therefore, in the case where the workpiece to be measured is a simple featureless surface workpiece and there is a large deviation between the actual workpiece and the theoretical workpiece model, the matching effect is poor, which easily leads to a large deviation in the measurement result; while this application is applied to a workpiece measurement system. Throughout the process, all regions of the workpiece can be completely measured by the depth sensor at one time, and the point cloud data obtained by the depth sensor detection has higher accuracy. Therefore, by using the coordinate system conversion between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece table, and the workpiece coordinate system of the workpiece to be measured to extract contour points, the overall morphology of the workpiece can be completely and accurately reflected. And in the case where the workpiece to be measured is a simple featureless surface workpiece and there is a large deviation between the actual workpiece and the theoretical workpiece model, since feature points are not used for measurement but coordinate system conversion is adopted, therefore, there is no need to compare feature points with the workpiece theoretical model, and accurate target point cloud data can be obtained without deviation. Therefore, it is more accurate to measure the workpiece by adopting the above solution.

[0112] In an exemplary embodiment, converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot includes:

[0113] Obtaining the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot; according to the first pose conversion relationship, converting the initial point cloud data from the sensor coordinate system to the robot coordinate system.

[0114] Specifically, obtain the current pose information of the mobile robot, that is, the first pose conversion relationship between the end coordinate system corresponding to the mobile robot and the base coordinate system corresponding to the mobile robot. Since the depth sensor is on the end effector of the robot, therefore, the first pose conversion relationship is also the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot. And according to the first pose conversion relationship, transfer the initial point cloud data to the robot coordinate system.

[0115] In an exemplary embodiment, the first pose conversion relationship includes the coordinate conversion parameters of the end effector of the mobile robot. Through the coordinate conversion parameters of the end effector of the mobile robot and the joint motion angles, transfer the initial point cloud data to the robot coordinate system.

[0116] Let the initial point cloud data be , and the coordinate conversion parameters of the end effector of the mobile robot be [R|t]. Then the first point cloud data is .

[0117] In an exemplary embodiment, the first pose conversion relationship can also be obtained by calculating the motion parameters of the mobile robot and the joint angular motion angles Ji of the mobile robot. Specifically, obtain the motion parameters of the mobile robot. For example, according to the structural characteristics of the mobile robot, determine its link parameters (such as length, twist angle, etc.) and joint types. Through the motion parameters of the mobile robot, use the Denavit-Hartenberg (D-H) coordinate system parameter method or other methods to establish the kinematic model of the mobile robot; read the current motion angles of each joint of the mobile robot through sensors or controllers. According to the kinematic model and the joint angular motion angles, use mathematical operations such as matrix multiplication to gradually calculate the first pose conversion relationship between the sensor coordinate system and the robot coordinate system.

[0118] Transfer the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, including:

[0119] Obtain the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; according to the second pose conversion relationship, transfer the first point cloud data from the robot coordinate system to the machining coordinate system.

[0120] Specifically, after completing the scanning actions of all contour points, obtain the pre-calibrated second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table to transfer the first point cloud data from the robot coordinate system to the machining coordinate system, where the second pose conversion relationship is a fixed conversion matrix.

[0121] Let the second pose conversion relationship be K0, and the first point cloud data be , then the second point cloud data .

[0122] In the above embodiments, by obtaining the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot, and the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table, the initial point cloud data can be accurately converted from the sensor coordinate system to the robot coordinate system, and then from the robot coordinate system to the machining coordinate system.

[0123] In an exemplary embodiment, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured includes:

[0124] Detecting the workpiece type of the workpiece to be measured; generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type; and converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0125] Wherein, the workpiece types of the workpieces to be measured include parts with different shapes (such as planes, curved surfaces, complex geometric bodies), sizes, materials, and machining requirements. For each workpiece type of the workpiece to be measured during machining, measurement, or assembly, a specific coordinate system matching the workpiece type is required for positioning, that is, the workpiece coordinate system.

[0126] Specifically, when the workpiece types of the workpieces to be measured are different, the pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured is also different. When the workpiece to be measured is placed on the workpiece table, it is necessary to establish a third pose conversion relationship between the workpiece coordinate system and the machining coordinate system according to the workpiece type of the workpiece to be measured, and convert the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0127] More specifically, obtain a matching database between the workpiece type and the third pose conversion relationship, and query the third pose conversion relationship matching the workpiece type from the matching database according to the workpiece type of the current workpiece to be measured. In practical applications, let the third pose conversion relationship be a fixed conversion matrix matching the workpiece type of the workpiece to be measured , the second point cloud data is , then the target point cloud data .

[0128] In an exemplary embodiment, the pose conversion relationship between the workpiece coordinate system and the machining coordinate system depends on the shape, size, placement method of the workpiece to be measured, and the design of the workpiece table.

[0129] Regarding the influence of geometric shapes: When the workpiece to be measured is a flat workpiece, simple translation and rotation can achieve coordinate system conversion; when the workpiece to be measured is a curved surface or a complex geometric workpiece, more complex transformations are required, such as non-linear transformation or feature-based transformation.

[0130] Regarding the influence of size and placement: The size and placement of the workpiece to be measured will affect its position and orientation on the workpiece table, thereby affecting the pose conversion relationship. For example, large workpieces may need to consider deformation caused by gravity, while small workpieces may be more susceptible to the accuracy of the workpiece table.

[0131] Regarding the influence of workpiece table design: The design of the workpiece table determines the fixing method and positioning accuracy of the workpiece on the workpiece table. Different workpiece table designs will result in different pose conversion relationships.

[0132] For example, when the workpiece to be measured is a flat workpiece with a rectangular shape, the workpiece is placed on a flat workpiece table and fixed by two positioning pins and a clamping device. In this case, the pose conversion relationship only needs to consider translation and rotation. When the workpiece to be measured is a curved surface workpiece with a spherical shape, the workpiece is placed on a specially designed curved surface workpiece table to ensure its stability and accuracy. In this case, the pose conversion relationship needs to consider factors such as the curvature of the curved surface, the position and orientation of the workpiece on the workpiece table, etc.

[0133] In the above embodiments, by setting the workpiece type of the workpiece to be measured as an index, and then obtaining the pose conversion matrix between the machining coordinate system and the measuring workpiece coordinate system, the second point cloud data can be accurately converted from the machining coordinate system to the workpiece coordinate system.

[0134] In an exemplary embodiment, the method of obtaining the pose conversion relationship between the machining coordinate system and the workpiece coordinate system by indexing with the workpiece type mentioned in the above embodiments can also be replaced by establishing the workpiece coordinate system by using the point cloud of the workpiece to be measured obtained, such as establishing the workpiece coordinate system by model matching.

[0135] In an exemplary embodiment, the method of obtaining the pose conversion relationship between the machining coordinate system and the workpiece coordinate system by indexing with the workpiece type mentioned in the above embodiments can also be replaced by the method of extracting and constructing the workpiece coordinate system using the reference points of the workpiece itself for point cloud data coordinate conversion.

[0136] In an exemplary embodiment, as Figure 3 shown, before S300, the method further includes:

[0137] S220, detecting whether all contour points of the workpiece to be measured are included in the first point cloud data.

[0138] S240. When not all contour points of the workpiece to be measured are included in the first point cloud data, send a movement instruction to the mobile robot, where the movement instruction carries updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information.

[0139] S260. Return to the step of detecting the initial point cloud data of the workpiece to be measured through the depth sensor until it is detected that all contour points of the workpiece to be measured are included in the first point cloud data.

[0140] Specifically, check whether all contour point data of the workpiece to be measured have been scanned. If not, return to collect data in the new detection area. If so, proceed to the next step of converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table.

[0141] In detail, when not all contour points of the workpiece to be measured are included in the first point cloud data, it is necessary to go to a new detection area to detect the initial point cloud data of the workpiece to be measured. Therefore, it is necessary to generate a movement instruction according to other pre-set detection area information and send the movement instruction to the mobile robot, so that the mobile robot moves the depth sensor to the detection area corresponding to the updated detection area information.

[0142] When the mobile robot moves the depth sensor to the new detection area, obtain the initial point cloud data of the workpiece to be measured detected by the depth sensor in the new detection area; convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data.

[0143] Then, re-detect whether all contour points of the workpiece to be measured are included in the first point cloud data, and repeat the above steps until it is detected that all contour points of the workpiece to be measured are included in the first point cloud data.

[0144] In the above embodiment, by detecting whether all contour points of the workpiece to be measured are included in the first point cloud data before converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, that is, after converting the first point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, the first point cloud data of the workpiece to be measured can be obtained more comprehensively.

[0145] In one embodiment, the step of detecting whether all contour points of the initial point cloud data include the workpiece to be measured can be set before converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot and executed after obtaining the initial point cloud data of the workpiece to be measured detected by the depth sensor.

[0146] In one embodiment, after converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, it is detected whether the target point cloud data contains all the contour points of the workpiece to be measured. If it does not contain all the contour points of the workpiece to be measured, a movement instruction is sent to the mobile robot to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information, and the acquisition of the initial point cloud data of the workpiece to be measured is restarted.

[0147] In an exemplary embodiment, the position information of multiple contour points in the workpiece to be measured is detected from the target point cloud data, including:

[0148] For each contour point in the workpiece to be measured, the theoretical position information of the contour point and the preset contour point detection area corresponding to the contour point are obtained; the target area point cloud data matching the preset contour point detection area is detected from the target point cloud data; according to the theoretical position information of the contour point, plane fitting is performed on the target area point cloud data to obtain a target plane; the theoretical position information of the contour point is projected onto the target plane to obtain the position information of the contour point.

[0149] Specifically, a contour point list [i, O in is preset, where i is the contour point index, and O in is the theoretical position information of the contour point corresponding to each contour point index, and O in contains the spatial coordinate information of [Xn, Yn, Zn]. At this time, using the contour point index i, the theoretical position information O in corresponding to the contour point index i is queried from the preset contour point list.

[0150] Furthermore, a contour point detection area list [i, pt i can also be preset, where i is the contour point index, and pt i is the theoretical position information of the contour point corresponding to each contour point index, and a series of spatial points in the space that define the surrounding space area of the contour point i. Here, the space area surrounded by pt i is the ROI (Region of Interest) contour point detection area , which is used to segment the target area point cloud data Proi-i that needs to extract contour points. Therefore, the target area point cloud data matching the preset contour point detection area can be detected from the target point cloud data .

[0151] Furthermore, based on the theoretical position information of the contour points, plane fitting can be performed on the point cloud data of the target area to obtain the target plane L. Among them, the method of plane fitting can be RANSAC (Random Sample Consensus) plane fitting. RANSAC plane fitting is a robust model estimation method, especially suitable for processing data containing a large amount of noise and outliers. The basic idea is to estimate the model parameters from the data by means of random sampling and model verification. In plane fitting, RANSAC randomly selects three points (because three non-collinear points can determine a plane), and then calculates the plane model determined by these three points. Then, RANSAC calculates the distances from all other points to this plane and determines whether these points conform to this plane model according to a preset threshold. This process is repeated multiple times, and finally the plane model with the most conforming points is selected as the final result.

[0152] Then, project the theoretical position information of the contour points along the normal vector of the target plane L onto the target plane to obtain the actual position information of the contour points.

[0153] In some other embodiments, the method of projecting the nominal points onto the fitted target plane through the plane normal vector mentioned above to obtain the spatial coordinates of the contour points can also be changed to projecting the nominal points onto the fitted target plane along any specified spatial vector.

[0154] In the above embodiments, a piece of point cloud, that is, a piece of point cloud around the point closest to the contour point, is used to improve the accuracy and stability of the actual value. Because directly using the point cloud to extract a single point will result in the inability to accurately and stably feedback the distance or spatial position due to the error of the sensor itself. However, after using a piece of point cloud for plane fitting to obtain the target plane and then performing the projection mapping from the point to the plane, it can ensure stable, accurate and interference-free calculation of the spatial coordinates of the required contour points, thereby improving the accuracy of the measurement of the contour points in the workpiece.

[0155] In an exemplary embodiment, detecting the position information of multiple contour points in the workpiece to be measured from the target point cloud data further includes: for each contour point of the workpiece to be measured, obtaining the theoretical position information of the contour point corresponding to the contour point; performing plane fitting on the target point cloud data according to the theoretical position information of the contour point to obtain the target plane; projecting the theoretical position information of the contour point onto the target plane to obtain the position information of the contour point. That is to say, the step of obtaining the preset contour point detection area can also be cancelled, and the detection of the contour point position information can be directly performed in the target point cloud data.

[0156] In an exemplary embodiment, performing plane fitting on the target area point cloud according to the theoretical position information of the contour point to obtain the target plane includes:

[0157] Query reference points in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset first distance threshold, and use the distance between the reference points and the contour points as the reference distance; query a set of target point clouds in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset ratio of the reference distance; perform plane fitting on the set of target point clouds to obtain the target plane.

[0158] Specifically, query reference points in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset first distance threshold. For example, query the point with the smallest distance from the theoretical position information of the contour points as the reference point, and use the distance between the reference point and the contour point as the reference distance. Specifically, let the theoretical position information of the contour point be (Xn, Yn, Zn), the reference point a(ax, ay, az), and the reference distance T can be expressed as:

[0159]

[0160]

[0161] Among them, is the target point cloud data.

[0162] Search for all sets of target point clouds in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset ratio K of the reference distance, where K is set according to historical experience, generally around 1.2 to 1.5, that is, ensure that the distance is slightly greater than the distance of the nearest reference point, so as to include the surrounding points that are a little farther away from the reference point.

[0163] Let the theoretical position information of the contour point be (Xn, Yn, Zn), then each point pt[ptx, pty, ptz] in the set of target point clouds needs to satisfy the following expression:

[0164]

[0165] Finally, perform plane fitting on the set of target point clouds to obtain the target plane L[A, B, C, D], where [A, B, C, D] are the four coefficients in the general equation of the plane.

[0166] In this embodiment, by querying reference points in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset first distance threshold, and using the distance between the reference points and the contour points as the reference distance, and querying a set of target point clouds in the target point cloud data whose distances from the theoretical position information of the contour points are less than a preset ratio of the reference distance, the set of target point clouds can be accurately plane-fitted to obtain the target plane for accurately projecting the contour points.

[0167] In an exemplary embodiment, for the method of obtaining the actual position information of the contour points by performing projection through a fitting plane as mentioned in the foregoing embodiment, the step of the fitting plane can also be cancelled, and a reference point can be directly selected, such as the spatial coordinates of the closest point to the contour point as the actual measurement position information of the contour point.

[0168] In an exemplary embodiment, the mobile robot at least includes a first mobile robot and a second mobile robot. The end effector of the first mobile robot is connected to a depth sensor. The method further includes:

[0169] When the first mobile robot moves the depth sensor to a preset sensor detection area, obtain the initial point cloud data of the workpiece to be measured detected by the depth sensor; convert the initial point cloud data from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot to obtain the third point cloud data; convert the third point cloud data from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain the fourth point cloud data; convert the third point cloud data from the second robot coordinate system to the machining coordinate system to obtain the fifth point cloud data; convert the fifth point cloud data from the machining coordinate system to the workpiece coordinate system to obtain the sixth point cloud data; detect the position information of multiple contour points in the workpiece to be measured from the sixth point cloud data.

[0170] Specifically, the measurement and coordinate transformation of a single mobile robot in the present application can also be changed to multiple mobile robots unifying all measurement data to the same coordinate system according to the relative coordinate transformation relationship between the mobile robots.

[0171] More specifically, first, calibrate the robot pose transformation relationship between multiple mobile robots. For example, the pose transformation relationship between the first mobile robot and the second mobile robot can be obtained, and when the first mobile robot moves the depth sensor to a preset sensor detection area, obtain the initial point cloud data of the workpiece to be measured detected by the depth sensor; according to the pose transformation relationship between the sensor coordinate system and the first robot coordinate system corresponding to the first mobile robot, convert the initial point cloud data from the sensor coordinate system to the first robot coordinate system to obtain the third point cloud data, and then through the pose transformation relationship between the first mobile robot and the second mobile robot, convert the third point cloud data from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain the fourth point cloud data. In practical applications, the end effector of the first mobile robot is connected to the depth sensor, and the second mobile robot can be placed between the first mobile robot and the processing table to reduce the inaccuracy of the pose transformation caused by the excessive distance between the first mobile robot and the processing table. In addition, the processing table can also be placed on the base of the second mobile robot.

[0172] Secondly, obtain the pose transformation relationship between the second robot coordinate system and the machining coordinate system. According to the pose transformation relationship between the second robot coordinate system and the machining coordinate system, transform the third point cloud data from the second robot coordinate system to the machining coordinate system to obtain the fifth point cloud data.

[0173] Then, according to the pose transformation relationship between the machining coordinate system and the workpiece coordinate system, transform the fifth point cloud data from the machining coordinate system to the workpiece coordinate system to obtain the sixth point cloud data; finally, detect the position information of multiple contour points of the workpiece to be measured from the sixth point cloud data.

[0174] Furthermore, the mobile robots in this application are not limited to two. The above takes two mobile robots as an example to describe how to implement the workpiece measurement method when there are more than one mobile robot.

[0175] In the above embodiments, by setting multiple mobile robots and according to the pose transformation relationship between the multiple mobile robots, accurate workpiece measurement can be performed even when the distance between the position of the workpiece table where the workpiece to be measured is placed and the mobile robot connected to the depth sensor is relatively far.

[0176] In an exemplary embodiment, this application is applied to a workpiece measurement system, such as Figure 4 shown in the actual system architecture diagram of a specific application example. The system at least includes a controller 1, a depth sensor 2, and a mobile robot 3. The end effector of the mobile robot 3 is connected to the depth sensor 2, and a workpiece table 4 is arranged on the base of the mobile robot 3. The workpiece 5 to be measured is placed on the workpiece table 4.

[0177] Based on several other structures, the controller 1 uses the data acquisition method of flexible three-dimensional scanning to obtain the surface topography data of the workpiece to be measured. After mapping and transforming the three-dimensional scanning data to the workpiece coordinate system of the workpiece to be measured, it cooperates with the surface contour point sampling algorithm to obtain the actual spatial coordinate values of the contour points of the workpiece to be measured, and thus outputs the final measurement data and error analysis report. Specifically, it includes:

[0178] S1. When the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor.

[0179] S2. Obtain the first pose transformation relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot; according to the first pose transformation relationship, transform the initial point cloud data from the sensor coordinate system to the robot coordinate system to obtain the first point cloud data.

[0180] S3. Detect whether all the contour points of the workpiece to be measured are included in the first point cloud data; in the case that not all the contour points of the workpiece to be measured are included in the first point cloud data, send a movement instruction to the mobile robot, where the movement instruction carries the updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information. Return to the step of detecting the initial point cloud data corresponding to the workpiece to be measured through the depth sensor until it is detected that all the contour points of the workpiece to be measured are included in the first point cloud data. This step S3 can be adaptively set as the next step of S2, or can be adaptively set as the next step of S1, or can also be adaptively set as the next step of S4 or S5.

[0181] S4. Obtain the second pose transformation relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; according to the second pose transformation relationship, transform the first point cloud data from the robot coordinate system to the machining coordinate system to obtain the second point cloud data.

[0182] S5. Detect the workpiece type of the workpiece to be measured; based on the workpiece type, generate the third pose transformation relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured; according to the third pose transformation relationship, transform the second point cloud data from the machining coordinate system to the workpiece coordinate system to obtain the target point cloud data.

[0183] S6. For each contour point of the workpiece to be measured, obtain the theoretical position information of the contour point and the preset contour point detection area; detect the target area point cloud data matching the preset contour point detection area from the target point cloud data; query the reference points in the target point cloud data whose distance from the theoretical position information of the contour point is less than the preset first distance threshold, and take the distance between the reference point and the contour point as the reference distance; query the target point cloud set in the target point cloud data whose distance from the theoretical position information of the contour point is less than the preset ratio of the reference distance; perform plane fitting on the target point cloud set to obtain the target plane; project the theoretical position information of the contour point onto the target plane to obtain the position information of the contour point.

[0184] S7. Compare the position information of the contour point with the theoretical position information of the contour point, and further output the error analysis value, which is output as the measurement report and the error analysis report.

[0185] It should be understood that although the steps in the flowcharts involved in the above embodiments are sequentially shown according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0186] Based on the same inventive concept, an embodiment of the present application also provides a workpiece measurement device for implementing the workpiece measurement method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the workpiece measurement device provided below can refer to the limitations on the workpiece measurement method in the above text, and will not be repeated here.

[0187] In an exemplary embodiment, as Figure 5 shown, applied to a workpiece measurement system, the system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected to the depth sensor; the present application provides a workpiece measurement device, including: an initial surface point cloud acquisition module 100, a first pose conversion module 200, a second pose conversion module 300, a third pose conversion module 400, and a contour point measurement module 500, where:

[0188] The initial surface point cloud acquisition module 100 is used to detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor when the mobile robot moves the depth sensor to a preset sensor detection area, where the workpiece to be measured is placed on a workpiece table.

[0189] The first pose conversion module 200 is used to convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data.

[0190] The second pose conversion module 300 is used to convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain the second point cloud data.

[0191] The third pose conversion module 400 is used to convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data.

[0192] The contour point measurement module 500 is used to detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0193] In one embodiment, the first pose conversion module 200 is further configured to obtain the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot; according to the first pose conversion relationship, convert the initial point cloud data from the sensor coordinate system to the robot coordinate system; the second pose conversion module 300 is further configured to obtain the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; according to the second pose conversion relationship, convert the first point cloud data from the robot coordinate system to the machining coordinate system.

[0194] In one embodiment, the third pose conversion module 400 is further configured to detect the workpiece type of the workpiece to be measured; based on the workpiece type, generate the third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured; according to the third pose conversion relationship, convert the second point cloud data from the machining coordinate system to the workpiece coordinate system.

[0195] In one embodiment, the workpiece measuring device further includes a contour point detection module, which is configured to detect whether all the contour points of the workpiece to be measured are included in the first point cloud data; in the case that all the contour points of the workpiece to be measured are not included in the first point cloud data, send a movement instruction to the mobile robot, where the movement instruction carries the updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information; return to the step of detecting the initial point cloud data corresponding to the workpiece to be measured through the depth sensor until it is detected that all the contour points of the workpiece to be measured are included in the first point cloud data.

[0196] In one embodiment, the contour point measurement module 500 is further configured to, for each contour point in the workpiece to be measured, obtain the theoretical position information of the contour point and the preset contour point detection area corresponding to the contour point; detect the target area point cloud data matching the preset contour point detection area from the target point cloud data; perform plane fitting on the target area point cloud data according to the theoretical position information of the contour point to obtain the target plane; project the theoretical position information of the contour point onto the target plane to obtain the position information of the contour point.

[0197] In one embodiment, the contour point measurement module 500 is further configured to query a reference point from the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset first distance threshold, and use the distance between the reference point and the contour point as the reference distance; query a target point cloud set from the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset ratio of the reference distance; perform plane fitting on the target point cloud set to obtain the target plane.

[0198] In one embodiment, the mobile robot at least includes a first mobile robot and a second mobile robot, and the end effector of the first mobile robot is connected to a depth sensor; the workpiece measuring device further includes a measuring module, and the measuring module is configured to obtain the initial point cloud data of the workpiece to be measured detected by the depth sensor when the first mobile robot moves the depth sensor to a preset sensor detection area; convert the initial point cloud data from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot to obtain the third point cloud data; convert the third point cloud data from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain the fourth point cloud data; convert the third point cloud data from the second robot coordinate system to the machining coordinate system to obtain the fifth point cloud data; convert the fifth point cloud data from the machining coordinate system to the workpiece coordinate system to obtain the sixth point cloud data; and detect the position information of a plurality of contour points in the workpiece to be measured from the sixth point cloud data.

[0199] Each module in the above workpiece measuring device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0200] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the initial point cloud data of the workpiece to be measured detected by the depth sensor. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a workpiece measuring method.

[0201] Those skilled in the art can understand that Figure 6The structure shown is a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. Specifically, the computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0202] In one embodiment, this application also provides a workpiece measurement system, which at least includes:

[0203] A depth sensor, which is used to detect the point cloud data of the workpiece to be measured;

[0204] A mobile robot, and the end effector of the mobile robot is connected to the depth sensor;

[0205] A controller, which is used to: when the mobile robot moves the depth sensor to a preset sensor detection area, detect the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on the workpiece table; convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain the first point cloud data; convert the first point cloud data from the robot coordinate system to the processing coordinate system corresponding to the workpiece table to obtain the second point cloud data; convert the second point cloud data from the processing coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain the target point cloud data; detect the position information of multiple contour points in the workpiece to be measured from the target point cloud data.

[0206] The specific implementation means are the same as those in the above workpiece measurement method, and will not be elaborated here.

[0207] In an example, the workpiece table on which the workpiece to be measured is placed may or may not belong to a device of the workpiece measurement system.

[0208] In one embodiment, a computer device is also provided, which includes a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0209] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0210] In one embodiment, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0211] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0212] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0213] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A workpiece measurement method, characterized in that, Applied to a workpiece measurement system, the workpiece measurement system at least includes a depth sensor and a mobile robot, and an end effector of the mobile robot is connected to the depth sensor; the method includes: When the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to a workpiece to be measured through the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; Converting the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; Converting the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table to obtain second point cloud data; Converting the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; Detecting position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

2. The method according to claim 1, wherein The converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot includes: Obtaining a first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot; Converting the initial point cloud data from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship; The converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table includes: Obtaining a second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; Converting the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

3. The method according to claim 1, characterized in that The converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured includes: Detecting the workpiece type of the workpiece to be measured; Generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type; Converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

4. The method according to claim 1, wherein Before the converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, the method further includes: Detecting whether all contour points of the workpiece to be measured are included in the first point cloud data; When all contour points of the workpiece to be measured are not included in the first point cloud data, sending a movement instruction to the mobile robot, wherein updated detection area information is carried in the movement instruction, and the movement instruction is used to control the mobile robot to move the depth sensor to a detection area corresponding to the updated detection area information; Returning to the step of detecting the initial point cloud data corresponding to the workpiece to be measured through the depth sensor until it is detected that all contour points of the workpiece to be measured are included in the first point cloud data.

5. The method according to claim 1, characterized in that, Detecting the position information of a plurality of contour points in the workpiece to be measured from the target point cloud data includes: For each contour point in the workpiece to be measured, obtaining the theoretical position information of the contour point and a preset contour point detection area corresponding to the contour point; Detecting target area point cloud data matching the preset contour point detection area from the target point cloud data; Performing plane fitting on the target area point cloud data according to the theoretical position information of the contour point to obtain a target plane; Projecting the theoretical position information of the contour point onto the target plane to obtain the position information of the contour point.

6. The method according to claim 5, wherein The performing plane fitting on the target area point cloud data according to the theoretical position information of the contour point to obtain a target plane includes: Querying a reference point from the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset first distance threshold, and taking the distance between the reference point and the contour point as a reference distance; Querying a target point cloud set from the target point cloud data whose distance from the theoretical position information of the contour point is less than a preset proportion of the reference distance; Performing plane fitting on the target point cloud set to obtain a target plane.

7. The method according to claim 1, wherein The mobile robot includes at least a first mobile robot and a second mobile robot, and an end effector of the first mobile robot is connected to the depth sensor. The method further includes: When the first mobile robot moves the depth sensor to a preset sensor detection area, obtaining initial point cloud data of the workpiece to be measured detected by the depth sensor; Converting the initial point cloud data from the sensor coordinate system of the sensor to the first robot coordinate system corresponding to the first mobile robot to obtain third point cloud data; Converting the third point cloud data from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain fourth point cloud data; Converting the third point cloud data from the second robot coordinate system to the processing coordinate system to obtain fifth point cloud data; Converting the fifth point cloud data from the processing coordinate system to the workpiece coordinate system to obtain sixth point cloud data; Detecting the position information of a plurality of contour points in the workpiece to be measured from the sixth point cloud data.

8. A workpiece measuring device, characterized in that, Applied to a workpiece measurement system, the workpiece measurement system includes at least a depth sensor and a mobile robot, and an end effector of the mobile robot is connected to the depth sensor. The device includes: An initial surface point cloud acquisition module, configured to, when the mobile robot moves the depth sensor to a preset sensor detection area, detect initial point cloud data corresponding to the workpiece to be measured through the depth sensor, where the workpiece to be measured is placed on a workpiece table; A first pose conversion module, configured to convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data; A second pose conversion module, configured to convert the first point cloud data from the robot coordinate system to the processing coordinate system corresponding to the workpiece table to obtain second point cloud data; The third pose conversion module is used to convert the second point cloud data from the processing coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, so as to obtain target point cloud data; The contour point measurement module is used to detect the position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

9. A workpiece measurement system, characterized in that, The system at least includes: A depth sensor, which is used to detect the point cloud data of the workpiece to be measured; A mobile robot, and the end effector of the mobile robot is connected to the depth sensor; A controller, and the controller is used for: When the mobile robot moves the depth sensor to a preset sensor detection area, detecting the initial point cloud data corresponding to the workpiece to be measured through the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; Converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data; Converting the first point cloud data from the robot coordinate system to the processing coordinate system corresponding to the workpiece table to obtain second point cloud data; Converting the second point cloud data from the processing coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; Detecting the position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images