Workpiece measurement method, apparatus, system, and readable storage medium

Abstract

The application relates to a workpiece measurement method, device, system and readable storage medium. The method is applied to a workpiece measurement system and comprises the following steps: in the case that a mobile robot moves a depth sensor to a preset sensor detection area, detecting initial point cloud data of a workpiece to be measured by the depth sensor; converting the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; converting the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to a workpiece table to obtain second point cloud data; converting the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; and detecting position information of a plurality of contour points in the workpiece to be measured from the target point cloud data. The method can accurately measure the contour points in the workpiece.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing, in particular to a workpiece measurement method, device, system and readable storage medium. BACKGROUND

[0002] Workpiece measurement technology is widely used in the fields of vehicle manufacturing, aerospace, precision instruments, etc. For example, in vehicle manufacturing, the geometry and size deviation of workpieces in vehicles can be detected through workpiece measurement technology to ensure the assembly quality and appearance quality of the vehicle body.

[0003] In the traditional technology, the workpiece measurement method mainly includes: using contact measurement to accurately obtain the shape data of the actual workpiece surface, and comparing the shape data of the actual workpiece surface with the feature points of the workpiece theoretical model or fixed position to obtain the numerical parameters of the feature points. Further, based on the numerical parameters of the feature points, it can be judged whether the workpiece is qualified.

[0004] However, the current workpiece measurement method has the problem of inaccuracy. SUMMARY

[0005] Therefore, it is necessary to provide an accurate workpiece measurement method, device, system, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0006] In a first aspect, the present application provides a workpiece measurement method applied to a workpiece measurement system, the workpiece measurement system comprising at least a depth sensor and a mobile robot, the end effector of the mobile robot being connected with the depth sensor; the method comprising:

[0007] In the case that the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to a workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table;

[0008] Converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data;

[0009] Converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data;

[0010] Converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data;

[0011] Detecting the position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

[0012] In one of the embodiments, the initial point cloud data is converted from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot, including:

[0013] Obtaining a first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot;

[0014] Converting the initial point cloud data from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship;

[0015] Converting the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the worktable, including:

[0016] Obtaining a second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the worktable;

[0017] Converting the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

[0018] In one of the embodiments, the second point cloud data is converted from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, including:

[0019] Detecting the workpiece type of the workpiece to be measured;

[0020] Based on the workpiece type, generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured;

[0021] Converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0022] In one of the embodiments, before converting the initial point cloud data from the robot coordinate system to the machining coordinate system corresponding to the worktable, the method further includes:

[0023] Detecting whether all contour points of the workpiece to be measured are contained in the first point cloud data;

[0024] In the case that all contour points of the workpiece to be measured are not contained in the first point cloud data, sending a moving instruction to the mobile robot, wherein the moving instruction carries updated detection area information, and the moving instruction is used to control the mobile robot to move the depth sensor to a detection area corresponding to the updated detection area information;

[0025] Returning to the step of obtaining the initial point cloud data corresponding to the workpiece to be measured by the depth sensor until all contour points of the workpiece to be measured are detected in the first point cloud data.

[0026] In one of the embodiments, from the target point cloud data, the position information of the plurality of contour points in the workpiece to be measured is detected, including:

[0027] For each profile point in the workpiece to be measured, the profile point corresponding to the profile point theoretical position information and the preset profile point detection region are obtained;

[0028] From the target point cloud data, the target region point cloud data matching the preset profile point detection region is detected;

[0029] According to the profile point theoretical position information, the target region point cloud data is plane fitted to obtain the target plane;

[0030] The profile point theoretical position information is projected to the target plane to obtain the position information of the profile point.

[0031] In one of the embodiments, according to the profile point theoretical position information, the target region point cloud is plane fitted to obtain the target plane, comprising:

[0032] From the target point cloud data, the reference point with a distance less than a preset first distance threshold from the profile point theoretical position information is queried, and the distance between the reference point and the profile point is taken as the reference distance;

[0033] From the target point cloud data, a target point cloud set with a distance less than a preset proportion of the reference distance from the profile point theoretical position information is queried;

[0034] The target point cloud set is plane fitted to obtain the target plane.

[0035] In one of the embodiments, the mobile robot at least includes a first mobile robot and a second mobile robot, the end effector of the first mobile robot is connected with the depth sensor, and the method further comprises:

[0036] In the case that the first mobile robot moves the depth sensor to the preset sensor detection region, the initial point cloud data of the workpiece to be measured detected by the depth sensor is obtained;

[0037] The initial point cloud data is converted from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot to obtain the third point cloud data;

[0038] The third point cloud data is converted from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain the fourth point cloud data;

[0039] The third point cloud data is converted from the second robot coordinate system to the machining coordinate system to obtain the fifth point cloud data;

[0040] The fifth point cloud data is converted from the machining coordinate system to the workpiece coordinate system to obtain the sixth point cloud data;

[0041] Detect position information of a plurality of profile points in the workpiece to be measured from the sixth point cloud data.

[0042] In a second aspect, the present application also provides a workpiece measurement device applied to a workpiece measurement system, the workpiece measurement system at least comprising a depth sensor and a mobile robot, an end effector of the mobile robot being connected with the depth sensor, and the device comprising:

[0043] An initial surface point cloud acquisition module is configured to, in a case where the mobile robot moves the depth sensor to a preset sensor detection area, detect initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table.

[0044] A first pose conversion module is configured to convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data.

[0045] A second pose conversion module is configured to convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table to obtain second point cloud data.

[0046] A third pose conversion module is configured to convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data.

[0047] A profile point measurement module is configured to detect position information of a plurality of profile points in the workpiece to be measured from the target point cloud data.

[0048] In a third aspect, the present application also provides a workpiece measurement system, the system at least comprising:

[0049] A depth sensor is configured to detect point cloud data of a workpiece to be measured.

[0050] A mobile robot, an end effector of the mobile robot being connected with the depth sensor.

[0051] A controller is configured to:

[0052] In a case where the mobile robot moves the depth sensor to a preset sensor detection area, the controller is configured to detect initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table.

[0053] The controller is configured to convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data.

[0054] The controller is configured to convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table to obtain second point cloud data.

[0055] convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data;

[0056] detect position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

[0057] In a fourth aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0058] In the case that the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table;

[0059] convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot, to obtain first point cloud data;

[0060] convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table, to obtain second point cloud data;

[0061] convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data;

[0062] detect position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

[0063] In a fifth aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0064] In the case that the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table;

[0065] convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot, to obtain first point cloud data;

[0066] convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table, to obtain second point cloud data;

[0067] convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data;

[0068] detect position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

[0069] In a sixth aspect, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0070] In the case where the mobile robot moves the depth sensor to the preset sensor detection area, the initial point cloud data corresponding to the workpiece to be measured is detected by the depth sensor, wherein the workpiece to be measured is placed on the workpiece table;

[0071] The initial point cloud data is converted from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, to obtain first point cloud data;

[0072] The first point cloud data is converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, to obtain second point cloud data;

[0073] The second point cloud data is converted from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data;

[0074] The position information of the plurality of contour points in the workpiece to be measured is detected from the target point cloud data.

[0075] The workpiece measurement method, device, system, computer equipment, computer readable storage medium and computer program product described above, at present, the method of measuring the workpiece by using the contact type is limited in the workpiece area for each measurement, and the workpiece to be measured cannot be measured comprehensively, and for a complex workpiece, the probe cannot touch, and the workpiece to be measured cannot be accurately measured. At the same time, when the shape data of the actual workpiece surface and the workpiece theoretical model are compared, since the shape data of the workpiece surface and the workpiece theoretical model need to be fitted and matched, the matching effect is poor in the case where the workpiece to be measured is a simple workpiece without a characteristic surface, and the deviation between the actual workpiece and the theoretical workpiece model is large, which easily leads to a large deviation of the measurement result. The present application is applied to a workpiece measurement system, and all areas of the workpiece can be completely measured by the depth sensor once in the whole process, and the point cloud data detected by the depth sensor has higher accuracy. Therefore, the coordinate system conversion between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece table and the workpiece coordinate system of the workpiece to be measured is used for contour point extraction, without the comparison of the characteristic points with the workpiece theoretical model, the overall appearance of the workpiece can be completely and accurately reflected, and therefore, the workpiece measurement by using the above scheme is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained on the basis of these drawings without creative labor.

[0077] Figure 1 An application environment diagram of the workpiece measurement method in an embodiment;

[0078] Figure 2 A flowchart of the workpiece measurement method in an embodiment;

[0079] Figure 3 A flowchart of the workpiece measurement method in another embodiment;

[0080] Figure 4 An actual architecture diagram of the workpiece measurement system in a specific application embodiment;

[0081] Figure 5 A structural block diagram of the workpiece measurement device in an embodiment;

[0082] Figure 6 An internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0083] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application, and are not used to limit the present application.

[0084] The workpiece measurement technology is widely used in the fields of vehicle manufacturing, aerospace, precision instruments, etc. For example, in vehicle manufacturing, the workpiece measurement technology can be used to detect the geometric shape and size deviation of the workpiece in the vehicle, so as to ensure the assembly quality and appearance quality of the vehicle body.

[0085] In the traditional technology, the workpiece measurement method mainly includes: using contact measurement to accurately obtain the shape data of the actual workpiece surface, and comparing the shape data of the actual workpiece surface with the characteristic points of the workpiece theoretical model or fixed position, to obtain the numerical parameters of the characteristic points. Further, based on the numerical parameters of the characteristic points, it can be judged whether the workpiece is qualified.

[0086] Specifically, the three-dimensional model is used for data conversion to construct a hypothetical measurement surface, and a three-coordinate machine is used to perform progressive contact detection measurement along the normal direction of the hypothetical measurement surface using a rod-shaped probe; or a workpiece to be measured is fixed in a workpiece coordinate system set in advance using a specific fixture, and the workpiece is measured using a fixed or mobile device such as a micrometer, a go-no-go gauge, or a laser range finder to obtain measurement data of the surface of the workpiece; or the scanned three-dimensional data is compared with the theoretical three-dimensional model to obtain the profile point measurement data of the workpiece by constructing the difference between the scanned three-dimensional point cloud and the theoretical surface model.

[0087] However, the method of using a three-coordinate rod-shaped probe to perform progressive contact detection measurement along the normal direction of the hypothetical measurement surface in the prior art requires the use of a three-coordinate machine, the hardware equipment is expensive, and only a part of the measurement data of the workpiece can be obtained at one time. Complex programs need to be written to assist in dense multi-point measurement, and the probe cannot reach some complex workpieces, so the measurement of the workpiece cannot be accurately performed. The method of detecting and measuring the surface of the workpiece by using a specific fixture and a measuring tool requires the use of a specific fixture to fix the workpiece in a fixed workpiece coordinate system, and a specific measuring tool is used for measurement. The measurement effect of the workpiece to be measured with a large deviation from the theoretical profile is poor, and only the pass or fail result can be provided, and accurate quantitative measurement indicators cannot be given. At the same time, the hardware equipment is only suitable for a single workpiece. The method of comparing the scanned three-dimensional data with the theoretical three-dimensional model to obtain the difference of the actual workpiece surface requires matching of the point cloud data and the theoretical surface model. The matching effect is poor for simple workpieces with no features, large deviations between the actual workpiece and the theoretical workpiece model, and the measured points deviate from the theoretical values. At the same time, the method cannot accurately feedback the overall workpiece offset caused by the deformation of the reference point of the fixed workpiece.

[0088] Therefore, this application proposes a workpiece measurement method, applied to a workpiece measurement system. The workpiece measurement system includes at least a depth sensor and a mobile robot, with the end effector of the mobile robot connected to the depth sensor. The method includes: when the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data; converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data; converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; and detecting the position information of multiple contour points in the workpiece to be measured from the target point cloud data. Throughout the process, the depth sensor can completely measure all areas of the workpiece in one measurement, and the depth sensor detects point cloud data with higher accuracy. Therefore, by using coordinate system transformation between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece stage, and the workpiece coordinate system of the workpiece to be measured for contour point extraction, the overall shape of the workpiece can be completely and accurately reflected. When the workpiece to be measured is a simple workpiece without feature surfaces, or when there is a large deviation between the actual workpiece and the theoretical workpiece model, since feature points are not used for measurement, but coordinate system transformation is used, there is no need to compare the feature points with the theoretical workpiece model to obtain accurate target point cloud data without offset. Therefore, the above scheme is more accurate for workpiece measurement.

[0089] The workpiece measurement method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is the workpiece measurement system 1000. The workpiece measurement system 1000 includes at least a depth sensor 102, a mobile robot 104, and a controller 108. The mobile robot 104 is connected to the depth sensor 102, which is used to detect the point cloud data of the workpiece 110 to be measured. The workpiece is placed on the workpiece stage 106. Figure 1 Taking the example that the workpiece stage 106 can be fixed on the base of the mobile robot 104, the workpiece stage 106 can also be placed outside the mobile robot 104, and there is no limitation here.

[0090] In a case where the controller 108 controls the mobile robot 104 to move the depth sensor 102 to a preset sensor detection area, initial point cloud data corresponding to the workpiece 110 to be measured is detected by the depth sensor 102; the initial point cloud data is converted from a sensor coordinate system corresponding to the depth sensor 102 to a robot coordinate system corresponding to the mobile robot 104, to obtain first point cloud data; the initial point cloud data is converted from the robot coordinate system to a machining coordinate system corresponding to the workpiece table 106, to obtain second point cloud data; the second point cloud data is converted from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece 110 to be measured, to obtain target point cloud data; and position information of a plurality of contour points in the workpiece 110 to be measured is detected from the target point cloud data.

[0091] In one exemplary embodiment, as shown in Figure 2 A workpiece measurement method is provided, applied to a workpiece measurement system 1000, the system at least including a depth sensor and a mobile robot, an end effector of the mobile robot being connected with the depth sensor; the method including:

[0092] S100, in a case where the mobile robot moves the depth sensor to a preset sensor detection area, initial point cloud data corresponding to a workpiece to be measured is detected by the depth sensor.

[0093] The workpiece to be measured is placed on a workpiece table. The end effector of the mobile robot is connected with the depth sensor. The workpiece table can be fixed on a base of the mobile robot, or can be placed outside the mobile robot, which is not limited herein. The depth sensor can be a sensor for detecting point cloud information, such as a three-dimensional scanning camera.

[0094] Specifically, the workpiece measurement system further includes a controller, the controller controls the mobile robot to drive the depth sensor to run to a preset detection area along a predetermined trajectory through the end effector. Generally, the preset detection area refers to an area in which all contour point information on the workpiece to be measured can be detected. For example, if there is a contour point on the workpiece to be measured, the end effector of the mobile robot will drive the depth sensor according to the programmed robot program to run to a spatial position where the contour point can be detected. These predetermined positions are taught in advance in the mobile robot, that is, the mobile robot will run to this position each time.

[0095] After the mobile robot moves the depth sensor to the preset sensor detection area, the controller sends a workpiece trigger signal to the depth sensor. After receiving the workpiece trigger signal, the depth sensor detects the workpiece to be measured and obtains the surface topography three-dimensional scanning data of the workpiece to be measured within the field of view, that is, the initial point cloud data, and feeds back the initial point cloud data to the controller. In practical applications, the surface topography of the workpiece to be measured can be a free-form surface, a plane, or other geometric three-dimensional scanning space structures.

[0096] Further, using the depth sensor in cooperation with the mobile robot can perform large-range workpiece contour point measurement and analysis, and a plurality of contour point data of the workpiece to be measured can be obtained at one time. Since the robot is flexible in action, complete measurement of the workpiece can be realized through the depth sensor even if the workpiece has a complex space structure, thereby improving the measurement accuracy and efficiency of the contour points in the workpiece.

[0097] In an exemplary embodiment, the controller is in communication connection with the mobile robot and the depth sensor through connection cables. The controller sends a movement trajectory instruction to the mobile robot through the connection cables, and the mobile robot moves the depth sensor to the preset sensor detection area after receiving the movement trajectory instruction. The controller sends a workpiece trigger signal to the depth sensor through the connection cables after confirming that the depth sensor reaches the preset sensor detection area, and the depth sensor detects the initial point cloud data of the workpiece to be measured after receiving the workpiece trigger signal.

[0098] S200, converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data.

[0099] The robot coordinate system is actually the base coordinate system corresponding to the mobile robot.

[0100] Specifically, the depth sensor is connected to the end effector of the mobile robot, that is, the relationship between the depth sensor and the mobile robot is an "eye on the hand" relationship. Therefore, the initial point cloud data in the sensor coordinate system corresponding to the depth sensor can be mapped to the base coordinate system corresponding to the mobile robot to obtain the first point cloud data of the workpiece to be measured in the robot coordinate system.

[0101] S300, converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data.

[0102] The workpiece table is used to fix and position the workpiece to ensure its stability and accuracy during machining or measurement. Different workpiece tables are needed to adapt to the shape and size of different workpieces.

[0103] Specifically, the workpiece table can be mounted on the base of the mobile robot, or can be mounted at a position outside the mobile robot. In order to obtain more accurate point cloud data of the workpiece to be measured, it is necessary to convert the initial point cloud data from the robot coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured. Therefore, the initial point cloud data can be first converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, to obtain second point cloud data in the machining coordinate system.

[0104] In addition, compared with the method of detecting the surface of the workpiece by a specific fixed clamp and a measuring tool in the prior art, the workpiece table for fixing the workpiece in the present application can have various forms, and can be well compatible with different types of workpieces of different shapes. Moreover, thanks to the non-contact characteristics of the scanning probe, the workpiece to be measured with a large deviation from the theoretical profile can still have good measurement effect, and can give quantitative measurement indexes.

[0105] S400, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data.

[0106] Specifically, after the workpiece to be measured is placed on the machining table, the second point cloud data in the machining coordinate system is obtained, and then the second point cloud data needs to be converted from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data in the workpiece coordinate system.

[0107] S500, detecting position information of a plurality of profile points in the workpiece to be measured from the target point cloud data.

[0108] Specifically, the position information of a plurality of profile points in the workpiece to be measured is detected from the target point cloud data by a surface profile point extraction measurement algorithm. Further, based on the position information of all the profile points, a measurement report and an analysis report of the profile points in the workpiece to be measured are generated. In some other embodiments, the present application is not limited to detecting the position information of the profile points, but can also detect the position information of other features such as holes and angles.

[0109] In an exemplary embodiment, based on the position information of all the profile points, the measurement report and the analysis report of the profile points in the workpiece to be measured are generated, including: obtaining theoretical position information of all the profile points and a customer-set or software-default allowable position error tolerance value, and based on the position information of all the profile points, the theoretical position information of all the profile points, and the customer-set or software-default allowable position error tolerance value, evaluating whether each profile point of the workpiece to be measured is qualified, and outputting evaluation results of all the profile points to a user.

[0110] The position error value between the position information and the theoretical position information is obtained for each profile point, and it is determined whether the position error value is greater than the allowable position error tolerance value. If the position error value is greater than the allowable position error tolerance value, the profile points of the workpiece to be measured are unqualified. If the position error value is less than or equal to the allowable position error tolerance value, the profile points of the workpiece to be measured are qualified.

[0111] In the workpiece measurement method, the contact measurement method is currently used to measure the workpiece, and the workpiece area measured each time is limited, and the workpiece to be measured cannot be measured comprehensively. For complex workpieces, the probe cannot be touched, and the workpiece to be measured cannot be accurately measured. When the shape data of the actual workpiece surface and the workpiece theoretical model are compared, the shape data of the workpiece surface and the workpiece theoretical model need to be fitted and matched. Therefore, in the case that the workpiece to be measured is a simple featureless surface workpiece and the deviation between the actual workpiece and the theoretical workpiece model is large, the matching effect is poor, which easily leads to large deviation of the measurement result. The application is applied to a workpiece measurement system. In the whole process, all areas of the workpiece can be measured at one time by the depth sensor, and the point cloud data obtained by the depth sensor has higher accuracy. Therefore, the profile points are extracted by coordinate system conversion between the robot coordinate system of the mobile robot, the sensor coordinate system of the depth sensor, the machining coordinate system of the workpiece table, and the workpiece coordinate system of the workpiece to be measured, which can accurately reflect the overall appearance of the workpiece. In the case that the workpiece to be measured is a simple featureless surface workpiece and the deviation between the actual workpiece and the theoretical workpiece model is large, the feature points are not used for measurement, but the coordinate system conversion is used. Therefore, the accurate target point cloud data can be obtained without comparing the feature points with the theoretical workpiece model, and the deviation is not generated. Therefore, the workpiece measurement is more accurate by using the above scheme.

[0112] In one exemplary embodiment, the initial point cloud data is converted from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, comprising:

[0113] The first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot is obtained. The initial point cloud data is converted from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship.

[0114] Specifically, the current pose information of the mobile robot, i.e., the first pose conversion relationship between the end coordinate system corresponding to the mobile robot and the base coordinate system corresponding to the mobile robot, is acquired, and since the depth sensor is on the end effector of the robot, the first pose conversion relationship is also the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot, and the initial point cloud data is transferred to the robot coordinate system according to the first pose conversion relationship.

[0115] In an exemplary embodiment, the first pose conversion relationship includes the coordinate conversion parameters of the end effector of the mobile robot, and the initial point cloud data is transferred to the robot coordinate system through the coordinate conversion parameters of the end effector of the mobile robot and the joint angular motion angles.

[0116] Let the initial point cloud data be , the coordinate conversion parameters of the end effector of the mobile robot be [R|t], and the first point cloud data be . .

[0117] In an exemplary embodiment, the first pose conversion relationship can also be calculated through the motion parameters of the mobile robot and the joint angular motion angles Ji of the mobile robot. Specifically, the motion parameters of the mobile robot are acquired, such as determining the link parameters (such as length, torsion angle, etc.) and joint types according to the structural characteristics of the mobile robot, and the kinematic model of the mobile robot is established through the motion parameters of the mobile robot using the Denavit-Hartenberg (D-H) coordinate system parameter method or other methods; the current motion angles of the joints of the mobile robot are read through the sensor or the controller, and the first pose conversion relationship from the sensor coordinate system to the robot coordinate system is gradually calculated using matrix multiplication and other mathematical operations according to the kinematic model and the joint angular motion angles.

[0118] Converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table includes:

[0119] Acquiring the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; and converting the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

[0120] Specifically, after completing the scanning action of all contour points, the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table, which is pre-calibrated, is acquired to convert the first point cloud data from the robot coordinate system to the machining coordinate system, wherein the second pose conversion relationship is a fixed conversion matrix.

[0121] Let the second pose conversion relationship be K0, and the first point cloud data be , the second point cloud data .

[0122] In the above embodiments, by obtaining the first pose conversion relationship between the sensor coordinate system corresponding to the depth sensor and the robot coordinate system corresponding to the mobile robot, and the second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the worktable, the initial point cloud data can be accurately converted from the sensor coordinate system to the robot coordinate system, and then from the robot coordinate system to the machining coordinate system.

[0123] In an exemplary embodiment, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured comprises:

[0124] detecting the workpiece type of the workpiece to be measured; generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type; and converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0125] In an exemplary embodiment, the workpiece type of the workpiece to be measured includes parts of different shapes (such as planes, curved surfaces, complex geometric bodies), sizes, materials, and machining requirements, and each workpiece type of the workpiece to be measured needs a specific coordinate system matching the workpiece type for positioning when machining, measuring, or assembling, that is, the workpiece coordinate system.

[0126] Specifically, the workpiece type of the workpiece to be measured is different, and the pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured is also different. When the workpiece to be measured is placed on the worktable, a third pose conversion relationship between the workpiece coordinate system and the machining coordinate system needs to be established according to the workpiece type of the workpiece to be measured, and the second point cloud data needs to be converted from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0127] More specifically, a matching database between the workpiece type and the third pose conversion relationship is obtained, and the third pose conversion relationship matching the workpiece type is queried from the matching database according to the workpiece type of the workpiece to be measured. In actual application, the third pose conversion relationship is a fixed conversion matrix matching the workpiece type of the workpiece to be measured , the second point cloud data , the target point cloud data .

[0128] In an exemplary embodiment, the pose conversion relationship between the workpiece coordinate system and the machining coordinate system depends on the shape, size, placement method of the workpiece to be measured, and the design of the worktable.

[0129] Influence of geometry: When the workpiece to be measured is a planar workpiece, simple translation and rotation can achieve coordinate system conversion; when the workpiece to be measured is a curved surface or a complex geometric workpiece, more complex transformations such as nonlinear transformation or feature-based transformation are needed.

[0130] Influence of size and placement: The size and placement of the workpiece to be measured can affect its position and orientation in the workpiece table, thus affecting the pose conversion relationship. For example, large workpieces may need to consider the deformation caused by gravity, while small workpieces may be more susceptible to the accuracy of the workpiece table.

[0131] Influence of workpiece table design: The design of the workpiece table determines the fixation and positioning accuracy of the workpiece in the workpiece table. Different workpiece table designs will result in different pose conversion relationships.

[0132] For example, when the workpiece to be measured is a planar workpiece with a rectangular shape, the workpiece is placed on a planar workpiece table and fixed by two positioning pins and a clamping device. In this case, the pose conversion relationship considers translation and rotation. When the workpiece to be measured is a curved surface workpiece with a spherical shape, the workpiece is placed on a specially designed curved surface workpiece table to ensure its stability and accuracy. In this case, the pose conversion relationship needs to consider factors such as the curvature of the curved surface, the position and orientation of the workpiece in the workpiece table, etc.

[0133] In the above embodiments, the workpiece type of the workpiece to be measured is set as an index to obtain the pose conversion matrix between the machining coordinate system and the workpiece coordinate system, which can accurately convert the second point cloud data from the machining coordinate system to the workpiece coordinate system.

[0134] In an exemplary embodiment, the method of obtaining the pose conversion relationship between the machining coordinate system and the workpiece coordinate system by indexing the workpiece type mentioned in the above embodiments can also be replaced by using the obtained point cloud of the workpiece to be measured to establish the workpiece coordinate system, such as establishing the workpiece coordinate system through model matching.

[0135] In an exemplary embodiment, the method of obtaining the pose conversion relationship between the machining coordinate system and the workpiece coordinate system by indexing the workpiece type mentioned in the above embodiments can also be replaced by extracting the workpiece itself reference point position to construct the workpiece coordinate system for point cloud data coordinate conversion.

[0136] In an exemplary embodiment, as shown in Figure 3 Before S300, the method further includes:

[0137] S220, detecting whether the first point cloud data contains all the contour points of the workpiece to be measured.

[0138] S240, in the case that the first point cloud data does not contain all the contour points of the workpiece to be measured, a movement instruction is sent to the mobile robot, wherein the updated detection area information is carried in the movement instruction, and the movement instruction is used to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information.

[0139] S260, returning to the step of detecting the initial point cloud data of the workpiece to be measured by the depth sensor until all the contour points of the workpiece to be measured are contained in the first point cloud data.

[0140] Specifically, it is checked whether the scanning of all the contour points of the workpiece to be measured is completed, if not, new data in the new detection area is collected, and if yes, the first point cloud data is converted from the robot coordinate system to the machining coordinate system corresponding to the workpiece table.

[0141] In detail, in the case that the first point cloud data does not contain all the contour points of the workpiece to be measured, the initial point cloud data of the workpiece to be measured needs to be detected in the new detection area, therefore, the movement instruction is generated according to the other detection area information preset in advance, and the movement instruction is sent to the mobile robot, so that the mobile robot moves the depth sensor to the detection area corresponding to the updated detection area information.

[0142] In the case that the mobile robot moves the depth sensor to the new detection area, the initial point cloud data of the workpiece to be measured detected by the depth sensor in the new detection area is acquired; and the initial point cloud data is converted from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, to obtain the first point cloud data.

[0143] Then it is re-detected whether all the contour points of the workpiece to be measured are contained in the first point cloud data, and the above steps are repeated until all the contour points of the workpiece to be measured are contained in the first point cloud data.

[0144] In the above embodiment, by detecting whether all the contour points of the workpiece to be measured are contained in the first point cloud data before converting the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, that is, after converting the first point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, the first point cloud data of the workpiece to be measured can be more comprehensively acquired.

[0145] In one embodiment, the step of detecting whether all the contour points of the workpiece to be measured are contained in the initial point cloud data can be set to be executed before converting the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot, and after acquiring the initial point cloud data of the workpiece to be measured detected by the depth sensor.

[0146] In one embodiment, after converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured, it is detected whether all contour points of the workpiece to be measured are contained in the target point cloud data. If not, a movement instruction is sent to the mobile robot to control the mobile robot to move the depth sensor to the detection area corresponding to the updated detection area information, and the acquisition of the initial point cloud data of the workpiece to be measured is re-performed.

[0147] In one exemplary embodiment, the position information of the plurality of contour points in the workpiece to be measured is detected from the target point cloud data, including:

[0148] For each contour point in the workpiece to be measured, the contour point theoretical position information corresponding to the contour point and the preset contour point detection area are acquired; the target area point cloud data matching the preset contour point detection area is detected from the target point cloud data; the target plane is obtained by performing plane fitting on the target area point cloud data according to the contour point theoretical position information; and the position information of the contour point is obtained by projecting the contour point theoretical position information onto the target plane.

[0149] Specifically, a contour point list [i, O in ] is preset, where i is a contour point index, O in is the contour point theoretical position information corresponding to each contour point index, and O in contains the spatial coordinate information of [Xn, Yn, Zn], at this time, the contour point theoretical position information O in corresponding to the contour point index i is queried from the preset contour point list by using the contour point index i.

[0150] Further, a contour point detection area list [i, pt i ] can also be preset, where i is a contour point index, pt i is the contour point theoretical position information corresponding to each contour point index, and a series of spatial points defined in the space that surround the spatial area around the contour point i, here pt i surrounded space area is the ROI (Region of Interest, region of interest) contour point detection area , which is used for segmentation to obtain the target area point cloud data Proi-i that needs to be extracted for contour point extraction, so that the target area point cloud data matching the preset contour point detection area can be detected from the target point cloud data . .

[0151] Further, the target region point cloud data can be fitted with a plane according to the contour point theoretical position information, to obtain a target plane L. The plane fitting method can be RANSAC (Random Sample Consensus) plane fitting. RANSAC plane fitting is a robust model estimation method, and is particularly suitable for processing data containing a large amount of noise and outliers. The basic idea is to estimate the model parameters from the data by means of random sampling and model verification. In plane fitting, RANSAC randomly selects three points (because three non-collinear points can determine a plane), and then calculates the plane model determined by the three points. Then, RANSAC calculates the distance of all other points to the plane, and determines whether the points conform to the plane model according to a preset threshold. This process is repeated multiple times, and finally the plane model with the most conforming points is selected as the final result.

[0152] The contour point theoretical position information is further projected along the normal vector of the target plane L to the target plane, to obtain actual position information of the contour point.

[0153] In some other embodiments, the method of projecting the nominal point to the fitted target plane by the plane normal vector to obtain the spatial coordinates of the contour point can also be modified to project the nominal point to the fitted target plane along any specified spatial vector.

[0154] In the above embodiments, the accuracy and stability of the actual value are improved by using a piece of point cloud, that is, a piece of point cloud around the nearest point to the contour point. Because direct use of the point cloud for single point extraction cannot accurately and stably feedback the distance or spatial position due to the error of the sensor itself, but using a piece of point cloud to fit the target plane and then projecting and mapping the point to the plane can ensure stable, accurate and interference-free calculation of the spatial coordinates of the required contour point, thereby improving the accuracy of the contour point measurement in the workpiece.

[0155] In an exemplary embodiment, the position information of the plurality of contour points in the workpiece to be measured is detected from the target point cloud data, which further includes: acquiring contour point theoretical position information corresponding to each contour point of the workpiece to be measured; fitting a plane according to the contour point theoretical position information, to obtain a target plane; and projecting the contour point theoretical position information to the target plane, to obtain the position information of the contour point. That is, the acquisition step of the preset contour point detection region can be cancelled, and the contour point position information can be directly detected in the target point cloud data.

[0156] In an exemplary embodiment, the target region point cloud is fitted with a plane according to the contour point theoretical position information, to obtain a target plane, which includes:

[0157] query a reference point from the target point cloud data, where a distance between the reference point and the contour point theoretical position information is less than a preset first distance threshold, and take the distance between the reference point and the contour point as a reference distance; query a target point cloud set from the target point cloud data, where a distance between each point in the target point cloud set and the contour point theoretical position information is less than a preset proportion of the reference distance; and perform plane fitting on the target point cloud set to obtain a target plane.

[0158] Specifically, the reference point is queried from the target point cloud data, where a distance between the reference point and the contour point theoretical position information is less than the preset first distance threshold, for example, a point with the smallest distance to the contour point theoretical position information is taken as the reference point, and the distance between the reference point and the contour point is taken as the reference distance, and specifically, the contour point theoretical position information is (Xn, Yn, Zn), and the reference point a (ax, ay, az) and the reference distance T can be represented as:

[0159]

[0160]

[0161] wherein, The target point cloud data is obtained.

[0162] All target point cloud sets with a distance between each point in the target point cloud set and the contour point theoretical position information less than a preset proportion K of the reference distance are searched from the target point cloud data, where K is set according to historical experience, and is generally about 1.2-1.5, that is, the distance is slightly greater than the distance of the nearest reference point, so as to realize that the points around the reference point can be contained.

[0163] The contour point theoretical position information is (Xn, Yn, Zn), and each point pt[ptx, pty, ptz] in the target point cloud set needs to satisfy the following expression:

[0164]

[0165] Finally, the target point cloud set is subjected to plane fitting to obtain a target plane L[A, B, C, D], where [A, B, C, D] are four coefficients in a general plane equation.

[0166] In the embodiment, the reference point is queried from the target point cloud data, where a distance between the reference point and the contour point theoretical position information is less than the preset first distance threshold, and the distance between the reference point and the contour point is taken as the reference distance, the target point cloud set is queried from the target point cloud data, where a distance between each point in the target point cloud set and the contour point theoretical position information is less than a preset proportion of the reference distance, and the target point cloud set can be accurately subjected to plane fitting to obtain the target plane for accurately projecting the contour point.

[0167] In one exemplary embodiment, the method of projecting by fitting a plane to obtain the actual position information of the contour points mentioned in the foregoing embodiments can also cancel the step of fitting a plane and directly select the spatial coordinates of the reference points, such as the nearest points to the contour points, as the actual measurement position information of the contour points.

[0168] In one exemplary embodiment, the mobile robot at least includes a first mobile robot and a second mobile robot, the end effector of the first mobile robot is connected with the depth sensor, and the method further includes:

[0169] In the case where the first mobile robot moves the depth sensor to the preset sensor detection area, the initial point cloud data of the workpiece to be measured detected by the depth sensor is obtained; the initial point cloud data is converted from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot to obtain third point cloud data; the third point cloud data is converted from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot to obtain fourth point cloud data; the third point cloud data is converted from the second robot coordinate system to the machining coordinate system to obtain fifth point cloud data; the fifth point cloud data is converted from the machining coordinate system to the workpiece coordinate system to obtain sixth point cloud data; and the position information of the plurality of contour points in the workpiece to be measured is detected from the sixth point cloud data.

[0170] Specifically, the single mobile robot measurement and coordinate conversion in the present application can also be replaced by a plurality of mobile robots to convert all measurement data to the same coordinate system according to the relative coordinate conversion relationship between the mobile robots.

[0171] More specifically, first, the robot pose conversion relationship between the plurality of mobile robots is calibrated, for example, the pose conversion relationship between the first mobile robot and the second mobile robot can be obtained, and in the case where the first mobile robot moves the depth sensor to the preset sensor detection area, the initial point cloud data of the workpiece to be measured detected by the depth sensor is obtained; after the initial point cloud data is converted from the sensor coordinate system to the first robot coordinate system corresponding to the first mobile robot according to the pose conversion relationship between the sensor coordinate system and the first robot coordinate system to obtain third point cloud data, the third point cloud data is converted from the first robot coordinate system to the second robot coordinate system corresponding to the second mobile robot through the pose conversion relationship between the first mobile robot and the second mobile robot to obtain fourth point cloud data. In actual application, the end effector of the first mobile robot is connected with the depth sensor, and the second mobile robot can be placed between the first mobile robot and the machining table to reduce the situation that the distance between the first mobile robot and the machining table is too far to cause inaccurate pose conversion, in addition, the machining table can also be placed on the base of the second mobile robot.

[0172] Secondly, a pose conversion relationship between the second robot coordinate system and the machining coordinate system is obtained, and third point cloud data is converted from the second robot coordinate system to the machining coordinate system according to the pose conversion relationship between the second robot coordinate system and the machining coordinate system, to obtain fifth point cloud data.

[0173] According to the pose conversion relationship between the machining coordinate system and the workpiece coordinate system, the fifth point cloud data is converted from the machining coordinate system to the workpiece coordinate system, to obtain sixth point cloud data; finally, the position information of the plurality of profile points of the workpiece to be measured is detected from the sixth point cloud data.

[0174] Further, the mobile robot of the present application is not limited to two, and the above is an example of two mobile robots to describe how to implement the workpiece measurement method when the mobile robot is more than one.

[0175] In the above embodiment, by setting a plurality of mobile robots and according to the pose conversion relationship between the plurality of mobile robots, the workpiece measurement can be accurately performed even in the case that the distance between the workpiece table where the workpiece to be measured is placed and the mobile robot connected with the depth sensor is far.

[0176] In an exemplary embodiment, the present application is applied to a workpiece measurement system, as shown in Figure 4 The actual system architecture diagram in a specific application example is shown, which at least includes a controller 1, a depth sensor 2 and a mobile robot 3, the end effector of the mobile robot 3 is connected with the depth sensor 2, the base of the mobile robot 3 is provided with a workpiece table 4, and a workpiece to be measured 5 is placed on the workpiece table 4.

[0177] The controller 1 obtains the surface topography data of the workpiece to be measured based on other structures by using the flexible three-dimensional scanning data acquisition method, and after mapping and converting the three-dimensional scanning data to the workpiece coordinate system of the workpiece to be measured, the actual spatial coordinate values of the profile points of the workpiece to be measured are obtained by cooperating with the surface profile point taking algorithm, so as to output the final measurement data and error analysis report. Specifically, it includes:

[0178] S1, in the case that the mobile robot moves the depth sensor to a preset sensor detection area, the initial point cloud data corresponding to the workpiece to be measured is detected by the depth sensor.

[0179] S2, a first pose conversion relationship between a sensor coordinate system corresponding to the depth sensor and a robot coordinate system corresponding to the mobile robot is obtained; according to the first pose conversion relationship, the initial point cloud data is converted from the sensor coordinate system to the robot coordinate system, to obtain first point cloud data.

[0180] S3, detecting whether all contour points of the workpiece to be measured are contained in the first point cloud data; in the case that all contour points of the workpiece to be measured are not contained in the first point cloud data, sending a moving instruction to the mobile robot, wherein the moving instruction carries updated detection area information, and the moving instruction is used to control the mobile robot to move the depth sensor to a detection area corresponding to the updated detection area information. Returning to the step of detecting the initial point cloud data corresponding to the workpiece to be measured by the depth sensor until all contour points of the workpiece to be measured are detected in the first point cloud data. The step S3 can be adaptively arranged after the next step of S2, or adaptively arranged after the next step of S1, or adaptively arranged after the next step of S4 or S5.

[0181] S4, obtaining a second pose conversion relationship between a robot coordinate system and a machining coordinate system corresponding to the workpiece table; converting the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship to obtain second point cloud data.

[0182] S5, detecting a workpiece type of the workpiece to be measured; generating a third pose conversion relationship between the machining coordinate system and a workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type; converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship to obtain target point cloud data.

[0183] S6, for each contour point of the workpiece to be measured, obtaining contour point theoretical position information corresponding to the contour point and a preset contour point detection area; detecting target area point cloud data matched with the preset contour point detection area from the target point cloud data; querying a reference point from the target point cloud data, wherein a distance between the reference point and the contour point theoretical position information is less than a preset first distance threshold, and a distance between the reference point and the contour point is taken as a reference distance; querying a target point cloud set from the target point cloud data, wherein a distance between the target point cloud set and the contour point theoretical position information is less than a preset proportion of the reference distance; performing plane fitting on the target point cloud set to obtain a target plane; projecting the contour point theoretical position information to the target plane to obtain position information of the contour point.

[0184] S7, comparing the position information of the contour point with the contour point theoretical position information, further outputting an error analysis value as a measurement report and an error analysis report.

[0185] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time but can be executed at different times, and the execution of the steps or stages is not necessarily sequential but can be executed alternately or alternately with at least some of the other steps or steps or stages in other steps.

[0186] Based on the same inventive concept, the embodiments of the present application also provide a workpiece measurement device for implementing the workpiece measurement method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more workpiece measurement device embodiments provided below can refer to the limitations of the workpiece measurement method described above, which will not be repeated here.

[0187] In one exemplary embodiment, as shown in Figure 5 applied to a workpiece measurement system, the system at least includes a depth sensor and a mobile robot, and the end effector of the mobile robot is connected with the depth sensor; the present application provides a workpiece measurement device, comprising: an initial surface point cloud acquisition module 100, a first pose conversion module 200, a second pose conversion module 300, a third pose conversion module 400 and a contour point measurement module 500, wherein:

[0188] The initial surface point cloud acquisition module 100 is configured to acquire initial point cloud data corresponding to a workpiece to be measured by the depth sensor when the mobile robot moves the depth sensor to a preset sensor detection area, wherein the workpiece to be measured is placed on a workpiece table.

[0189] The first pose conversion module 200 is configured to convert the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot to obtain first point cloud data.

[0190] The second pose conversion module 300 is configured to convert the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table to obtain second point cloud data.

[0191] The third pose conversion module 400 is configured to convert the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data.

[0192] The profile point measurement module 500 is configured to detect position information of a plurality of profile points in the workpiece to be measured from the target point cloud data.

[0193] In an embodiment, the first pose conversion module 200 is further configured to obtain a first pose conversion relationship between a sensor coordinate system corresponding to the depth sensor and a robot coordinate system corresponding to the mobile robot; convert the initial point cloud data from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship; and the second pose conversion module 300 is further configured to obtain a second pose conversion relationship between the robot coordinate system and a machining coordinate system corresponding to the workpiece table; convert the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

[0194] In an embodiment, the third pose conversion module 400 is further configured to detect a workpiece type of the workpiece to be measured; generate a third pose conversion relationship between the machining coordinate system and a workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type; and convert the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

[0195] In an embodiment, the workpiece measurement device further comprises a profile point detection module, the profile point detection module is configured to detect whether all profile points of the workpiece to be measured are contained in the first point cloud data; in the case that all profile points of the workpiece to be measured are not contained in the first point cloud data, send a movement instruction to the mobile robot, wherein the movement instruction carries updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to a detection area corresponding to the updated detection area information; return to the step of detecting the initial point cloud data corresponding to the workpiece to be measured by the depth sensor until all profile points of the workpiece to be measured are contained in the first point cloud data.

[0196] In an embodiment, the profile point measurement module 500 is further configured to, for each profile point in the workpiece to be measured, obtain profile point theoretical position information corresponding to the profile point and a preset profile point detection area; detect target area point cloud data matching the preset profile point detection area from the target point cloud data; perform plane fitting on the target area point cloud data according to the profile point theoretical position information to obtain a target plane; and project the profile point theoretical position information to the target plane to obtain position information of the profile point.

[0197] In an embodiment, the profile point measurement module 500 is further configured to query a reference point from the target point cloud data, the distance between the reference point and the profile point theoretical position information is less than a preset first distance threshold, and the distance between the reference point and the profile point is taken as a reference distance; query a target point cloud set from the target point cloud data, the distance between the target point cloud set and the profile point theoretical position information is less than a preset proportion of the reference distance; and perform plane fitting on the target point cloud set to obtain a target plane.

[0198] In one embodiment, the mobile robot at least comprises a first mobile robot and a second mobile robot, an end effector of the first mobile robot is connected with a depth sensor; the workpiece measuring device further comprises a measuring module, the measuring module is configured to: in a case that the first mobile robot moves the depth sensor to a preset sensor detection area, acquire initial point cloud data of a workpiece to be measured detected by the depth sensor; convert the initial point cloud data from a sensor coordinate system to a first robot coordinate system corresponding to the first mobile robot to obtain third point cloud data; convert the third point cloud data from the first robot coordinate system to a second robot coordinate system corresponding to the second mobile robot to obtain fourth point cloud data; convert the third point cloud data from the second robot coordinate system to a machining coordinate system to obtain fifth point cloud data; convert the fifth point cloud data from the machining coordinate system to a workpiece coordinate system to obtain sixth point cloud data; and detect position information of a plurality of contour points in the workpiece to be measured from the sixth point cloud data.

[0199] The above various modules in the workpiece measuring device can be realized by software, hardware and combinations thereof in whole or in part. The above various modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the above various modules.

[0200] In one exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram thereof can be as shown in Figure 6 The computer device comprises a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store initial point cloud data of a workpiece to be measured detected by a depth sensor and other data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a workpiece measuring method.

[0201] Those skilled in the art can understand that Figure 6The structure shown in the figure is a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0202] In one embodiment, the present application also provides a workpiece measurement system, which at least includes:

[0203] a depth sensor, the depth sensor being configured to detect point cloud data of a workpiece to be measured;

[0204] a mobile robot, an end effector of the mobile robot being connected to the depth sensor;

[0205] a controller, the controller being configured to: in a case where the mobile robot moves the depth sensor to a preset sensor detection area, detect initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table to obtain second point cloud data; convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; and detect position information of a plurality of contour points in the workpiece to be measured from the target point cloud data.

[0206] The specific implementation means are the same as those in the workpiece measurement method described above, and will not be described here again.

[0207] In one example, the workpiece table on which the workpiece to be measured is placed can or can not belong to a device of the workpiece measurement system.

[0208] In one embodiment, a computer device is also provided, which includes a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0209] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0210] In one embodiment, a computer program product is provided, which includes a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0211] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., and is not limited thereto. The processor involved in the embodiments provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., and is not limited thereto.

[0212] The technical features of the above embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0213] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A workpiece measurement method, characterized by, The application is applied to a workpiece measurement system, the workpiece measurement system at least comprises a depth sensor and a mobile robot, an end effector of the mobile robot is connected with the depth sensor; the method comprises: In the case that the mobile robot moves the depth sensor to a preset sensor detection area, detecting initial point cloud data corresponding to a workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; Converting the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; Converting the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table to obtain second point cloud data; Converting the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured to obtain target point cloud data; For each contour point in the workpiece to be measured, acquiring contour point theoretical position information corresponding to the contour point and a preset contour point detection area; The intersection of the target point cloud data and the preset contour point detection area is determined as target area point cloud data; According to the contour point theoretical position information, performing plane fitting on the target area point cloud data to obtain a target plane; Projecting the contour point theoretical position information to the target plane to obtain position information of the contour point; The converting of the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured comprises: Detecting a workpiece type of the workpiece to be measured; Based on the workpiece type, generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured, the third pose conversion relationship depends on the shape, size, placement mode of the workpiece to be measured and the design of the workpiece table; According to the third pose conversion relationship, converting the second point cloud data from the machining coordinate system to the workpiece coordinate system.

2. The method of claim 1, wherein, The converting of the initial point cloud data from the sensor coordinate system corresponding to the depth sensor to the robot coordinate system corresponding to the mobile robot comprises: Acquiring a first pose conversion relationship between a sensor coordinate system corresponding to the depth sensor and a robot coordinate system corresponding to the mobile robot; According to the first pose conversion relationship, converting the initial point cloud data from the sensor coordinate system to the robot coordinate system; The converting of the first point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table comprises: Acquiring a second pose conversion relationship between the robot coordinate system and the machining coordinate system corresponding to the workpiece table; According to the second pose conversion relationship, converting the first point cloud data from the robot coordinate system to the machining coordinate system.

3. The method of claim 1, wherein, Before the converting of the initial point cloud data from the robot coordinate system to the machining coordinate system corresponding to the workpiece table, the method further comprises: Detecting whether all contour points of the workpiece to be measured are contained in the first point cloud data; In the case that the first point cloud data does not contain all contour points of the workpiece to be measured, a movement instruction is sent to the mobile robot, wherein the movement instruction carries updated detection area information, and the movement instruction is used to control the mobile robot to move the depth sensor to a detection area corresponding to the updated detection area information; The step of detecting the initial point cloud data corresponding to the workpiece to be measured by the depth sensor is returned until all contour points of the workpiece to be measured are contained in the first point cloud data.

4. The method of claim 1, wherein, The target plane is obtained by performing plane fitting on the target region point cloud data according to the contour point theoretical position information, comprising: The reference distance between the reference point and the contour point is obtained by querying the reference point from the target point cloud data, wherein the distance between the reference point and the contour point theoretical position information is less than a preset first distance threshold value; The target point cloud set is obtained by querying the target point cloud data, wherein the distance between the target point cloud set and the contour point theoretical position information is less than a preset proportion of the reference distance; The target plane is obtained by performing plane fitting on the target point cloud set.

5. The method of claim 1, wherein, The mobile robot at least includes a first mobile robot and a second mobile robot, the end effector of the first mobile robot is connected with the depth sensor, and the method further comprises: In the case that the first mobile robot moves the depth sensor to a preset sensor detection area, initial point cloud data of a workpiece to be measured detected by the depth sensor is obtained; The initial point cloud data is converted from the sensor coordinate system into a first robot coordinate system corresponding to the first mobile robot to obtain third point cloud data; The third point cloud data is converted from the first robot coordinate system into a second robot coordinate system corresponding to the second mobile robot to obtain fourth point cloud data; The fourth point cloud data is converted from the second robot coordinate system into the machining coordinate system to obtain fifth point cloud data; The fifth point cloud data is converted from the machining coordinate system into the workpiece coordinate system to obtain sixth point cloud data; The position information of a plurality of contour points in the workpiece to be measured is detected from the sixth point cloud data.

6. A workpiece measuring device, characterized by, Applied to a workpiece measurement system, the workpiece measurement system at least includes a depth sensor and a mobile robot, the end effector of the mobile robot is connected with the depth sensor, and the device comprises: An initial surface point cloud acquisition module is configured to detect initial point cloud data corresponding to a workpiece to be measured by the depth sensor in the case that the mobile robot moves the depth sensor to a preset sensor detection area, wherein the workpiece to be measured is placed on a workpiece table; A first pose conversion module is configured to convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor into a robot coordinate system corresponding to the mobile robot to obtain first point cloud data; A second pose conversion module is configured to convert the first point cloud data from the robot coordinate system into a machining coordinate system corresponding to the workpiece table to obtain second point cloud data; a third pose conversion module, configured to convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data; a contour point measurement module, configured to, for each contour point in the workpiece to be measured, acquire contour point theoretical position information corresponding to the contour point and a preset contour point detection region; determine an intersection of the target point cloud data and the preset contour point detection region as target region point cloud data; perform plane fitting on the target region point cloud data according to the contour point theoretical position information, to obtain a target plane; and project the contour point theoretical position information to the target plane, to obtain position information of the contour point; the third pose conversion module is further configured to: detect a workpiece type of the workpiece to be measured; generate a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type, the third pose conversion relationship being dependent on a shape, a size, a placement manner of the workpiece to be measured, and a design of the workpiece table; convert the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

7. The apparatus of claim 6, wherein, the first pose conversion module is further configured to: acquire a first pose conversion relationship between a sensor coordinate system corresponding to the depth sensor and a robot coordinate system corresponding to the mobile robot; and convert the initial point cloud data from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship. the second pose conversion module is further configured to: acquire a second pose conversion relationship between the robot coordinate system and a machining coordinate system corresponding to the workpiece table; and convert the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

8. A workpiece measurement system characterized by, the system at least includes: a depth sensor, configured to detect point cloud data of a workpiece to be measured; a mobile robot, an end effector of the mobile robot being connected to the depth sensor; a controller, configured to: in a case where the mobile robot moves the depth sensor to a preset sensor detection region, detect initial point cloud data corresponding to the workpiece to be measured by the depth sensor, wherein the workpiece to be measured is placed on a workpiece table; convert the initial point cloud data from a sensor coordinate system corresponding to the depth sensor to a robot coordinate system corresponding to the mobile robot, to obtain first point cloud data; convert the first point cloud data from the robot coordinate system to a machining coordinate system corresponding to the workpiece table, to obtain second point cloud data; convert the second point cloud data from the machining coordinate system to a workpiece coordinate system corresponding to the workpiece to be measured, to obtain target point cloud data; for each contour point in the workpiece to be measured, acquire contour point theoretical position information corresponding to the contour point and a preset contour point detection region; determine an intersection of the target point cloud data and the preset contour point detection region as target region point cloud data; According to the contour point theoretical position information, plane fitting is performed on the target region point cloud data to obtain a target plane; The contour point theoretical position information is projected to the target plane to obtain position information of the contour points; The converting the second point cloud data from the machining coordinate system to the workpiece coordinate system corresponding to the workpiece to be measured comprises: detecting a workpiece type of the workpiece to be measured; generating a third pose conversion relationship between the machining coordinate system and the workpiece coordinate system corresponding to the workpiece to be measured based on the workpiece type, the third pose conversion relationship being dependent on a shape, a size, a placement manner of the workpiece to be measured, and a design of the workpiece table; converting the second point cloud data from the machining coordinate system to the workpiece coordinate system according to the third pose conversion relationship.

9. The system of claim 8, wherein, The controller is further configured to: obtain a first pose conversion relationship between a sensor coordinate system corresponding to the depth sensor and a robot coordinate system corresponding to the mobile robot; and convert the initial point cloud data from the sensor coordinate system to the robot coordinate system according to the first pose conversion relationship; obtain a second pose conversion relationship between the robot coordinate system and a machining coordinate system corresponding to the workpiece table; and convert the first point cloud data from the robot coordinate system to the machining coordinate system according to the second pose conversion relationship.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 5.

Images