Stereoscopic parts correction method based on 3D camera
Through the three-dimensional part correction method based on 3D cameras, the part offset value is determined using calibration files and point cloud data, and the precise correction of three-dimensional parts is achieved, which solves the problem of insufficient accuracy of large-field cameras and improves the parts in industrial automation Material accuracy and efficiency.
Patent Information
- Application Number
- CN202411389387.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-10-08
AI Technical Summary
In industrial automation, when using a large field of view camera to estimate the posture of three-dimensional parts, the accuracy is insufficient, resulting in the inability to meet the accuracy requirements of industrial production during subsequent sorting and loading.
Using the three-dimensional part correction method based on the 3D camera, by obtaining calibration files, controlling the grabbing parts to grab parts and moving them to preset correction positions, obtaining point cloud data based on the 3D camera, determining the offset value of the part and correcting it to ensure the parts Accurate placement.
It improves the loading accuracy of three-dimensional parts, meets the accuracy requirements of industrial production, and enhances the efficiency and accuracy of equipment operation.
Smart Images

Figure CN118898654B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual technology, and in particular to a 3D camera-based three-dimensional part correction method. Background Art
[0002] With the development of industrial automation, when detecting and identifying materials on the production line, it is necessary to automatically classify, sort or locate the materials in order to improve production efficiency and accuracy.
[0003] Usually, a wide-field camera is used to capture images of the material and estimate the spatial position of the material. However, due to the insufficient accuracy of the camera and the fact that most materials are three-dimensional, the error in estimating the object's posture is large, which cannot meet the accuracy requirements of industrial production during subsequent material sorting and loading.
[0004] Therefore, a 3D camera-based three-dimensional parts correction method is provided, which has high accuracy and can meet production requirements. Summary of the invention
[0005] One or more embodiments of the present specification provide a method for correcting three-dimensional parts based on a 3D camera, the method comprising: obtaining a calibration file based on a three-dimensional part and a gripper for gripping the three-dimensional part; the calibration file comprising a calibration plane, a normal vector of the calibration plane, a reference point, and distance information between the calibration plane and a preset point; controlling the gripper to grip the three-dimensional part and move it to a preset correction position; at the preset correction position, obtaining point cloud data of the three-dimensional part and the gripper based on a 3D camera; determining an offset value of the three-dimensional part based on the calibration file and the point cloud data; and correcting the three-dimensional part based on the offset value of the three-dimensional part.
[0006] In some embodiments, the calibration plane includes three intersecting planes, and at least one of the three planes is located on the three-dimensional part.
[0007] In some embodiments, determining the offset value of the three-dimensional part based on the calibration file and the point cloud data includes: performing plane segmentation on the point cloud data to determine a plane list; based on the calibration file, filtering out a target plane corresponding to the calibration plane that meets a first preset filtering condition from the plane list; the first preset filtering condition includes the angle range between the plane in the plane list and the calibration plane, and the distance range between the plane in the plane list and the reference point; based on the target plane, determining the edge line of the three-dimensional part; based on the edge line, determining the offset value of the three-dimensional part.
[0008] In some embodiments, the calibration plane includes a first plane located on the three-dimensional part, and a second plane and a third plane located on the gripping member; filtering out a target plane corresponding to the calibration plane that meets a first preset filtering condition from the plane list includes: based on the first preset filtering condition, determining from the plane list a first target plane corresponding to the first plane in the calibration plane, a second target plane corresponding to the second plane, and a third target plane corresponding to the third plane.
[0009] In some embodiments, determining the edge lines of the three-dimensional part based on the target plane includes: determining a straight line list based on the first target plane; determining a first edge line set of the first target plane from the straight line list based on a second preset filtering condition; the second preset filtering condition includes: the angle range of the direction vectors of the intersection lines of the straight lines in the straight line list and the first target plane and the second target plane; determining a second edge line set of the first target plane from the straight line list based on a third preset filtering condition; the third preset filtering condition includes the angle range of the direction vectors of the intersection lines of the straight lines in the straight line list and the first target plane and the third target plane; determining the edge lines of the three-dimensional part based on the first edge line set and the second edge line set.
[0010] In some embodiments, determining the offset value of the three-dimensional part based on the edge line includes: determining a first straight line farthest from the reference point based on the first edge line set; the reference point is a first distance away from the first straight line; determining a second straight line farthest from the reference point based on the second edge line set; the reference point is a second distance away from the second straight line; and determining the offset value of the three-dimensional part based on the first distance and the second distance.
[0011] In some embodiments, determining the offset value of the three-dimensional part based on the edge line includes:
[0012] Based on the first edge line set, determine the first straight line farthest from the reference point; the reference point is at a first distance from the first straight line; based on the intersection of the first target plane and the second target plane, determine a first preset reference point; the first preset reference point is at a third distance from the first straight line; based on the second edge line set, determine the second straight line farthest from the reference point; the reference point is at a second distance from the second straight line; based on the intersection of the first target plane and the third target plane, determine a second preset reference point; the second preset reference point is at a fourth distance from the second straight line; based on the first distance, the second distance, the third distance and the fourth distance, determine the offset value of the three-dimensional part.
[0013] One or more embodiments of the present specification provide a 3D camera-based three-dimensional part correction system, including a calibration module, a control module and a correction module; the calibration module obtains a calibration file based on the three-dimensional part and a gripper for gripping the three-dimensional part; the calibration file includes a calibration plane, a normal vector of the calibration plane, a reference point and distance information between the calibration plane and a preset point; the control module controls the gripper to grip the three-dimensional part and move it to a preset correction position; the correction module obtains point cloud data of the three-dimensional part and the gripper based on a 3D camera at the preset correction position; determines an offset value of the three-dimensional part based on the calibration file and the point cloud data; and corrects the three-dimensional part based on the offset value of the three-dimensional part.
[0014] One or more embodiments of the present specification provide a 3D camera-based stereo part correction device, including a processor, wherein the processor is used to execute a 3D camera-based stereo part correction method.
[0015] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes a 3D camera-based three-dimensional part correction method. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:
[0017] Figure 1 is a schematic diagram of a 3D camera-based three-dimensional part correction system according to some embodiments of this specification;
[0018] Figure 2 is an exemplary flow chart of a 3D camera-based three-dimensional part calibration method according to some embodiments of this specification;
[0019] Figure 3 is a schematic diagram of an application scenario of obtaining a calibration file according to some embodiments of this specification;
[0020] Figure 4 is an exemplary flow chart of determining an offset value of a three-dimensional part according to some embodiments of the present specification;
[0021] Figure 5 is an exemplary flow chart of determining the edge line of a three-dimensional part according to some embodiments of this specification;
[0022] Figure 6is an exemplary flow chart of determining an offset value of a three-dimensional part according to some embodiments of the present specification;
[0023] Figure 7 is another exemplary flow chart of determining an offset value of a three-dimensional part according to some embodiments of the present specification;
[0024] Figure 8 It is a model schematic diagram of the posture model shown in some embodiments of this specification. DETAILED DESCRIPTION
[0025] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.
[0026] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
[0027] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0028] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0029] With the development of industrial automation production, it is necessary to automatically classify, sort or locate materials. Materials are usually three-dimensional objects, such as three-dimensional parts. In some embodiments, three-dimensional parts can be automatically classified, sorted or located based on a wide field of view camera and a 3D camera. The wide field of view camera can be set on a high-precision 3D camera. Before the robot arm grabs the three-dimensional parts through a grabber such as an electromagnet, the system can perform initial calculations based on the captured image of the wide field of view camera to roughly determine the posture of the three-dimensional parts on the workstation, and then the system can control the grabber of the robot arm to grab the three-dimensional parts in a preset fixed posture. The preset fixed posture refers to the preset posture of the grabber relative to the three-dimensional part. Due to the shooting accuracy limitation of the wide field of view camera and the shaking of the grabber during the movement after grabbing, when the grabber grabs the three-dimensional part, the three-dimensional part may not be located at the preset position, that is, the three-dimensional part may have an offset value relative to the preset position. The preset position refers to the preset spatial position of the grabbed three-dimensional part relative to the grabber.
[0030] Therefore, in some embodiments of the present specification, a calibration file is obtained based on a 3D camera, an offset value of a three-dimensional part is determined based on the calibration file, and the three-dimensional part is corrected based on the offset value, so as to improve the accuracy of subsequent material loading.
[0031] Figure 1 Schematic diagram of a 3D camera-based three-dimensional part correction system according to some embodiments of this specification. Figure 1 As shown, the 3D camera-based three-dimensional part correction system 100 may include a calibration module 110 , a control module 120 and a correction module 130 .
[0032] The calibration module 110 is used to calibrate the three-dimensional part and the gripper for gripping the three-dimensional part. In some embodiments, the calibration module 110 can obtain a calibration file based on the three-dimensional part and the gripper for gripping the three-dimensional part. For more information about the calibration file and how to obtain the calibration file, please refer to Figure 2 and its related description.
[0033] The control module 120 is used to control the action of the gripper. In some embodiments, the control module 120 can control the gripper to grip the three-dimensional part and move it to a preset correction position. For more information about the preset correction position, please refer to Figure 2 and its related description.
[0034] The correction module 130 is used to determine the offset value and perform correction based on the offset value. In some embodiments, the correction module 130 can obtain point cloud data of the three-dimensional part and the gripper based on the 3D camera at a preset correction position; determine the offset value of the three-dimensional part based on the calibration file and the point cloud data; and correct the three-dimensional part based on the offset value of the three-dimensional part. For more information about determining the offset value and correction, please refer to Figure 2 and its related description.
[0035] In some embodiments of the present specification, a 3D camera-based three-dimensional part correction system can realize intelligent automatic correction of three-dimensional parts and improve feeding accuracy by obtaining a calibration file through a calibration module, controlling a gripper through a control module, and determining an offset value and performing correction through a correction module.
[0036] It should be noted that the above description of the 3D camera-based stereoscopic parts calibration system and its modules is only for the convenience of description and cannot limit the present specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, Figure 1 The calibration module, control module and correction module disclosed in the specification can be different modules in a system, or one module can realize the functions of two or more modules. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the scope of protection of this specification.
[0037] Figure 2 is an exemplary flow chart of a 3D camera-based three-dimensional part calibration method according to some embodiments of this specification. Figure 2 As shown, the process 200 includes the following steps. In some embodiments, the process 200 may be executed by the 3D camera-based three-dimensional part calibration system 100 .
[0038] Step 210 , obtaining a calibration file based on the three-dimensional part and a gripper for gripping the three-dimensional part.
[0039] Figure 3 It is a schematic diagram of an application scenario of obtaining a calibration file according to some embodiments of this specification.
[0040] Three-dimensional parts 1 refers to parts with three-dimensional structure, see Figure 3 In some embodiments, the three-dimensional part 1 may have a planar structure that is convenient for grasping and positioning. In some embodiments, the three-dimensional part 1 may be made of a metal material that can be magnetically attracted, or other materials.
[0041] A gripper is a mechanical device used to locate, pick up, and move parts. Figure 3 As shown, the gripper may be an electromagnet 2 of a robot arm 3, and the electromagnet 2 may magnetically absorb the metal three-dimensional part 1 placed on the workstation. In some embodiments, the gripper may be a vacuum suction cup structure. In some embodiments, the gripper may also be other structural members for gripping, such as a gripper structure that can be opened and closed.
[0042] The calibration file is a reference file for part calibration. In some embodiments, the calibration module 110 calibrates the three-dimensional part and the gripper in a preset position to obtain a calibration file, which can reflect the spatial position information of the three-dimensional part and the gripper when the three-dimensional part and the gripper are in a preset fixed posture, that is, the theoretical position. In some embodiments, the calibration file can include a calibration plane, a normal vector of the calibration plane, a reference point, and distance information between the calibration plane and the preset point.
[0043] In some embodiments, the calibration module 110 may represent the position information of the gripper and the three-dimensional part, ie, the theoretical position, when the gripper grips the three-dimensional part in a preset fixed posture based on a preset spatial coordinate system.
[0044] In some embodiments, the preset spatial coordinate system used for calibration can be established based on any preset position point as the coordinate origin. For example, it can be established based on a position point on the gripper as the coordinate origin. For another example, it can be established based on the optical center of the 3D camera as the coordinate origin.
[0045] The calibration plane refers to the position of the preset plane of the three-dimensional part in the preset space coordinate system when the gripper grips the three-dimensional part in a preset fixed posture. In some embodiments, the calibration plane may include three intersecting planes, and at least one of the three planes may be located on the three-dimensional part.
[0046] In some embodiments, the calibration plane may include a first plane, a second plane, and a third plane that intersect each other, and the surface of the three-dimensional part that contacts the gripping member may be the first plane.
[0047] In some embodiments, the first plane, the second plane, and the third plane may be perpendicular to each other.
[0048] The reference point is the origin of the preset spatial coordinate system. In some embodiments, the reference point can be preset as any convenient point in space based on prior experience. In some embodiments, the reference point can be the intersection of the first plane, the second plane and the third plane, for example Figure 3 In some embodiments, the origin O in the Figure 3As shown, the calibration module 110 can establish a preset spatial rectangular coordinate system with the origin O as the reference point, the XOY plane as the first plane, the XOZ plane as the second plane, and the YOZ plane as the third plane. The spatial positions of the grasping parts and the three-dimensional parts are determined by determining their coordinates in the preset spatial rectangular coordinate system, thereby achieving calibration.
[0049] The preset point refers to a predetermined position point. In some embodiments, the preset point can be the optical center of the 3D camera, that is, the intersection between the main optical axis of the 3D camera and the image plane.
[0050] In some embodiments, the preset point may also be set to other position points whose coordinate positions are unchanged relative to the preset spatial coordinate system. In some embodiments, the correction module 130 may establish a scanning spatial coordinate system based on the preset point as the origin to represent the actual position of the gripper and the three-dimensional part.
[0051] The distance information between the calibration plane and the preset point refers to the spatial relationship information between the calibration plane and the preset point. In some embodiments, the distance information can be represented by a vector.
[0052] In some embodiments, the calibration module 110 can determine the position and direction of the preset point relative to the preset space coordinate system based on the distance information between the calibration plane and the preset point, thereby constructing a transformation matrix to map the lines and surfaces represented by the scanning space coordinate system to the preset space coordinate system, and vice versa.
[0053] In some embodiments, the calibration module 110 can obtain the calibration file in a variety of ways. For example, the calibration module 110 can scan based on the camera to obtain the calibration plane and the distance information between the calibration plane and the preset point; obtain the normal vector of the calibration plane by a calculation method such as vector cross multiplication; and determine the intersection of the three calibration planes as the reference point.
[0054] Step 220, controlling the grasping member to grasp the three-dimensional part and move it to a preset correction position.
[0055] The preset calibration position refers to the predetermined position of the robot arm for obtaining the point cloud data of the gripper and the three-dimensional part. It can be understood that at the preset calibration position, the first plane, the second plane and the third plane should be within the field of view of the 3D camera. In some embodiments, the preset calibration position can be preset based on prior experience.
[0056] In some embodiments, the gripping member can perform gripping operations on the three-dimensional parts in a variety of ways based on the control of the control module 120. For example, when the gripping member is an electromagnet, the three-dimensional parts can be gripped by the attraction generated by the electromagnetic field. For another example, when the gripping member is a suction cup structure, the three-dimensional parts can be gripped by the pressure difference generated by the vacuum. For another example, when the gripping member is a clamping claw structure, the three-dimensional parts can be gripped by mechanical clamping. When the gripping of the three-dimensional parts is completed, the gripping member can move to a preset correction position according to the movement instruction of the control module 120.
[0057] Step 230, at a preset calibration position, based on a 3D camera, obtain point cloud data of the three-dimensional part and the gripper.
[0058] Point cloud data refers to a data set consisting of sampling points obtained by scanning a three-dimensional part and a gripper with a 3D camera. In some embodiments, the point cloud data may include a number of sampling points, each of which includes a three-dimensional coordinate in a scanning space coordinate system.
[0059] In some embodiments, point cloud data may be acquired through a 3D camera. For example, the 3D camera may determine the distance from the sampling point to the preset point by emitting laser pulses and measuring the time difference of the reflected echo. The relative direction from the sampling point to the preset point is determined by information such as the laser scanning angle. Based on the distance and relative direction from the sampling point to the preset point, the sampling point is mapped to the scanning space coordinate system to obtain the three-dimensional coordinates of the sampling point, thereby acquiring point cloud data.
[0060] Step 240 , determining the offset value of the three-dimensional part based on the calibration file and the point cloud data.
[0061] The offset value can reflect the difference between the actual position and theoretical position of the three-dimensional part. The theoretical position is described in the previous section. The actual position is the spatial position information of the gripper and the three-dimensional part when the gripper actually grips the three-dimensional part.
[0062] In some embodiments, the offset value may be represented by an X-axis offset distance, a Y-axis offset distance, and a rotation angle between the actual position and the theoretical position of the three-dimensional part, wherein a positive rotation angle value may represent a clockwise rotation, and a negative rotation angle value may represent a counterclockwise rotation.
[0063] In some embodiments, the correction module 130 can model the three-dimensional part and the gripper based on the point cloud data, and obtain the three-dimensional part model and the gripper model in the scanning space coordinate system as the actual position. Through the conversion matrix, the three-dimensional part model and the gripper model in the scanning space coordinate system are mapped to the preset space coordinate system, or the three-dimensional part model and the gripper model in the preset space coordinate system are mapped to the scanning space coordinate system. In the same coordinate system, the actual position is compared with the theoretical position to determine the offset distance and the offset angle of the three-dimensional part, that is, to determine the X-axis offset distance, the Y-axis offset distance and the rotation angle.
[0064] For more information on determining offset values, see Figure 4 and its related contents.
[0065] Step 250: calibrate the three-dimensional part based on the offset value of the three-dimensional part.
[0066] In some embodiments, the correction module 130 may determine a correction amount of the 3D part based on the offset value of the 3D part, and correct the 3D part based on the correction amount.
[0067] The correction amount is a control parameter used to control the movement and adjustment of the gripper. In some embodiments, the correction amount may correspond to the offset value, that is, the correction amount may include an X-axis offset distance, a Y-axis offset distance, and a rotation angle.
[0068] In some embodiments, the value of the correction amount can be the same as the value of the offset value and in the opposite direction. For example, when the geometric center of the three-dimensional part is offset by +1 cm in the X-axis, offset by -5 cm in the Y-axis, and rotated by +3°, the correction amount can be: control the gripper to be offset by -1 cm in the X-axis, offset by +5 cm in the Y-axis, and rotated by -3°.
[0069] In some embodiments, the correction module 130 can generate corresponding gripper control instructions according to the correction amount, and control the gripper to move and rotate in equal amplitude in the opposite direction and angle of the offset of the three-dimensional part, thereby offsetting the placement error caused by the offset value and achieving position correction of the three-dimensional part.
[0070] In some embodiments of this specification, by using a 3D camera to collect point cloud data of a three-dimensional part and its gripper, high-precision measurement of the three-dimensional spatial position of the part can be achieved. By combining the calibration file, the offset value of the three-dimensional part relative to the preset position can be accurately calculated. Correction based on these offset values ensures that the part is accurately placed at the target position, improves accuracy, and improves equipment operation efficiency.
[0071] Figure 4 FIG. 1 is an exemplary flow chart of determining the offset value of a three-dimensional part according to some embodiments of the present specification. Figure 4 As shown, the process 300 may include the following steps. In some embodiments, the process 300 may be performed by the correction module 130 .
[0072] Step 310: perform plane segmentation on the point cloud data and determine a plane list.
[0073] The plane list refers to a set of several planes obtained after differentiating each plane based on the inconsistency of normal vectors. In some embodiments, the correction module 130 may perform plane fitting based on the point cloud data to obtain several planes.
[0074] In some embodiments, the correction module 130 can perform plane segmentation on the point cloud data in a variety of ways based on the several planes obtained after plane fitting. For example, the processor can perform plane segmentation on the point cloud data by using the RANSAC algorithm, and the processor can also perform plane segmentation on the point cloud data by using methods such as Euclidean clustering segmentation, neural network model, support vector machine, etc. to obtain a plane list containing several planes.
[0075] In some embodiments, the correction module 130 may pre-process the point cloud data before performing plane fitting on the point cloud data. In some embodiments, the pre-processing includes filtering, clipping, and downsampling. Among them, filtering, clipping can exclude abnormal points (such as noise points, isolated points, etc.) in the point cloud data, and downsampling can reduce the number of sampling points in the point cloud, reduce the computational complexity of subsequent steps, and improve the computational speed.
[0076] In some embodiments, point cloud data may be affected by sensor errors, environmental interference and other factors during the acquisition process, so point cloud data usually includes abnormal points. The presence of abnormal points will affect subsequent steps such as plane fitting and segmentation. By preprocessing the point cloud data, the stability and robustness of plane fitting and segmentation can be improved, ensuring the accuracy and reliability of the plane list.
[0077] Step 320 : Based on the calibration file, select a target plane from the plane list that satisfies a first preset screening condition and corresponds to the calibration plane.
[0078] The target plane refers to the plane corresponding to the calibration plane in the calibration file, for example, the plane corresponding to the first plane.
[0079] The first preset screening condition refers to a judgment condition for determining the target plane. In some embodiments, the first preset screening condition may include an angle range between a plane in the plane list and the calibration plane, and a distance range between a plane in the plane list and the reference point. In some embodiments, the angle range and the distance range may be preset based on prior experience.
[0080] In some embodiments, the angle range can be divided into a positive angle range and a negative angle range. When the normal vector of the plane points to the reference point, that is, the angle range is within 90°-180°, it is the negative angle range; when the normal vector direction of the plane is in the opposite direction of the reference point, that is, the angle range is within 0°-90°, it is the positive angle range.
[0081] In some embodiments, when the angle range of the target plane is a reverse angle range, the correction module 130 may flip the normal vector to ensure that the normal vector of the target plane always points to the reference point.
[0082] In some embodiments, when the positive angle range between the plane in the plane list and the calibration plane is 0°-15° or the reverse angle range is 175°-180°, and the distance range between the plane in the plane list and the reference point is less than 5 mm, the first preset screening condition is met.
[0083] In some embodiments, the angle range in the first preset screening condition can be determined based on the structural complexity of the three-dimensional part and the number of objects in the surrounding environment.
[0084] In some embodiments, the angle range in the first preset screening condition may be negatively correlated to the structural complexity of the three-dimensional part and the number of objects in the surrounding environment. For example, when the shape of the three-dimensional part is more complex and the number of objects in the surrounding environment is more, the point cloud data acquired based on the 3D camera is more noisy, and the plane list contains more planes. At this time, a more stringent screening condition can be set to appropriately reduce the positive angle range and the negative angle range to more accurately determine the target plane.
[0085] The positive angle range and the negative angle range may be the same or different; the positive angle range and the negative angle range may increase or decrease at the same time; when one of the positive angle range and the negative angle range increases or decreases, the other may remain unchanged.
[0086] In some embodiments of the present specification, the angle range is determined based on the structural complexity of the three-dimensional parts and the number of objects in the surrounding environment. When the plane list contains many planes, if the screening conditions are relatively loose, there may be multiple planes that meet the first preset screening conditions, and it will take time to further distinguish and identify. At this time, the screening criteria can be improved to quickly and accurately determine the target plane.
[0087] In some embodiments, the distance range in the first preset screening condition may be determined based on the distance between the interference plane set and the first plane.
[0088] The interference plane set refers to a set of planes on the three-dimensional part that are parallel to the first plane. In some embodiments, since the three-dimensional part may have a multi-layer structure and / or a repeated structure, the plane list may contain multiple interference planes parallel to the first plane. In order to accurately determine the target plane, the distance range between the plane in the plane list and the reference point can be set to be less than the shortest distance between the first plane and the interference plane.
[0089] For example, when the three-dimensional part is an I-beam, there are four parallel surfaces, namely, the upper surface of the upper part, the lower surface of the upper part, the upper surface of the lower part, and the upper surface of the lower part. When the grasping operation is performed, the upper surface of the upper part contacts the grasping part, which is the first plane. At this time, in order to accurately determine the target plane corresponding to the first plane, the distance range between the plane in the plane list and the reference point can be set to be smaller than the distance between the upper surface of the upper part (i.e., the first plane) and the lower surface of the upper part.
[0090] In some embodiments of the present specification, determining the distance range based on the distance between the part plane and the first plane can ensure the uniqueness of the target plane and avoid the situation where multiple parallel planes are determined as target planes due to the complexity of the three-dimensional part.
[0091] In some embodiments, the calibration plane includes a first plane located on the three-dimensional part, and a second plane and a third plane located on the gripping member.
[0092] In some embodiments, the correction module 130 may determine, from the plane list based on the first preset screening condition, a first target plane corresponding to the first plane in the calibration plane, a second target plane corresponding to the second plane, and a third target plane corresponding to the third plane.
[0093] In some embodiments, the method of screening the first target plane, the second target plane, and the third target plane is similar to the method of screening the target plane in the foregoing text, and reference may be made to the foregoing text and will not be repeated herein.
[0094] In some embodiments of the present specification, the first target plane, the second target plane, and the third target plane corresponding to the first plane, the second plane, and the third plane in the calibration file are screened out by the first preset screening condition, which helps to more accurately determine the target planes corresponding to different planes and perform targeted screening and determination of the target planes.
[0095] Step 330 : determining the edge line of the three-dimensional part based on the target plane.
[0096] The edge line refers to the boundary line of the first plane of a three-dimensional part.
[0097] In some embodiments, the correction module 130 may determine the edge lines of the three-dimensional part by determining a first edge line set and a second edge line set based on the target plane. For more information about this part, please refer to Figure 5 and its related contents.
[0098] Step 340 : determining the offset value of the three-dimensional part based on the edge line.
[0099] In some embodiments, the correction module 130 can determine the offset distance and rotation angle of the 3D part based on the edge line of the 3D part by comparing the edge line of the 3D part in the theoretical position with the edge line of the 3D part in the actual position, and then determine the offset value of the 3D part.
[0100] In some embodiments, the correction module 130 may also determine the offset value of the three-dimensional part based on the first edge line set and the second edge line set. Figure 6 and its related contents.
[0101] In some embodiments, the correction module 130 may also determine the offset value of the three-dimensional part based on the reference point, the first edge line set and the second edge line set. Figure 7 and its related contents.
[0102] In some embodiments of the present specification, by determining the target plane through segmentation and screening of point cloud data, interference caused by abnormal points in the point cloud data can be eliminated, and the obtained target plane can accurately reflect the external contour of the three-dimensional part, thereby laying the foundation for quickly and accurately determining the offset value.
[0103] Figure 5 FIG. 1 is an exemplary flow chart of determining the edge line of a three-dimensional part according to some embodiments of the present specification. Figure 5 As shown, the process 400 may include the following steps. In some embodiments, the process 400 may be performed by the correction module 130 .
[0104] Step 410: Determine a list of straight lines based on the first target plane.
[0105] The line list refers to a set of several groups of lines located in the first target plane.
[0106] In some embodiments, the correction module 130 can perform boundary screening on the sampling points included in the first target plane through a preset algorithm, and retain the boundary points therein, that is, the convex hull point set. Line fitting is performed on the boundary points, and the obtained fitting line is assigned to the line list. Among them, the boundary point refers to the sampling point located at the boundary of the point cloud data. The boundary screening method includes but is not limited to the grid division method, the normal estimation method, the boundary growth algorithm, etc.
[0107] Step 420: Determine a first edge line set of the first target plane from the straight line list based on a second preset screening condition.
[0108] The first edge set refers to the edge set in the line list that meets the second preset screening condition. The first edge set can reflect an edge in the first plane of the three-dimensional part (for example, Figure 3 In some embodiments, the first edge line set may include several groups of first edge lines.
[0109] The second preset screening condition is a judgment condition for screening the first edge line set. In some embodiments, the second preset screening condition includes: an angle range of a direction vector of an intersection line between a straight line in the straight line list and the first target plane and the second target plane. The angle range of the second preset screening condition can be determined based on prior experience.
[0110] The intersection line of the first target plane and the second target plane can be determined by a variety of methods. For example, the correction module 130 can obtain the plane equations and plane normal vectors corresponding to the first target plane and the second target plane, calculate the cross product of the corresponding plane normal vectors, and obtain the direction vector of the corresponding intersection line. The correction module 130 can set a common point on the edge line as (0, y, z), substitute the common point into the two plane equations, and calculate the coordinates of the common point. Based on the common point coordinates and the direction vector of the intersection line, the intersection line corresponding to the two planes can be determined.
[0111] In some embodiments, when the positive angle range of the direction vector of the intersection line between the straight line and the first target plane and the second target plane is 0°-15° or the negative angle range is 175°-180°, the second preset screening condition is met.
[0112] Step 430 : determining a second edge line set of the first target plane from the straight line list based on a third preset screening condition.
[0113] The second edge set refers to the edge set in the line list that meets the third preset screening condition. The second edge set can reflect another edge in the first plane of the three-dimensional part (for example, Figure 3 In some embodiments, the second edge line set may include several groups of second edge lines.
[0114] The third preset screening condition is a judgment condition for screening the second edge line set. In some embodiments, the third preset screening condition includes an angle range between a straight line in the straight line list and a direction vector of an intersection line between the first target plane and the third target plane. The method for determining the intersection line of the first target plane and the third target plane is similar to the method for determining the intersection line of the first target plane and the second target plane, and will not be described in detail.
[0115] The third preset screening condition is similar to the second preset screening condition, and reference may be made to the relevant content above.
[0116] In some embodiments, when the positive angle range of the direction vector of the intersection of the straight line with the first plane and the third plane is 0°-15° or the negative angle range is 175°-180°, the third preset screening condition is met.
[0117] Step 440 : determining the edges of the three-dimensional part based on the first edge set and the second edge set.
[0118] In some embodiments, the correction module 130 may use the straight line with the longest distance from the reference point in the first edge line set and the straight line with the longest distance from the reference point in the second edge line set as the edge line of the three-dimensional part.
[0119] In some embodiments of the present specification, by screening the edge set contained in the straight line list and determining the edge of the three-dimensional part, when the three-dimensional part has a complex shape, the edge that can reflect the outer contour of the three-dimensional part can be accurately selected, thereby laying the foundation for subsequent steps such as accurately measuring the offset value.
[0120] Figure 6 FIG. 1 is an exemplary flow chart of determining the offset value of a three-dimensional part according to some embodiments of the present specification. Figure 6 As shown, the process 500 may include the following steps. In some embodiments, the process 500 may be performed by the correction module 130 .
[0121] Step 510: Determine a first straight line farthest from a reference point based on the first edge line set; the reference point and the first straight line are at a first distance.
[0122] The first straight line refers to the edge line in the first edge line set that has the longest distance from the reference point.
[0123] In some embodiments, the correction module 130 may draw a perpendicular line from the reference point to the first sideline, take the point where the perpendicular line intersects the first sideline as the foot of the perpendicular, and calculate the distance from the foot of the perpendicular to the reference point, which is the distance between the first sideline and the reference point. The correction module 130 may calculate the distances between a plurality of first sidelines included in the first sideline set and the reference point by the above method, and select the first sideline with the longest distance as the first straight line.
[0124] Step 520: Based on the second edge line set, determine a second straight line farthest from the reference point; the reference point and the second straight line are at a second distance.
[0125] The second straight line refers to the edge line in the second edge line set that has the longest distance from the reference point.
[0126] The second straight line is determined in a similar manner to the first straight line, as described above.
[0127] Step 530: Determine an offset value of the three-dimensional part based on the first distance and the second distance.
[0128] In some embodiments, the correction module 130 can obtain the actual position of the edge of the three-dimensional part in the scanning space coordinate system through the first distance and the second distance, and determine the offset value by comparing the actual position with the calibrated theoretical position. For more information about the theoretical position, please refer to Figure 2 and related instructions.
[0129] In some embodiments of the present specification, by screening the edge set contained in the straight line list and determining the offset value by the straight line in the set that is farthest from the reference point, it is possible to accurately select the edge that can reflect the outer contour of the three-dimensional part when the three-dimensional part has a complex shape, thereby laying the foundation for subsequent steps such as accurately determining the offset value.
[0130] Figure 7 FIG. 1 is another exemplary flow chart for determining the offset value of a three-dimensional part according to some embodiments of the present specification. Figure 7 As shown, the process 600 may include the following steps. In some embodiments, the process 600 may be performed by the correction module 130 .
[0131] In step 610, the correction module 130 may determine, based on the first edge set, a first straight line farthest from the reference point; the reference point is a first distance away from the first straight line. The steps of determining the first straight line and the first distance are similar to step 510, and may be referred to as Figure 6 and its related contents.
[0132] Step 620: Determine a first preset reference point based on the intersection line of the first target plane and the second target plane; the first preset reference point is at a third distance from the first straight line.
[0133] The steps for determining the third distance are similar to those for determining the first distance. Figure 6 and its related contents.
[0134] The first preset reference point refers to one of the reference points used to determine the offset value of the three-dimensional part. The first preset reference point is set at the intersection of the first target plane and the second target plane. The position of the first preset reference point can be preset based on prior experience or randomly selected.
[0135] Step 630: Based on the second edge set, determine the second straight line farthest from the reference point; the reference point is a second distance away from the second straight line. The steps of determining the second straight line and the second distance are similar to step 520, and can be seen in Figure 6 and its related contents.
[0136] Step 640: Determine a second preset reference point based on the intersection line of the first target plane and the third target plane; the second preset reference point is at a fourth distance from the second straight line.
[0137] The steps for determining the fourth distance are similar to those for determining the first distance, and can be found in Figure 6 and its related contents.
[0138] The second preset reference point refers to one of the reference points used to determine the offset value of the three-dimensional part. The second preset reference point is set on the intersection of the first target plane and the third target plane. The position of the second preset reference point can be preset based on prior experience or randomly selected.
[0139] Step 650 , determining a three-dimensional part offset value based on the first distance, the second distance, the third distance, and the fourth distance.
[0140] In some embodiments, the processor can calculate the actual position of the three-dimensional part in the scanning space coordinate system through the first distance, the second distance, the third distance and the fourth distance, and determine the offset value by comparing the actual position with the theoretical position. For more information about the theoretical position, please refer to Figure 2 and related instructions.
[0141] In some embodiments of the present specification, two preset reference points are set to determine four distance values, thereby determining the offset value of the three-dimensional part. Compared with the calculation using two distance values in the previous text, the calculation accuracy of the offset value can be further improved, thereby ensuring that the offset of the three-dimensional part can be accurately calibrated.
[0142] In some embodiments, at a preset calibration position, the gripper may have a preset scanning posture.
[0143] The preset calibration position can be found in Figure 2 Related instructions.
[0144] The preset scanning posture refers to the preset posture of the gripper relative to the 3D camera. The gripper of the robot arm can form a preset scanning posture based on the instruction of the control module 120. In some embodiments, the gripper is collected by the 3D camera under the preset scanning posture. The preset scanning posture can be represented by the coordinates of the spatial coordinate system.
[0145] In some embodiments, the control module 120 can determine the preset scanning posture in a variety of ways. For example, the control module 120 can establish a first preset table based on historical three-dimensional part parameters (such as type, size, etc.) and historical preset scanning postures. The second preset table includes the correspondence between historical three-dimensional part parameters and different historical preset scanning postures. The control module 120 can determine the current preset scanning posture according to the current three-dimensional part parameters by consulting the first preset table.
[0146] In some embodiments, the control module 120 can generate multiple candidate scanning postures, and predict the effectiveness of each candidate scanning posture through a posture model. The effectiveness can reflect the probability that the candidate scanning posture can effectively correct a three-dimensional part; based on the effectiveness, the optimal scanning posture is determined, and the optimal scanning posture is used as the preset scanning posture.
[0147] The posture model is a prediction model for determining the effectiveness of a candidate scanning posture. In some embodiments, the posture model may be a machine learning model, such as a deep neural network (DNN) model, a support vector machine (SVM) model, or a combination thereof.
[0148] Figure 8 It is a model schematic diagram of the posture model shown in some embodiments of this specification.
[0149] In some embodiments, Figure 8 As shown, the input of the posture model 750 may include candidate scanning postures 710, gripper structure parameters 720, three-dimensional part structure parameters 730 and preset fixed postures 740; the output may be effectiveness 760. In some embodiments, the posture model outputs the effectiveness of each candidate scanning posture in the plurality of candidate scanning postures.
[0150] Candidate scanning postures refer to candidate preset scanning postures. In some embodiments, the gripper may have multiple postures relative to the 3D camera, and the control module 120 may randomly generate coordinates corresponding to other postures within the spatial range of at least one preset posture based on at least one preset posture of the gripper and the coordinate representation in the scanning space coordinate system as candidate scanning postures. The preset posture of the gripper may be obtained through default parameters or prior experience. The control module 120 may use the physical center of the gripper corresponding to the preset posture of the gripper as the center of the circle, and determine the range within the preset radius as the spatial range of the preset posture.
[0151] The three-dimensional part structural parameters refer to parameters that can reflect the properties of the three-dimensional part. In some embodiments, the three-dimensional part structural parameters may include the type, size, weight, etc. of the three-dimensional part. The three-dimensional part structural parameters may be manually input into the control module 120 .
[0152] The gripper structure parameters refer to parameters that can reflect the properties of the gripper. In some embodiments, the gripper structure parameters may include the type, size, maximum gripping load, etc. of the gripper. The gripper structure parameters may be manually input into the control module 120.
[0153] For instructions on preset fixed postures, please refer to the previous article.
[0154] The effectiveness can be reflected in the probability that the candidate scanning posture can effectively correct the offset of the three-dimensional part. In some embodiments, the effectiveness can be represented by a numerical value, and the larger the numerical value, the greater the probability that the offset of the three-dimensional part can be effectively corrected.
[0155] In some embodiments, the input of the posture model may also include the degree of suction tilt.
[0156] The degree of suction tilt refers to the degree to which the gripper deviates from the preset fixed posture relative to the three-dimensional part. In some embodiments, the control module 120 can control the gripper to grip the three-dimensional part in a preset fixed posture, and move to a preset correction position, and control the gripper to rotate in all directions until the 3D camera collects comprehensive point cloud data of the three-dimensional part and the preset part of the gripper in the preset fixed posture, and processes the comprehensive point cloud data, removes other point cloud data outside the preset part, and uses the remaining point cloud data as reference point cloud data. The preset part can be a default parameter or determined based on prior experience.
[0157] In some embodiments, when the gripper actually grips the three-dimensional part and moves it to the preset correction position, the point cloud data of the three-dimensional part and the preset part of the gripper can be collected by the 3D camera, that is, the actual point cloud data. The reference point cloud data is compared with the actual point cloud data, and the similarity between the two is calculated to obtain the degree of tilt absorption. For example, the degree of tilt absorption = 100% - similarity. Among them, the similarity can be calculated by methods such as Manhattan distance and Euclidean distance.
[0158] In some embodiments, the posture model can be obtained by training the initial posture model based on a large number of training samples with training labels. The training samples may include sample candidate scanning postures, sample gripper structure parameters, sample three-dimensional part structure parameters, and sample preset fixed postures. In some embodiments, the training samples may also include the degree of sample suction tilt. The training samples may be determined based on historical data.
[0159] The training label can be the corresponding sample validity under the training sample condition. The first training label can be obtained by marking the historical correction situation. For example, the historical correction record corresponding to the training sample is obtained. If the three-dimensional part can be placed in the set target placement position after correction according to the historical offset value, it is marked as 1; if it cannot be placed in the target placement position, it is marked as 0.
[0160] During training, the training samples are input into the initial posture model, a loss function is constructed based on the output of the initial posture model and the training labels, and the parameters of the initial posture model are iteratively updated based on the loss function until the preset conditions are met, the training ends, and a trained posture model is obtained. The preset conditions may include but are not limited to the convergence of the loss function and the reaching of a threshold value of the training cycle.
[0161] In some embodiments of the present specification, the input of the posture model also includes the degree of absorption tilt, which can enrich the data richness of the model and help improve the generalization ability and adaptability of the model when facing different three-dimensional parts.
[0162] In some embodiments, the control module 120 may determine the candidate scanning posture with the highest validity degree corresponding to the candidate scanning posture being higher than the validity threshold as the optimal scanning posture, and determine the optimal scanning posture as the preset scanning posture. The validity threshold may be determined based on prior experience.
[0163] In some embodiments, the effective threshold value may also be determined based on the structural complexity of the three-dimensional part. For example, the higher the structural complexity of the three-dimensional part, the higher the effective threshold value.
[0164] In some embodiments, the structural complexity of a three-dimensional part may be related to the type and corresponding number of faces included in the three-dimensional part structure. The type of face may include planes and curved surfaces, wherein mutually parallel planes are the same type of planes, and identical curved surfaces are the same type of curved surfaces. The structural complexity of a three-dimensional part may be positively correlated to the type and corresponding number of faces. For example:
[0165]
[0166] in, Indicates the structural complexity of three-dimensional parts. Represents the weight coefficient of the nth type of face of a three-dimensional part, Represents the number of n-th type faces of the three-dimensional part structure. Among them, the weight coefficient , , , The value of can be negatively correlated to the angle between the face and the first plane of the three-dimensional part; the weight coefficient , , , The value of can be preset based on prior experience.
[0167] In some embodiments of the present specification, the effective threshold is determined based on the complexity of the three-dimensional part structure. When the three-dimensional part structure is more complex, the effective threshold can be appropriately increased to avoid interference caused by subsequent plane screening steps due to the three-dimensional part containing the surface.
[0168] Some embodiments of the present specification also provide a 3D camera-based stereoscopic part correction device, which may include a processor, and the processor may execute the aforementioned 3D camera-based stereoscopic part correction method.
[0169] Some embodiments of the present specification also provide a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer can execute the aforementioned 3D camera-based three-dimensional part correction method.
[0170] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.
[0171] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.
[0172] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.
[0173] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.
[0174] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining the digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.
[0175] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification is hereby incorporated by reference in its entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.
[0176] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.
Claims
1. A 3D camera-based three-dimensional parts correction method, characterized in that: The method comprises: Based on the three-dimensional part and the gripper for gripping the three-dimensional part, a calibration file is acquired; the calibration file includes a calibration plane, a normal vector of the calibration plane, a reference point, and distance information between the calibration plane and a preset point; Controlling the grabbing member to grab the three-dimensional part and move it to a preset correction position; At the preset correction position, based on a 3D camera, acquiring point cloud data of the three-dimensional part and the gripping member; Based on the calibration file and the point cloud data, determining the offset value of the three-dimensional part; the calibration plane includes three intersecting planes, and at least one of the three planes is located on the three-dimensional part; Correcting the three-dimensional part based on the offset value of the three-dimensional part; Wherein, determining the offset value of the three-dimensional part based on the calibration file and the point cloud data includes: Performing plane segmentation on the point cloud data to determine a plane list; Based on the calibration file, a target plane that satisfies a first preset screening condition and corresponds to the calibration plane is screened out from the plane list; the first preset screening condition includes an angle range between a plane in the plane list and the calibration plane, and a distance range between a plane in the plane list and the reference point; the calibration plane includes a first plane located on the three-dimensional part, and a second plane and a third plane located on the gripper; Based on the target plane, determining the edge line of the three-dimensional part; Based on the edge line, determining an offset value of the three-dimensional part; The step of selecting a target plane that satisfies a first preset screening condition and corresponds to the calibration plane from the plane list includes: Based on the first preset screening condition, determining from the plane list a first target plane corresponding to the first plane in the calibration plane, a second target plane corresponding to the second plane, and a third target plane corresponding to the third plane; Wherein, determining the edge line of the three-dimensional part based on the target plane includes: Based on the first target plane, determining a list of straight lines; Based on a second preset screening condition, determining a first edge line set of the first target plane from the line list; the second preset screening condition includes: an angle range of a direction vector of an intersection line between a line in the line list and the first target plane and the second target plane; Based on a third preset screening condition, determining a second edge line set of the first target plane from the straight line list; the third preset screening condition includes an angle range of a direction vector of an intersection line between a straight line in the straight line list and the first target plane and the third target plane; Determining the edge lines of the three-dimensional part based on the first edge line set and the second edge line set; Wherein, determining the offset value of the three-dimensional part based on the edge line includes: Based on the first edge line set, determining a first straight line farthest from the reference point; the reference point is at a first distance from the first straight line; Based on the second edge line set, determining a second straight line farthest from the reference point; the reference point is a second distance away from the second straight line; An offset value of the three-dimensional part is determined based on the first distance and the second distance.
2. The method according to claim 1, characterized in that The step of determining the offset value of the three-dimensional part based on the edge line includes: Based on the first edge line set, determining a first straight line farthest from the reference point; the reference point is at a first distance from the first straight line; Determine a first preset reference point based on the intersection of the first target plane and the second target plane; the first preset reference point is a third distance away from the first straight line; Based on the second edge line set, determining a second straight line farthest from the reference point; the reference point is a second distance away from the second straight line; Based on the intersection line of the first target plane and the third target plane, determining a second preset reference point; the second preset reference point is at a fourth distance from the second straight line; The three-dimensional part offset value is determined based on the first distance, the second distance, the third distance, and the fourth distance.
3. A 3D camera-based three-dimensional parts correction system, characterized in that: Execute the method according to claim 1 or 2; It includes a calibration module, a control module and a correction module; The calibration module acquires a calibration file based on the three-dimensional part and a gripper for gripping the three-dimensional part; the calibration file includes a calibration plane, a normal vector of the calibration plane, a reference point, and distance information between the calibration plane and a preset point; The control module controls the grabbing member to grab the three-dimensional part and move it to a preset correction position; The correction module obtains the point cloud data of the three-dimensional part and the gripper based on a 3D camera at the preset correction position; determines the offset value of the three-dimensional part based on the calibration file and the point cloud data; and corrects the three-dimensional part based on the offset value of the three-dimensional part.
4. A 3D camera-based three-dimensional parts calibration device, characterized in that: The method comprises a processor which executes the method as claimed in claim 1 or 2.
5. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the method according to claim 1 or 2.
Citation Information
Patent Citations
Robot grabbing method, system and device based on 3D vision and medium
CN113715016A
Robot 3D visual grabbing method, deviation rectifying method and system
CN113927606A
Method for controlling robots, parts processing equipment and use of the equipment
EP1671757A2
Apparatus for correcting image distortion of stereo-camera and method thereof
WO2006062325A1
Attitude correction method, apparatus and system for laser radar
WO2021189468A1
