Multi-workpiece positioning method, device and computer equipment

Abstract

The application relates to a multi-workpiece positioning method, device and computer equipment. The method comprises the following steps: acquiring region point cloud data collected for a workpiece placement region under the condition that at least two workpieces are placed in the workpiece placement region; clustering the region point cloud data to obtain workpiece point cloud data of each workpiece; performing two-dimensional projection on the workpiece point cloud data of each workpiece to determine a first geometric feature of the workpiece under a current collection visual angle; screening a target point cloud model corresponding to a second geometric feature matching the first geometric feature under a preset collection visual angle from a plurality of workpiece point cloud models; the preset collection visual angle is similar to the current collection visual angle; performing point cloud registration on point cloud data of the target point cloud model and the workpiece point cloud data to obtain positioning information of the workpiece. The method can improve the positioning accuracy of the multi-workpiece.

Description

Technical Field

[0001] This application relates to the field of welding control technology, and in particular to a multi-workpiece positioning method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In the fields of automated manufacturing and intelligent welding, the positioning accuracy of the workpiece before welding will directly affect the weld quality, welding trajectory stability and overall production efficiency. Especially in robotic automatic welding systems, the spatial pose information of the workpiece is a prerequisite for path planning and welding execution.

[0003] Currently, common workpiece positioning methods include mechanical limit positioning and vision-based point cloud registration positioning. However, mechanical limit positioning relies heavily on specialized fixtures, and different workpiece models typically require different tooling structures. When product models change, fixtures need to be replaced or equipment needs to be re-adjusted, making it difficult to meet the needs of multi-variety, small-batch production. Vision-based point cloud registration, on the other hand, usually involves creating a model for a single workpiece model for matching, making it difficult to handle multiple different workpiece models simultaneously within the same work area. Summary of the Invention

[0004] Therefore, it is necessary to provide a multi-workpiece positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the positioning accuracy of multiple workpieces, in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for positioning multiple workpieces. The method includes:

[0006] When at least two workpieces are placed in the workpiece placement area, acquire regional point cloud data collected for the workpiece placement area.

[0007] Cluster the point cloud data of the region to obtain the point cloud data of each workpiece.

[0008] For each workpiece, the workpiece point cloud data of the workpiece is subjected to two-dimensional projection to determine the first geometric feature of the workpiece under the current acquisition view.

[0009] From multiple workpiece point cloud models, a target point cloud model whose second geometric feature matches the first geometric feature under a corresponding preset acquisition view is selected; the preset acquisition view is similar to the current acquisition view.

[0010] The point cloud data of the target point cloud model is registered with the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0011] Secondly, this application also provides a multi-workpiece positioning device. The device includes:

[0012] The regional point cloud acquisition module is used to acquire regional point cloud data collected for the workpiece placement area when at least two workpieces are placed in the workpiece placement area.

[0013] The clustering module is used to cluster the regional point cloud data to obtain the point cloud data of each workpiece.

[0014] The projection module is used to perform two-dimensional projection on the workpiece point cloud data of each workpiece to determine the first geometric feature of the workpiece under the current acquisition view.

[0015] The model filtering module is used to filter target point cloud models from multiple workpiece point cloud models, where the second geometric feature under the corresponding preset acquisition view matches the first geometric feature; the preset acquisition view is similar to the current acquisition view.

[0016] The workpiece positioning module is used to perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method.

[0018] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0019] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0020] The aforementioned multi-workpiece positioning method, apparatus, computer equipment, storage medium, and computer program product, when at least two workpieces are placed in the workpiece placement area, acquire regional point cloud data collected for the workpiece placement area, cluster the regional point cloud data, and effectively separate the regional point cloud data through clustering to obtain individual workpiece point cloud data for each workpiece. For each workpiece, the workpiece point cloud data can first be projected in two dimensions to determine the first geometric feature of the workpiece under the current acquisition view. Then, from multiple workpiece point cloud models, a target point cloud model that matches the second geometric feature under the corresponding preset acquisition view is selected. At this time, the preset acquisition view is similar to the current acquisition view. This matching process introduces the preset acquisition view constraint similar to the current acquisition view, making the model retrieval process have viewpoint consistency. Therefore, it is not necessary to know the workpiece model corresponding to each workpiece in advance to automatically complete the point cloud registration operation and obtain workpiece positioning information, effectively improving the adaptability of the workpiece positioning method in various production scenarios. Since the target point cloud model and the workpiece have similar perspectives and matching geometric features, point cloud registration using the point cloud data of the target point cloud model and the point cloud data of the workpiece can effectively improve the positioning accuracy of each workpiece. Attached Figure Description

[0021] Figure 1 This is an application environment diagram of a multi-workpiece positioning method in one embodiment;

[0022] Figure 2 This is a flowchart illustrating a multi-workpiece positioning method in one embodiment;

[0023] Figure 3 This is a schematic diagram of the process of clustering regional point cloud data to obtain the individual workpiece point cloud data of each workpiece in one embodiment.

[0024] Figure 4 This is a flowchart illustrating the process of determining the first geometric feature of a workpiece under the current acquisition view by performing a two-dimensional projection on the workpiece point cloud data in one embodiment.

[0025] Figure 5 This is a flowchart illustrating how, in one embodiment, a workpiece's external contour area, external contour perimeter, and compactness index are determined based on two-dimensional projection information from the current acquisition viewpoint.

[0026] Figure 6 This is a flowchart illustrating the process of selecting a target point cloud model from multiple workpiece point cloud models that matches the second geometric feature and the first geometric feature under a corresponding preset acquisition viewpoint in one embodiment.

[0027] Figure 7 This is a schematic diagram of a standard workpiece model and its corresponding viewpoint in one embodiment;

[0028] Figure 8 This is a flowchart illustrating the process of registering the point cloud data of the target point cloud model with the point cloud data of the workpiece to obtain the positioning information of the workpiece in one embodiment.

[0029] Figure 9 This is a schematic diagram of a workpiece positioning scenario in one embodiment;

[0030] Figure 10 This is a schematic diagram illustrating the positioning effect of multiple workpieces in one embodiment;

[0031] Figure 11 This is a flowchart illustrating a multi-workpiece positioning method in another embodiment;

[0032] Figure 12 This is a structural block diagram of a multi-workpiece positioning device in one embodiment;

[0033] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] The multi-workpiece positioning method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the workpiece positioning system 102 communicates with the point cloud acquisition device 104 via a network. A data storage system can store the data that the workpiece positioning system 102 needs to process. The data storage system can be integrated into the workpiece positioning system 102 or placed in the cloud or on another network server. When at least two workpieces are placed in the workpiece placement area, the workpiece positioning system 102 can acquire regional point cloud data for the workpiece placement area through the point cloud acquisition device 104, cluster the regional point cloud data, and obtain the individual workpiece point cloud data for each workpiece. For each workpiece, the workpiece positioning system 102 can perform two-dimensional projection on the workpiece point cloud data to determine the first geometric feature of the workpiece under the current acquisition viewpoint. From multiple workpiece point cloud models, it selects a target point cloud model whose second geometric feature under a corresponding preset acquisition viewpoint matches the first geometric feature. The preset acquisition viewpoint is similar to the current acquisition viewpoint. The point cloud data of the target point cloud model is then registered with the workpiece point cloud data to obtain the workpiece's positioning information.

[0036] Among them, the workpiece positioning system 102 refers to a system used to determine the accurate position and orientation of the workpiece in space during the automated manufacturing process. Its core functions may include providing the workpiece coordinate information to robots or automated equipment, enabling the equipment to plan the welding trajectory according to the actual placement position of the workpiece, thereby completing precise processing or operation.

[0037] The workpiece positioning system 102 can be integrated into a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be various production management devices in the production area. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. The server can be a standalone server or a server cluster consisting of multiple servers.

[0038] Among them, the point cloud acquisition device 104 is a hardware device that acquires the three-dimensional spatial coordinates of the surface of the acquisition object and outputs the data in the form of a set of points. For example, the workpiece placement area used to place the workpiece is a tooling table. The point cloud acquisition device 104 can be a three-dimensional camera set on the outer axis of the tooling table. It can move to the shooting position according to a preset path. The workpiece placement area is located within the effective field of view of the three-dimensional camera. The three-dimensional camera can acquire clear and complete point cloud data of the workpiece acquisition area.

[0039] In one embodiment, such as Figure 2 As shown, a multi-workpiece positioning method is provided, which can be applied to... Figure 1 Taking the workpiece positioning system 102 as an example, the following steps are included:

[0040] S202, when at least two workpieces are placed in the workpiece placement area, acquire regional point cloud data collected for the workpiece placement area.

[0041] In this context, the workpiece placement area refers to the pre-defined physical space within the production environment used to place workpieces, typically defined by a specific location on a workbench, tray, pallet, or conveyor belt. The workpiece, on the other hand, refers to an object being processed during the production process. In a welding positioning scenario, the workpiece can refer to a part or component whose spatial orientation needs to be determined through positioning.

[0042] Regional point cloud data is a discrete three-dimensional point set obtained after collecting point cloud data of the workpiece placement area. In addition to the three-dimensional coordinates of each point, it can also include attribute information such as color and reflection intensity of each point.

[0043] For example, when at least two workpieces are placed in the workpiece placement area, the workpiece positioning system can acquire regional point cloud data of the workpiece placement area through a point cloud acquisition device.

[0044] In one embodiment, after acquiring the initial area point cloud data collected by the point cloud acquisition device, the workpiece positioning system can transform the initial area point cloud data from the acquisition device coordinate system to the robot base coordinate system using preset hand-eye calibration parameters, thus obtaining the area point cloud data. For example, it can transform the data from the camera coordinate system to the robot base coordinate system.

[0045] S204, cluster the regional point cloud data to obtain the individual point cloud data of each workpiece.

[0046] Clustering is an unsupervised machine learning or geometric processing method that can divide a discrete spatial point cloud into multiple independent workpiece point cloud data based on the spatial similarity of each point in the regional point cloud data. Each workpiece point cloud data can be similar to a point cluster. The points in each point cluster are spatially adjacent and continuous, belonging to the same connected region or the same object surface. Therefore, each point cluster can correspond to a workpiece.

[0047] For example, the workpiece positioning system can use a preset clustering method to cluster the regional point cloud data to obtain the workpiece point cloud data of each workpiece.

[0048] In one embodiment, the workpiece positioning system is equipped with a clustering model, which can input regional point cloud data into the clustering model to obtain the workpiece point cloud data corresponding to each workpiece.

[0049] In one embodiment, the workpiece positioning system can divide the workpiece point cloud based on a Euclidean cluster extraction algorithm. Specifically, the workpiece point cloud system can calculate the Euclidean distance between points based on the regional point cloud information, compare the Euclidean distance with a preset distance threshold, and group points with Euclidean distances less than the preset distance threshold into the same category. After traversing this process, the point clouds of each workpiece can be effectively separated, forming their own independent point cloud clusters. For example, the preset distance threshold can be 20 millimeters (mm).

[0050] S206. For each workpiece, perform a two-dimensional projection on the workpiece point cloud data to determine the first geometric feature of the workpiece under the current acquisition view.

[0051] Two-dimensional projection refers to the process of mapping the coordinates of a point cloud in three-dimensional space to a two-dimensional plane, i.e., the projection plane. By performing two-dimensional projection on the point cloud data of the workpiece, the workpiece contour information on the corresponding projection plane can be obtained.

[0052] Here, the current acquisition viewpoint refers to the spatial orientation of the point cloud acquisition device relative to the workpiece placement area during actual point cloud data acquisition, such as a top-down vertical view. The first geometric feature is the shape description parameter presented by the workpiece point cloud data after two-dimensional projection under the current acquisition viewpoint. It can be a feature vector or a set of features, such as the projected area or perimeter. Since the projection plane of the two-dimensional projection is a two-dimensional plane perpendicular to the corresponding acquisition line of sight, the first geometric feature of the workpiece under the current acquisition viewpoint can be determined through the workpiece contour information after two-dimensional projection.

[0053] For example, for each workpiece, the workpiece positioning system can perform a two-dimensional projection of the workpiece point cloud data to determine the first geometric feature of the workpiece under the current acquisition view.

[0054] In one embodiment, the workpiece positioning system can pre-configure a geometric feature extraction model, input the workpiece point cloud data into the geometric feature extraction model, and perform two-dimensional projection through the geometric feature extraction model to obtain the first geometric feature of the workpiece under the current acquisition view.

[0055] S208: From multiple workpiece point cloud models, select the target point cloud model whose second geometric feature matches the first geometric feature under the corresponding preset acquisition view.

[0056] The workpiece point cloud model is a reference point cloud data file obtained in advance by scanning standard workpiece parts, used for comparison with the measured workpiece point cloud. Each workpiece point cloud model can correspond to a workpiece model and a collection viewpoint, and the file completely records the point cloud data of the corresponding workpiece model under the corresponding collection viewpoint.

[0057] In one embodiment, workers can simulate actual camera acquisition conditions for each workpiece model and collect visible point cloud data of the workpiece model from various acquisition angles to obtain the point cloud model of each workpiece. In actual use, the second geometric features of the extracted workpiece point cloud model are matched with the first geometric features of the actual captured point cloud to achieve rapid identification of the workpiece model and establishment of correspondence, and to select the optimal matching viewpoint point cloud. Therefore, it can effectively adapt to complex placement scenarios with multiple workpieces and multiple postures.

[0058] Each workpiece point cloud model has its own corresponding acquisition viewpoint. If the second geometric feature of the workpiece point cloud model matches the first geometric feature of the workpiece, then the preset acquisition viewpoint of the workpiece point cloud model is similar to the current acquisition viewpoint. Here, "similar to the current acquisition viewpoint" means that the difference between the preset and current acquisition viewpoints is less than a preset angle threshold, such as 5°.

[0059] For example, after determining the first geometric feature of the workpiece under the current acquisition view, the workpiece positioning system can call up multiple pre-configured workpiece point cloud models, perform feature matching between the first geometric feature and the second geometric feature of each workpiece point cloud model under their respective acquisition view, and filter out the target point cloud model whose second geometric feature matches the first geometric feature. The preset acquisition view of the target point cloud model is similar to the current acquisition view.

[0060] In one embodiment, the workpiece positioning system can call upon pre-collected simulated point clouds of standard workpieces for each workpiece type to obtain point cloud models for each workpiece. The system then performs feature matching between the first geometric feature and the second geometric feature of each workpiece point cloud model from its respective acquisition perspective, filtering out target point cloud models whose second geometric feature matches the first geometric feature. In this way, even if the operator is unaware of the workpiece type of each workpiece placed within the workpiece placement area, they can directly determine the workpiece type and the target point cloud model simultaneously through feature matching.

[0061] In one embodiment, if the workpiece types of all workpieces placed within the workpiece placement area are known, but it is unclear which workpiece matches which workpiece type, the workpiece positioning system can first determine the workpiece types included in the workpiece placement area based on the workpiece attribute information. Then, according to each workpiece type, it retrieves the workpiece point cloud models obtained from the model library by simulating the acquisition of standard workpieces corresponding to each workpiece type under multiple preset acquisition perspectives. Next, it performs feature matching between the first geometric feature and the second geometric feature of each workpiece point cloud model under their respective acquisition perspectives, filtering out the target point cloud models whose second geometric feature matches the first geometric feature. In this way, when the workpiece type is known, the number of workpiece point cloud models requiring matching operations can be effectively reduced, further improving workpiece positioning efficiency.

[0062] In one embodiment, feature matching of the first geometric feature and the second geometric feature can be performed by using a preset matching model or by using a preset feature matching algorithm, such as Euclidean distance, cosine similarity or the absolute difference of each component of the feature vector.

[0063] S210, perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0064] Point cloud registration is a process of aligning the workpiece point cloud data and the target point cloud model's point cloud data in the same coordinate system by calculating the spatial transformation relationship between two point clouds. Point cloud registration uses the target point cloud model as a reference and the workpiece point cloud data as the object to be registered, seeking the optimal rigid body transformation to maximize the overlap between the transformed workpiece point cloud and the target point cloud model.

[0065] Positioning information refers to a set of parameters that describe the precise position and orientation of a workpiece in actual physical space. It is usually given in the form of a transformation relative to a reference coordinate system. For example, it can be a six-degree-of-freedom pose data, including three translational components and three rotational components, which directly indicates the actual orientation of the workpiece in the current space and can be used by robots for path planning, welding or grasping operations.

[0066] For example, the workpiece positioning system can perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0067] In one embodiment, the workpiece positioning system can call a pre-configured point cloud registration model to perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece, thereby obtaining the positioning information of the workpiece.

[0068] In the aforementioned multi-workpiece positioning method, when at least two workpieces are placed in the workpiece placement area, regional point cloud data of the workpiece placement area is acquired. This regional point cloud data is then clustered to effectively separate the data, resulting in individual workpiece point cloud data for each workpiece. For each workpiece, the workpiece point cloud data is first projected in two dimensions to determine the workpiece's first geometric feature from the current acquisition viewpoint. Then, from multiple workpiece point cloud models, a target point cloud model whose second geometric feature matches the first geometric feature from a preset acquisition viewpoint is selected. Since the preset acquisition viewpoint is similar to the current acquisition viewpoint, this matching process introduces a preset acquisition viewpoint constraint, ensuring viewpoint consistency in the model retrieval process. Therefore, point cloud registration can be automatically completed without prior knowledge of the workpiece model corresponding to each workpiece, obtaining workpiece positioning information and effectively improving the adaptability of the workpiece positioning method in various production scenarios. Because the target point cloud model and the workpiece have similar viewpoints and matching geometric features, point cloud registration using the target point cloud model's point cloud data and the workpiece point cloud data can effectively improve the positioning accuracy of each workpiece.

[0069] In one embodiment, such as Figure 3 As shown in step S204, the regional point cloud data is clustered to obtain the individual workpiece point cloud data for each workpiece, including:

[0070] S302, perform planar fitting on the regional point cloud data to determine the position information of the workpiece placement platform in the workpiece placement area.

[0071] Among them, plane fitting refers to the operation process of fitting a planar model of the workpiece placement platform from point cloud data. The workpiece placement platform is the physical surface used to support the workpiece in the workpiece placement area. It is usually a flat metal or plastic platform. Since the workpiece placement platform generally occupies a large planar area in the point cloud, a planar model of the workpiece placement platform can be obtained by performing plane fitting on the regional point cloud data. The planar model can represent the position information of the workpiece placement platform.

[0072] For example, the workpiece positioning system can perform planar fitting on the regional point cloud data to determine the position information of the workpiece placement platform in the workpiece placement area.

[0073] S304: For each sampling point in the regional point cloud data, determine the distance between the sampling point and the workpiece placement platform based on the location information.

[0074] Here, the sampling point is each independent point in the regional point cloud data, and the distance between the sampling point and the workpiece placement plane is the offset of the sampling point in the direction perpendicular to the table surface, reflecting the height difference of the point relative to the table surface.

[0075] For example, for each sampling point in the regional point cloud data, the workpiece positioning system can determine the distance between the sampling point and the workpiece placement platform based on the position information of the workpiece placement platform.

[0076] In one embodiment, the workpiece positioning system can use the Random Sample Consensus (RANSAC) algorithm to fit a plane and obtain a planar model of the workpiece placement platform. The general equation of the plane can be expressed as:

[0077] .

[0078] Where a, b, and c are the components of the plane normal vector, and d is the offset of the plane from the origin. For any point P(x, y, z), its directed distance to the plane can be calculated as:

[0079] .

[0080] S306, if the distance is less than or equal to the plane fitting tolerance, the sampling point is determined as a valid sampling point.

[0081] The plane fitting tolerance is a preset parameter used to determine whether a sampling point belongs to the workpiece placement platform. If the distance is less than or equal to the plane fitting tolerance, the sampling point is considered to belong to a specific workpiece, not the platform, and is therefore a valid sampling point. If the distance is greater than the plane fitting tolerance, the sampling point belongs to the platform but not the workpiece, and is therefore invalid and discarded. Understandably, the plane fitting tolerance can be preset by the operator based on the actual scenario. For example, if it's necessary to remove point cloud data close to the platform and only retain the point cloud data from the upper part of the workpiece, the plane fitting tolerance can be set larger.

[0082] For example, after obtaining the distance between the sampling point and the workpiece placement platform, the workpiece positioning system can compare the distance with a preset plane fitting tolerance. If the distance is less than or equal to the plane fitting tolerance, the sampling point is determined as a valid sampling point.

[0083] For example, δt is the tolerance for plane fitting. If |dist(P)|≥δt, then the point is retained; otherwise, the point is discarded.

[0084] S308 performs clustering processing on each valid sampling point to obtain the workpiece point cloud data for each workpiece.

[0085] For example, after traversing all sampling points and obtaining each valid sampling point, the workpiece positioning system can perform clustering processing on each valid sampling point to obtain the workpiece point cloud data for each workpiece.

[0086] In the above embodiments, by performing planar fitting on the regional point cloud data, the position information of the workpiece placement platform is determined. Based on the position information and the planar fitting tolerance, each sampling point is screened to determine the valid sampling points belonging to the workpiece. Then, clustering is performed on the valid sampling points to effectively remove point clouds that are close to the platform, retaining only the workpiece point cloud, improving the purity of the workpiece point cloud data, reducing the interference of the platform background point cloud, and providing high-quality input for the subsequent independent feature extraction and registration of each workpiece.

[0087] In one embodiment, such as Figure 4 As shown, in S206, the workpiece point cloud data is projected in two dimensions to determine the first geometric feature of the workpiece under the current acquisition viewpoint, including:

[0088] S402, based on the workpiece point cloud data, determine the projection plane of the workpiece under the current acquisition view.

[0089] The projection plane is a two-dimensional mathematical plane used to receive the projection of three-dimensional points. Its spatial orientation is related to the current acquisition viewpoint. For example, the normal vector of the projection plane can be parallel to the direction of the current acquisition viewpoint.

[0090] For example, the workpiece positioning system can determine the projection plane of the workpiece under the current acquisition viewpoint based on the workpiece point cloud data.

[0091] S404 projects the workpiece point cloud data onto the projection plane to obtain the workpiece's two-dimensional projection information.

[0092] Two-dimensional projection information is a two-dimensional point set composed of all projected points, which can reflect the contour shape and surface coverage of the workpiece under the current acquisition viewpoint. The two-dimensional projection information can be represented as a two-dimensional coordinate set, a binary image, a point density cloud, etc. This information loses height information but retains the planar geometry of the workpiece.

[0093] For example, the workpiece positioning system can map each sampling point in the workpiece point cloud data to a corresponding point on the projection plane along the current acquisition viewpoint to obtain the two-dimensional projection information of the workpiece.

[0094] In one embodiment, the workpiece positioning system can use a random sample consistency method to estimate the planar parameters of the workpiece point cloud data, obtaining the normal vector n of the principal plane where the workpiece point cloud lies. The planar parameter estimation function can be:

[0095] .

[0096] Where n = (a, b, c), the normal vector is normalized. , Then, two mutually orthogonal unit vectors u and v are selected in the plane as the plane basis vectors, satisfying:

[0097] .

[0098] And normalize it:

[0099] .

[0100] Construct a local orthogonal coordinate system in the plane:

[0101] .

[0102] For any sampling point P = (x, y, z), its two-dimensional projected coordinates in the plane coordinate system are:

[0103]

[0104] .

[0105] Thus, two-dimensional points are obtained:

[0106] .

[0107] S406, Based on the two-dimensional projection information, determine the first geometric feature of the workpiece under the current acquisition view.

[0108] For example, after obtaining the two-dimensional projection information, the workpiece positioning system can determine the first geometric feature of the workpiece under the current acquisition viewpoint based on the two-dimensional projection information.

[0109] In one embodiment, the first geometric feature may include at least one of the workpiece's outer contour area, outer contour perimeter, or compactness index.

[0110] In the above embodiments, by further projecting the filtered effective point cloud along the current acquisition viewpoint, two-dimensional projection information reflecting the contour shape and surface coverage of the workpiece under the current acquisition viewpoint can be obtained. The first geometric feature of the workpiece is determined based on the two-dimensional projection information, which effectively improves the accuracy of the first geometric feature and provides an accurate data foundation for subsequent feature matching.

[0111] Furthermore, in one embodiment, the first geometric feature includes the outer contour area of ​​the workpiece, the outer contour perimeter, and a compactness index. For example... Figure 5 As shown in S406, based on the two-dimensional projection information, the first geometric feature of the workpiece under the current acquisition viewpoint is determined, including:

[0112] S502, extract the contour trajectory from the two-dimensional projection information to obtain the external contour trajectory of the workpiece.

[0113] Contour trajectory extraction refers to the process of identifying and extracting a series of continuous points on the outermost boundary of a workpiece from two-dimensional projection information. The outer contour trajectory is a closed path formed by connecting all the projection points located on the edge of the workpiece in the direction of connection, such as a closed path formed by connecting in a clockwise or counterclockwise direction, which can describe the geometric curve of the outermost boundary of the two-dimensional projection shape of the workpiece.

[0114] For example, the workpiece positioning system can extract the contour trajectory from the two-dimensional projection information to obtain the external contour trajectory of the workpiece.

[0115] In one embodiment, the workpiece positioning system is pre-configured with a contour trajectory extraction model. Two-dimensional projection information can be input into the contour trajectory extraction model to obtain the external contour trajectory of the workpiece.

[0116] In one embodiment, the workpiece positioning system may use a preset contour trajectory extraction algorithm to extract the external contour trajectory of the workpiece. The contour trajectory extraction algorithm may include, but is not limited to, convex hull algorithm, edge tracking algorithm based on grid image, etc.

[0117] Taking the acquisition of the external contour trajectory using a two-dimensional convex hull algorithm as an example, the two-dimensional projection information can include the set of coordinates of each two-dimensional projection point. The workpiece positioning system can select the two-dimensional projection point with the smallest abscissa or ordinate as the starting point, and then sort the other two-dimensional projection points according to their polar angles relative to the starting point. Then, it traverses the sorted point set in sequence, and determines the turning relationship between the current point and the previous point through the vector cross product. When a right turn or concave structure is formed, the point is removed. Finally, the remaining points are connected to form a convex polygon, which is the external contour trajectory. At the same time, a set of closed polygon contour vertices arranged in sequence can be obtained. This set constitutes the minimum convex boundary surrounding all input points.

[0118] S504, based on the trajectory vertices of the external contour trajectory, determines the area and perimeter of the workpiece's external contour.

[0119] Here, the trajectory vertex is the inflection point of the polyline of the outer contour trajectory. The area of ​​the workpiece's outer contour is the size of the two-dimensional planar region enclosed by the outer contour trajectory, that is, the sum of the areas of all sampling points contained within the closed contour trajectory. The perimeter of the outer contour is the total length of one lap along the outer contour trajectory, that is, the sum of the path lengths traversed from a certain trajectory point along the contour trajectory, passing through all trajectory points in sequence, and returning to the starting point.

[0120] For example, the workpiece positioning system can determine the area and perimeter of the outer contour of the workpiece based on the vertices of the outer contour trajectory.

[0121] In one embodiment, the external contour trajectory may include a set of two-dimensional convex hull vertices:

[0122] .

[0123] The workpiece positioning system can use the polygon area calculation formula (Shoelace formula) to calculate the external contour area A:

[0124] .

[0125] in:

[0126] .

[0127] The workpiece positioning system can calculate the length of the outer contour boundary and obtain the outer contour perimeter parameter P.

[0128] .

[0129] S506, construct the workpiece compactness index based on the area and perimeter of the outer contour.

[0130] The compactness index is a parameter used to quantify the compactness of a two-dimensional shape. It reflects the relationship between the contour area and the perimeter. The index takes a maximum value of 1 for a circle. The narrower the shape or the more complex the edge, the smaller the index value.

[0131] For example, after obtaining the outer contour area and outer contour perimeter of the workpiece, the workpiece positioning system can construct a compactness index of the workpiece based on the outer contour area and outer contour perimeter.

[0132] In one embodiment, the formula for calculating the tightness index of the workpiece positioning system is as follows:

[0133] .

[0134] At this point, the workpiece positioning system can obtain the first geometric features (A, P, C) of the workpiece.

[0135] In the above embodiments, by transforming discrete projection points into closed external contour trajectories, the contour area and contour perimeter under the observation view are accurately obtained. The resulting compactness index essentially reduces the linear influence of absolute size and retains the essential features related to shape complexity, edge curvature, and concavity and convexity characteristics, providing an accurate data foundation for subsequent selection of target point cloud models.

[0136] After obtaining the first geometric features, how to select the target point cloud model based on the first geometric features is a key step in workpiece positioning. In one embodiment, such as... Figure 6 As shown, S208 involves selecting a target point cloud model from multiple workpiece point cloud models whose second geometric feature matches the first geometric feature under a preset acquisition viewpoint, including:

[0137] S602, acquire the second geometric features of multiple workpiece point cloud models under their respective preset acquisition viewpoints.

[0138] The preset acquisition viewpoint is the acquisition viewpoint used when acquiring point clouds for each standard workpiece during the model library construction. Each workpiece point cloud model corresponds to a different preset acquisition viewpoint, meaning that a standard workpiece of a workpiece type can have multiple preset acquisition viewpoints, and each preset acquisition viewpoint corresponds to a workpiece point cloud model.

[0139] In one embodiment, the preset acquisition viewpoint may include acquisition viewpoints in six typical observation directions: front, back, left, right, up, and down.

[0140] The second geometric feature is a shape description parameter extracted from the workpiece point cloud model through two-dimensional projection and contour analysis under a preset acquisition view. It can be understood that the types of features included in the second geometric feature are the same as those included in the first geometric feature. For example, the first geometric feature includes the outer contour area, outer contour perimeter, and compactness index of the workpiece under actual shooting conditions, while the second geometric feature is the outer contour area, outer contour perimeter, and compactness index of the standard workpiece under simulated view acquisition.

[0141] In one embodiment, the second geometric feature can be pre-generated and stored after the workpiece point cloud model is acquired.

[0142] For example, the workpiece positioning system can acquire the second geometric features of multiple workpiece point cloud models under their respective preset acquisition views.

[0143] In one embodiment, such as Figure 7 As shown, for each standard workpiece's CAD model, workers can generate simulated viewpoint clouds in six typical directions (front, back, left, right, top, and bottom) within the model coordinate system. This simulates the point cloud distribution under different camera observation directions. The CAD model under different simulated viewpoints is then defined as a single workpiece point cloud model, and the workpiece point cloud model and its corresponding point cloud data are uploaded to the workpiece positioning system. The workpiece positioning system generates simulated viewpoint point clouds for each standard workpiece under each preset acquisition viewpoint, performs planar extraction and projection, and extracts the outer contour of the projected two-dimensional point set. It also calculates the multidimensional features of the geometric statistics of each standard workpiece at each viewpoint, i.e., each second geometric feature.

[0144] Assuming there are N standard workpieces in the system, for each standard workpiece, a set of multi-dimensional geometric feature vectors, i.e., the second geometric features, can be obtained from the six typical observation directions, ultimately forming the scale:

[0145] .

[0146] Where i represents the standard workpiece number and j represents the observation direction number, this feature database can be used as a condition for subsequent rapid matching with the real-world point cloud.

[0147] S604, based on the first geometric feature and each of the second geometric features, determines the respective feature distances between the workpiece and each workpiece point cloud model.

[0148] Among them, the feature distance is a measure of the difference between the first geometric feature and the second geometric feature, which can be used to reflect the degree of similarity in geometric shape between the workpiece and the corresponding workpiece point cloud model.

[0149] For example, a workpiece positioning system can determine the respective feature distances between the workpiece and each workpiece point cloud model based on a first geometric feature and each of the second geometric features.

[0150] In one embodiment, the statistical feature vector corresponding to the first geometric feature can be represented as:

[0151] .

[0152] The feature database formed by each second geometric feature can be represented as:

[0153] .

[0154] Set the normalized distance metric:

[0155] .

[0156] Where r is the clustering index of the workpiece point cloud in the real-shot point cloud, and W A W P W C For example, the weighting coefficient can take the value W. A =0.3, W P =0.3, W C =0.4.

[0157] S606, for each workpiece point cloud model, if the feature distance corresponding to the workpiece point cloud model is less than the preset distance matching threshold, the workpiece point cloud model is determined as the selected point cloud model of the workpiece.

[0158] The preset distance matching threshold is used to determine whether the workpiece point cloud model and the currently photographed workpiece belong to the same workpiece type. If the feature distance is less than the preset distance matching threshold, it means that the photographed workpiece and the workpiece point cloud model meet the similarity condition in terms of geometry. If the feature distance is greater than or equal to the preset distance matching threshold, it means that the photographed workpiece and the workpiece point cloud model do not meet the similarity condition in terms of geometry and can be directly excluded.

[0159] For example, for each workpiece point cloud model, the workpiece positioning system can compare the feature distance corresponding to the workpiece point cloud model with a preset distance matching threshold. If the feature distance corresponding to the workpiece point cloud model is less than the preset distance matching threshold, the workpiece point cloud model is determined as the selected point cloud model of the workpiece.

[0160] S608, determine the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the workpiece's visual attributes.

[0161] The number of selected point cloud models refers to the number of candidate models remaining after the initial distance screening.

[0162] Workpiece visual attributes are visual attribute features used to further distinguish the similarity between workpieces, in addition to their geometric shape. For example, they may include attributes of complete symmetry, partial symmetry, etc.

[0163] For example, according to normal matching logic, if the feature distance corresponding to the workpiece point cloud model is less than the preset distance matching threshold, it means that the workpiece shape acquired from the preset acquisition viewpoint corresponding to the workpiece point cloud model, for a standard workpiece, is similar to the workpiece shape acquired from the current acquisition viewpoint for the workpiece to be located. Therefore, if the selected point cloud model is a single model, the workpiece positioning system can directly determine the selected point cloud model as the target point cloud model corresponding to the workpiece. If the selected point cloud model is multiple, it means that the workpiece may have the same or similar workpiece shapes under different acquisition views. Therefore, it is necessary to further combine the workpiece's visual attributes to determine the target point cloud model corresponding to the workpiece.

[0164] In one embodiment, a standard workpiece can only be matched with the workpiece point cloud data of one workpiece in the real-world scene using a preset acquisition viewpoint. Regarding matching constraints, the following limitations can be imposed:

[0165] Uniqueness constraint, that is, each standard workpiece is only allowed to establish a matching relationship with a point cloud data in the actual shooting scene from a certain perspective.

[0166] Sequential consistency constraint: The matching process is carried out one by one according to the input order of the workpiece point cloud model.

[0167] Assuming the preset distance matching threshold can be τ, the feature distance between the workpiece and the workpiece point cloud model can be D. i,j,r For the i-th standard workpiece, calculate the feature distance D between the workpiece point cloud model from all viewpoints j of the i-th standard workpiece and all workpiece point cloud data that have not yet been matched. i,j,r And select those that satisfy:

[0168]

[0169] The combination (j*, r*) satisfies:

[0170] ,

[0171] If the point cloud model of the workpiece under viewpoint j is successfully matched with the workpiece r, this workpiece point cloud model is considered the selected point cloud model, and workpiece r is marked as a matched state and will no longer participate in subsequent model matching calculations. The matching results are output sequentially according to the input order to ensure a consistent correspondence between the system output and the workpiece number, facilitating subsequent control flow calls.

[0172] If for all viewpoints of a certain standard workpiece, the following condition is not met under any workpiece point cloud data:

[0173] ,

[0174] This indicates that no matching point cloud data was found for the standard workpiece, meaning there is no corresponding real-world workpiece.

[0175] In the above embodiments, by measuring the feature space distance between the geometric features of the measured workpiece and the second geometric features under the preset acquisition view in the model library, a rapid coarse screening of the workpiece point cloud model is achieved, which effectively reduces the computational complexity of subsequent point cloud matching. At the same time, the target point cloud model corresponding to the workpiece is determined according to the number of selected point cloud models and / or the visual attributes of the workpiece, which effectively reduces the probability of incorrectly matching locally similar or mirror-similar model point clouds with the workpiece due to relying solely on the contour shape, and improves the robustness of matching.

[0176] Furthermore, in one embodiment, the visual attributes of the workpiece include a completely symmetrical attribute. S608, determining the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the visual attributes of the workpiece may include: when there are multiple selected point cloud models and the visual attributes of the workpiece are completely symmetrical, determining the selected point cloud model with the smallest feature distance as the target point cloud model corresponding to the workpiece.

[0177] Among them, the complete symmetry attribute refers to the workpiece having complete rotational symmetry or axisymmetry. If the visual attribute of the workpiece is the complete symmetry attribute, it can be considered that the geometric features generated by the workpiece under each preset acquisition view may be consistent or similar. In this case, the different viewpoints are geometrically equivalent. Therefore, the preset acquisition viewpoint corresponding to any selected point cloud model can be used as a valid matching result, without affecting the subsequent localization and attitude estimation.

[0178] For example, when the number of selected point cloud models is multiple, the workpiece positioning system determines whether the standard workpiece corresponding to each selected point cloud model is a workpiece with complete symmetry attributes based on the model attributes of each selected point cloud model. If so, the workpiece visual attribute is determined to be a complete symmetry attribute, and the selected point cloud model with the minimum feature distance can be determined as the target point cloud model corresponding to the workpiece.

[0179] In one embodiment, for a workpiece with perfect rotational symmetry or axisymmetry, the geometric feature vectors generated from multiple viewpoints may be completely consistent, i.e.:

[0180] .

[0181] In this case, different viewpoints are geometrically equivalent, so any viewpoint that meets the threshold condition can be used as a valid matching result without affecting subsequent localization and attitude estimation. The system can default to selecting the viewpoint corresponding to the minimum distance as the output target point cloud model.

[0182] In the above embodiments, when there are multiple selected point cloud models, it can be first determined whether the visual attributes of the workpiece are completely symmetrical. If they are completely symmetrical, the selected point cloud model with the smallest feature distance can be determined as the target point cloud model corresponding to the workpiece. While meeting the accuracy requirements of subsequent point cloud matching based on the target point cloud model, the matching error can be further reduced and the matching accuracy improved.

[0183] In another embodiment, the visual attributes of the workpiece include partially symmetric attributes. S608, determining the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the workpiece's visual attributes may include: when there are multiple selected point cloud models and the workpiece's visual attributes are partially symmetric, for each selected point cloud model, determining the absolute difference between the feature distance of the selected point cloud model and the feature distances of other selected point cloud models. If all absolute differences are less than a preset scale discrimination threshold, each selected point cloud model is determined as the target point cloud model corresponding to the workpiece.

[0184] Partial symmetry attributes refer to a workpiece that does not possess global rotational symmetry or mirror symmetry in its overall geometry, but only exhibits symmetry from one or more viewpoints. If the workpiece's visual attributes are partially symmetric, it can be assumed that the geometric features generated by the workpiece under one or more preset acquisition viewpoints are likely to be consistent or similar. Further analysis based on the distances between these features can then determine whether the target point cloud model can be directly determined from the geometric shape.

[0185] Among them, the preset scale discrimination threshold is a preset judgment threshold parameter used to determine whether the specific posture direction of the workpiece can be determined at the geometric statistical feature level. If each absolute difference is less than the preset scale discrimination threshold, it means that the specific posture direction of the workpiece in the current real shot cannot be uniquely determined at the geometric statistical feature level. Therefore, each selected point cloud model can be used as the target point cloud model for subsequent matching.

[0186] For example, when the number of selected point cloud models is multiple, the workpiece positioning system determines whether the standard workpiece corresponding to each selected point cloud model is a workpiece with partial symmetry attributes based on the model attributes of each selected point cloud model. If so, the workpiece visual attribute is determined to be a partial symmetry attribute. For each selected point cloud model, the workpiece positioning system can calculate the absolute difference between the feature distance of the selected point cloud model and the feature distance of other selected point cloud models, and compare each absolute difference with a preset scale discrimination threshold. If each absolute difference is less than the preset scale discrimination threshold, each selected point cloud model is determined as the target point cloud model corresponding to the workpiece.

[0187] In one embodiment, for partially symmetrical workpieces, there may be two or more viewpoints where the feature distances all satisfy the following:

[0188] ,

[0189] And the distance difference satisfies:

[0190] .

[0191] in, A preset scale discrimination threshold is used. This situation indicates that the specific orientation of the current workpiece cannot be uniquely determined at the level of geometric statistical features. To avoid misjudgment, taking the existence of two selected point cloud models as an example, in this case, both selected point cloud models can be retained:

[0192] ,

[0193] For use in subsequent steps.

[0194] In the above embodiments, the absolute difference of the feature distance between each selected model and all other selected point cloud models is calculated and compared with a preset scale discrimination threshold to determine whether the target point cloud model cannot be accurately selected from the geometric shape. If it is determined that fine screening cannot be performed, all selected point cloud models are directly determined as the target point cloud model, thereby reducing the risk of matching errors caused by screening mistakes and improving the robustness and success rate of positioning of some symmetrical workpieces.

[0195] In one embodiment, such as Figure 8 As shown, in step S210, the point cloud data of the target point cloud model is registered with the point cloud data of the workpiece to obtain the workpiece's positioning information, including:

[0196] S802: Based on the point cloud data of the target point cloud model and the point cloud data of the workpiece, construct multiple sets of homogeneous transformation matrices.

[0197] Among them, the multiple homogeneous transformation matrices are a collection of rigid body transformation matrices with different parameters generated by the coarse registration algorithm. The homogeneous transformation matrix is ​​a 4×4 matrix, including a 3×3 rotation matrix in the upper left corner, a 3×1 translation vector in the upper right corner, and the last row is used to represent rigid body transformations in three-dimensional space, namely rotation and translation.

[0198] For example, the workpiece positioning system can construct multiple sets of homogeneous transformation matrices based on the point cloud data of the target point cloud model and the workpiece point cloud data.

[0199] In one embodiment, a schematic diagram of the workpiece positioning scenario can be as follows: Figure 9 As shown, the 3D structured light camera 901 is used to acquire point cloud data of the tooling table, and the inverted robot 902 is used to process the workpiece on the tooling table 903 according to production requirements, such as welding. The point cloud data of the target point cloud model is point cloud data transformed to the robot coordinate system, which can be obtained in advance by transforming the model point cloud data captured from different viewpoints to the robot coordinate system. The 3D structured light camera 901 is usually vertically mounted at the end of the robotic arm, and its optical axis direction 904 exhibits a fixed negative Z-axis (Z-Down) characteristic in the robot base coordinate system. A set of "viewpoint-gravity alignment matrix" is preset, denoted as R_pre. For the simulated point cloud P_sim in the six directions (X+, X-, Y+, Y-, Z+, Z-) generated by HPR, rigid body rotation is performed using the corresponding R_pre to uniformly transform its observation principal axis to the negative Z-axis direction. For example, the HPR point cloud generated from the Z+ viewpoint uses the following rotation matrix:

[0200] ,

[0201] It can be converted into point cloud data with Z-axis downward orientation.

[0202] In one embodiment, the workpiece positioning system can use principal component analysis (PCA) to estimate the coarse registration matrix for the point cloud data src of the target point cloud model and the point cloud data tgt of the workpiece.

[0203] (1) Centroid calculation: Calculate the centroids of the source point cloud (point cloud data of the target point cloud model) and the target point cloud (workpiece point cloud data) respectively.

[0204]

[0205] .

[0206] (2) The principal axis coordinates can be obtained from the covariance matrix, eigenvalue decomposition, and right-handed system constraints:

[0207]

[0208] .

[0209] Because there is a sign uncertainty in the principal axis direction of PCA, namely:

[0210] .

[0211] They are mathematically equivalent, therefore, by enumerating and combining the principal axis symbols, we can ensure that:

[0212]

[0213] Under the given conditions, construct a finite set of candidate rotation matrices:

[0214] .

[0215] Then, based on different values ​​of R, solve for the corresponding translation vector t:

[0216] .

[0217] This ultimately results in multiple sets of homogeneous transformation matrices T:

[0218] .

[0219] S804 uses each homogeneous transformation matrix to perform point cloud transformation on the point cloud data of the target point cloud model, and obtains the transformed point cloud data.

[0220] For example, the workpiece positioning system can use each homogeneous transformation matrix to perform point cloud transformation on the point cloud data of the target point cloud model to obtain the transformed point cloud data.

[0221] S806 calculates the root mean square error for each transformed point cloud data to obtain the root mean square error corresponding to each homogeneous transformation matrix.

[0222] The root mean square error (RMSE) calculation involves finding the closest point in the workpiece point cloud data for each sampled point in the transformed point cloud data, calculating the Euclidean distance between them, squaring the squares of these distances, and then taking the square root. This method can be used to quantify the overall deviation between a transformed workpiece point cloud and a target point cloud model. The RMSE, on the other hand, is an indicator of the degree of registration between two sets of point clouds. A smaller RMSE indicates a higher degree of overlap; that is, the RMSE is inversely proportional to the degree of matching between the workpiece point cloud data and the target point cloud model.

[0223] For example, the workpiece positioning system can calculate the root mean square error of each transformed point cloud data to obtain the root mean square error corresponding to each homogeneous transformation matrix.

[0224] S808 determines the homogeneous transformation matrix corresponding to the smallest root mean square error among all root mean square errors as the coarse registration matrix of the workpiece point cloud data.

[0225] Among them, the homogeneous transformation matrix corresponding to the minimum root mean square error is the homogeneous transformation matrix that can make the transformed point cloud data match the workpiece point cloud data to the highest degree.

[0226] The coarse registration matrix is ​​the optimal rigid body transformation matrix that is selected through the coarse registration stage and makes the workpiece point cloud roughly aligned with the model.

[0227] For example, the workpiece positioning system can determine the homogeneous transformation matrix corresponding to the smallest root mean square error among all root mean square errors as the coarse registration matrix of the workpiece point cloud data.

[0228] In one embodiment, the workpiece positioning system can calculate the root mean square error of the transformed source point:

[0229] .

[0230] Candidate matrices with significantly smaller RMSEs are preferred as the coarse registration matrix T. init .

[0231] S810 uses a coarse registration matrix as the initial pose to perform fine registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece, thereby obtaining the workpiece's positioning information.

[0232] The initial pose is the starting spatial position and orientation of the workpiece point cloud relative to the target point cloud model before the fine registration algorithm begins iterative optimization. It can be understood as a rigid body transformation, which gives the initial alignment relationship between the workpiece point cloud and the model point cloud. A good initial pose can prevent fine registration from getting stuck in local optima and accelerate convergence.

[0233] Fine registration is a process that further reduces the distance error between two sets of point clouds by iterative optimization based on coarse registration. It can be achieved by using variant algorithms such as Iterative Closest Point (ICP). The coarse registration matrix is ​​used as the initial value. The closest point correspondence between the two sets of point clouds is repeatedly searched and the sum of squared distances between the point pairs is minimized. The transformation matrix is ​​updated step by step until it converges to a local optimum, and a more accurate rigid body transformation is output.

[0234] For example, the workpiece positioning system can use a coarse registration matrix as the initial pose to perform fine registration of the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0235] In one embodiment, the workpiece positioning system obtains the coarse registration transformation matrix T. initThen, using this as the initial pose, iterative close-point fine registration is performed between the model point cloud and the actual image point cloud to further optimize the rigid body transformation parameters and achieve high-precision alignment. Once a single workpiece has completed fine registration, its final pose in the robot coordinate system can be obtained. By sequentially executing the same process on all input models and their corresponding actual image point cloud matching results, batch localization in multi-workpiece scenarios can be completed, achieving stable and automated pose solving for multiple targets. The final multi-workpiece localization results are shown in the attached figure. Figure 10 As shown, (a) is an evaporator inlet pipe end cap assembly and a compressor bracket, and (b) are different models of evaporator inlet pipe end cap assemblies. The black part in the figure is the actual point cloud, and the white part is the model point cloud.

[0236] In one embodiment, a multi-workpiece positioning method is provided, which can be applied to, for example... Figure 9 In the workpiece positioning scenario shown, before positioning, the operator can perform relevant preparatory work, such as placing the workpiece to be measured on the tooling table and manually confirming its model and quantity. A 3D camera is mounted on the external axis and can move to the shooting position according to a preset path. This shooting point can be flexibly set by the operator according to the workpiece size, posture, and imaging requirements to ensure that the key areas of the workpiece are within the effective field of view of the camera to obtain clear and complete point cloud data. When the external axis moves to the preset shooting point, it triggers the 3D camera to collect data and obtain the original 3D point cloud data of the workpiece surface. After the acquisition is completed, the point cloud data is transformed from the camera coordinate system to the robot base coordinate system through the pre-calibrated hand-eye calibration parameters of the system to obtain the initial area point cloud data.

[0237] like Figure 11 As shown, the method may specifically include the following steps:

[0238] S1101, when at least two workpieces are placed in the workpiece placement area, acquire regional point cloud data collected for the workpiece placement area.

[0239] For example, a workpiece positioning system can use a clipping box filter to extract point clouds within a specified region from the initial region point cloud data to eliminate irrelevant noise interference in the space. The point cloud retained after filtering typically includes platform and workpiece point clouds, along with a small amount of sparse noise. A statistical filtering algorithm is then used to remove residual sparse outlier noise points from the point cloud. The basic principle of this algorithm is to perform statistical analysis on the neighborhood of each point and calculate its average distance to neighboring points. Assuming that the average neighborhood distances of all points form a Gaussian distribution, points whose average distance exceeds the confidence interval can be considered outliers and removed based on a set standard deviation threshold, resulting in updated region point cloud data.

[0240] S1102, Perform planar fitting on the regional point cloud data to determine the position information of the workpiece placement platform in the workpiece placement area.

[0241] S1103, if the distance is less than or equal to the plane fitting tolerance, the sampling point is determined as a valid sampling point.

[0242] S1104 performs clustering processing on each valid sampling point to obtain the workpiece point cloud data for each workpiece.

[0243] S1105, Based on the workpiece point cloud data, determine the projection plane of the workpiece under the current acquisition view.

[0244] S1106, Project the workpiece point cloud data onto the projection plane to obtain the two-dimensional projection information of the workpiece.

[0245] S1107, extract the contour trajectory from the two-dimensional projection information to obtain the external contour trajectory of the workpiece.

[0246] S1108, based on the trajectory vertices of the external contour trajectory, determine the area and perimeter of the workpiece's external contour.

[0247] S1109, construct the workpiece compactness index based on the area and perimeter of the outer contour.

[0248] S1110 defines the external contour area, external contour perimeter, and compactness index as the first geometric feature.

[0249] S1111, acquire the second geometric features of multiple workpiece point cloud models under their respective preset acquisition viewpoints.

[0250] S1112, based on the first geometric feature and each of the second geometric features, determine the respective feature distances between the workpiece and each workpiece point cloud model.

[0251] S1113, For each workpiece point cloud model, if the feature distance corresponding to the workpiece point cloud model is less than the preset distance matching threshold, the workpiece point cloud model is determined as the selected point cloud model of the workpiece.

[0252] S1114, when there are multiple selected point cloud models and the visual attributes of the workpiece are completely symmetrical, the selected point cloud model with the smallest feature distance is determined as the target point cloud model corresponding to the workpiece, and S1118 is executed.

[0253] S1115, when there are multiple selected point cloud models and the visual attributes of the workpiece are partially symmetrical attributes, for each selected point cloud model, determine the absolute difference between the feature distance of the selected point cloud model and the feature distance of other selected point cloud models.

[0254] S1116, if all absolute differences are less than the preset scale discrimination threshold, determine each selected point cloud model as the target point cloud model corresponding to the workpiece, and execute S1118.

[0255] S1117, if the number of selected point cloud models is one, the selected point cloud model is determined as the target point cloud model corresponding to the workpiece, and S1118 is executed.

[0256] S1118. Based on the point cloud data of the target point cloud model and the point cloud data of the workpiece, construct multiple sets of homogeneous transformation matrices.

[0257] S1119, use each homogeneous transformation matrix to perform point cloud transformation on the point cloud data of the target point cloud model to obtain the transformed point cloud data.

[0258] S1120, calculate the root mean square error for each transformed point cloud data to obtain the root mean square error corresponding to each homogeneous transformation matrix.

[0259] S1121, the homogeneous transformation matrix corresponding to the smallest root mean square error among all root mean square errors is determined as the coarse registration matrix of the workpiece point cloud data.

[0260] S1122, using the coarse registration matrix as the initial pose, performs fine registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0261] The aforementioned multi-workpiece positioning method has three key advantages. First, by establishing a multi-model feature set and combining it with the point cloud segmentation results from real-world scenes for unified matching, it can simultaneously identify and locate multiple different workpiece models within the same work area. Compared to existing methods that only register for a single model, this significantly improves the system's versatility and adaptability. Second, by generating multi-view point clouds of the models and establishing corresponding feature sets, the system can cover the appearance features of workpieces under different orientations and perspectives. When similar workpieces are placed in different directions and postures, feature matching can still determine their optimal perspective and spatial pose, avoiding dependence on a uniform placement orientation and improving positioning robustness. Third, it enables automatic identification and precise positioning of different workpiece models in multi-variety, small-batch production scenarios, shortening changeover time, reducing manual intervention, and significantly enhancing the flexible processing capabilities of welding production lines.

[0262] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0263] Based on the same inventive concept, this application also provides a multi-workpiece positioning device for implementing the multi-workpiece positioning method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the multi-workpiece positioning device provided below can be found in the limitations of the multi-workpiece positioning method described above, and will not be repeated here.

[0264] In one embodiment, such as Figure 12 As shown, a multi-workpiece positioning device 1200 is provided, including: a region point cloud acquisition module 1201, a clustering module 1202, a projection module 1203, a model filtering module 1204, and a workpiece positioning module 1205, wherein:

[0265] The area point cloud acquisition module 1201 is used to acquire area point cloud data collected for the workpiece placement area when at least two workpieces are placed in the workpiece placement area.

[0266] Clustering module 1202 is used to cluster regional point cloud data to obtain the point cloud data of each workpiece.

[0267] The projection module 1203 is used to perform two-dimensional projection on the workpiece point cloud data for each workpiece to determine the first geometric feature of the workpiece under the current acquisition view.

[0268] The model filtering module 1204 is used to filter target point cloud models from multiple workpiece point cloud models, where the second geometric feature matches the first geometric feature under the corresponding preset acquisition view; the preset acquisition view is similar to the current acquisition view.

[0269] The workpiece positioning module 1205 is used to perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

[0270] In one embodiment, the clustering module 1202 is used to: perform planar fitting on the regional point cloud data to determine the position information of the workpiece placement platform in the workpiece placement area; for each sampling point in the regional point cloud data, determine the distance between the sampling point and the workpiece placement platform according to the position information; if the distance is less than or equal to the planar fitting tolerance, determine the sampling point as a valid sampling point; and perform clustering processing on each valid sampling point to obtain the workpiece point cloud data of each workpiece.

[0271] In one embodiment, the projection module 1203 is used to: determine the projection plane of the workpiece under the current acquisition view based on the workpiece point cloud data; project the workpiece point cloud data onto the projection plane to obtain the two-dimensional projection information of the workpiece; and determine the first geometric feature of the workpiece under the current acquisition view based on the two-dimensional projection information.

[0272] In one embodiment, the first geometric feature includes the outer contour area, outer contour perimeter, and compactness index of the workpiece. The projection module 1203 is used to: extract the contour trajectory from the two-dimensional projection information to obtain the outer contour trajectory of the workpiece; determine the outer contour area and outer contour perimeter of the workpiece based on the trajectory vertices of the outer contour trajectory; and construct the compactness index of the workpiece based on the outer contour area and outer contour perimeter.

[0273] In one embodiment, the model selection module 1204 is used to: acquire the second geometric features of multiple workpiece point cloud models under their respective corresponding preset acquisition views; determine the feature distances between the workpiece and each workpiece point cloud model based on the first geometric features and each second geometric feature; for each workpiece point cloud model, if the feature distance corresponding to the workpiece point cloud model is less than a preset distance matching threshold, determine the workpiece point cloud model as the selected point cloud model of the workpiece; and determine the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the workpiece visual attributes.

[0274] In one embodiment, the visual attributes of the workpiece include a completely symmetrical attribute. The model selection module 1204 is used to: when there are multiple selected point cloud models and the visual attributes of the workpiece are completely symmetrical, determine the selected point cloud model with the smallest feature distance as the target point cloud model corresponding to the workpiece.

[0275] In one embodiment, the visual attributes of the workpiece include partially symmetrical attributes. The model selection module 1204 is used to: when there are multiple selected point cloud models and the visual attributes of the workpiece are partially symmetrical attributes, for each selected point cloud model, determine the absolute difference between the feature distance of the selected point cloud model and the feature distance of other selected point cloud models; if each absolute difference is less than a preset scale discrimination threshold, determine each selected point cloud model as the target point cloud model corresponding to the workpiece.

[0276] In one embodiment, the workpiece positioning module 1205 is used to: construct multiple sets of homogeneous transformation matrices based on the point cloud data of the target point cloud model and the workpiece point cloud data; perform point cloud transformation on the point cloud data of the target point cloud model using each homogeneous transformation matrix to obtain transformed point cloud data; calculate the root mean square error (RMSE) of each transformed point cloud data to obtain the RMSE corresponding to each homogeneous transformation matrix; determine the homogeneous transformation matrix corresponding to the smallest RMSE among the RMSEs as the coarse registration matrix of the workpiece point cloud data; and perform fine registration on the point cloud data of the target point cloud model and the workpiece point cloud data using the coarse registration matrix as the initial pose to obtain the workpiece positioning information.

[0277] Each module in the aforementioned multi-workpiece positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0278] In one embodiment, a computer device is provided, which may be a server integrating a workpiece positioning system, and its internal structure diagram may be as follows. Figure 13 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data for a multi-workpiece positioning method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-workpiece positioning method.

[0279] Those skilled in the art will understand that Figure 13The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0280] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the specific steps of the above-described multi-workpiece positioning method embodiment.

[0281] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the specific steps of the above-described multi-workpiece positioning method embodiment.

[0282] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the specific steps of the above-described multi-workpiece positioning method embodiment.

[0283] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the acquisition, storage, processing, and transmission of the data all comply with relevant laws and regulations.

[0284] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0285] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0286] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for positioning multiple workpieces, characterized in that, The method includes: When at least two workpieces are placed in the workpiece placement area, acquire regional point cloud data collected for the workpiece placement area. Cluster the point cloud data of the region to obtain the point cloud data of each workpiece. For each workpiece, the workpiece point cloud data of the workpiece is subjected to two-dimensional projection to determine the first geometric feature of the workpiece under the current acquisition view. From multiple workpiece point cloud models, a target point cloud model whose second geometric feature matches the first geometric feature under a corresponding preset acquisition view is selected; the preset acquisition view is similar to the current acquisition view. The point cloud data of the target point cloud model is registered with the point cloud data of the workpiece to obtain the positioning information of the workpiece.

2. The method according to claim 1, characterized in that, The step of clustering the point cloud data of the region to obtain the point cloud data of each workpiece includes: Perform planar fitting on the point cloud data of the region to determine the position information of the workpiece placement platform in the workpiece placement area; For each sampling point in the regional point cloud data, the distance between the sampling point and the workpiece placement platform is determined based on the location information; If the distance is less than or equal to the plane fitting tolerance, the sampling point is determined as a valid sampling point; Clustering is performed on each of the effective sampling points to obtain the workpiece point cloud data for each workpiece.

3. The method according to claim 1, characterized in that, The step of performing a two-dimensional projection on the workpiece point cloud data to determine the first geometric feature of the workpiece under the current acquisition view includes: Based on the workpiece point cloud data, determine the projection plane of the workpiece under the current acquisition view. The workpiece point cloud data is projected onto the projection plane to obtain the two-dimensional projection information of the workpiece; Based on the two-dimensional projection information, the first geometric feature of the workpiece under the current acquisition viewpoint is determined.

4. The method according to claim 3, characterized in that, The first geometric feature includes the outer contour area, outer contour perimeter, and compactness index of the workpiece; The step of determining the first geometric feature of the workpiece under the current acquisition viewpoint based on the two-dimensional projection information includes: The contour trajectory of the workpiece is obtained by extracting the contour trajectory from the two-dimensional projection information. Based on the trajectory vertices of the external contour trajectory, the area and perimeter of the external contour of the workpiece are determined. The compactness index of the workpiece is constructed based on the area and perimeter of the outer contour.

5. The method according to any one of claims 1 to 4, characterized in that, The step of selecting a target point cloud model from multiple workpiece point cloud models whose second geometric feature matches the first geometric feature under a corresponding preset acquisition viewpoint includes: Acquire the second geometric features of multiple workpiece point cloud models under their respective preset acquisition views; Based on the first geometric feature and each of the second geometric features, the feature distance between the workpiece and each of the workpiece point cloud models is determined. For each workpiece point cloud model, if the feature distance corresponding to the workpiece point cloud model is less than a preset distance matching threshold, the workpiece point cloud model is determined as the selected point cloud model of the workpiece. Based on the number of selected point cloud models and / or the visual attributes of the workpiece, determine the target point cloud model corresponding to the workpiece.

6. The method according to claim 5, characterized in that, Workpiece visual attributes include perfect symmetry attributes; The step of determining the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the workpiece's visual attributes includes: When there are multiple selected point cloud models and the visual attributes of the workpiece are completely symmetrical, the selected point cloud model with the smallest feature distance is determined as the target point cloud model corresponding to the workpiece.

7. The method according to claim 5, characterized in that, Workpiece visual attributes include some symmetry attributes; The step of determining the target point cloud model corresponding to the workpiece based on the number of selected point cloud models and / or the workpiece's visual attributes includes: When there are multiple selected point cloud models and the visual attributes of the workpiece are partially symmetrical, for each selected point cloud model, the absolute difference between the feature distance of the selected point cloud model and the feature distance of other selected point cloud models is determined. If each absolute difference is less than a preset scale discrimination threshold, each selected point cloud model is determined as the target point cloud model corresponding to the workpiece.

8. The method according to any one of claims 1 to 4, characterized in that, The step of performing point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece includes: Based on the point cloud data of the target point cloud model and the point cloud data of the workpiece, construct multiple sets of homogeneous transformation matrices; The point cloud data of the target point cloud model are transformed using each homogeneous transformation matrix to obtain the transformed point cloud data. The root mean square error of each transformed point cloud data is calculated to obtain the root mean square error of each homogeneous transformation matrix. The homogeneous transformation matrix corresponding to the smallest root mean square error among all the root mean square errors is determined as the coarse registration matrix of the workpiece point cloud data. Using the coarse registration matrix as the initial pose, fine registration is performed on the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

9. A multi-workpiece positioning device, characterized in that, The device includes: The regional point cloud acquisition module is used to acquire regional point cloud data collected for the workpiece placement area when at least two workpieces are placed in the workpiece placement area. The clustering module is used to cluster the regional point cloud data to obtain the point cloud data of each workpiece. The projection module is used to perform two-dimensional projection on the workpiece point cloud data of each workpiece to determine the first geometric feature of the workpiece under the current acquisition view. The model filtering module is used to filter target point cloud models from multiple workpiece point cloud models, where the second geometric feature under the corresponding preset acquisition view matches the first geometric feature; the preset acquisition view is similar to the current acquisition view. The workpiece positioning module is used to perform point cloud registration between the point cloud data of the target point cloud model and the point cloud data of the workpiece to obtain the positioning information of the workpiece.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images