Part positioning method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a part positioning method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a part nesting drawing containing a part graph of a to-be-positioned part and a part point cloud obtained by shooting the to-be-positioned part; performing point cloud filling on the part nesting drawing to obtain template point cloud; carrying out edge feature extraction on the part point cloud by adopting a plurality of edge detection algorithms, and fusing the extracted edge features to obtain a fused point cloud; and based on the feature description of the fusion point cloud, registering the template point cloud and the part point cloud to obtain the part pose of the to-be-positioned part. By adopting the method, the part positioning precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing technology, and in particular to a part positioning method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] In the manufacturing industry, the diversified production and manufacturing modes have put forward higher requirements for the accuracy, speed and flexibility of parts positioning, especially in the field of laser cutting steel plate sorting, which has the characteristics of high precision, non-contact processing, wide range of material applications, high degree of automation, strong customization and safety. This requires the robot to be able to perceive, identify and locate the different parts on a laser-cut steel plate, so as to complete the task of accurately grasping or assembling parts. However, due to the various customized demands for steel plate parts in the market, the required parts are of various types, so it is currently impossible to scan steel plate parts in real time to make positioning templates.

[0003] On the other hand, the gaps between parts after laser cutting of steel plates are small, and the different application scenarios of laser cutting and the influence of the on-site environment will also lead to problems such as skewed parts and defects caused by laser sputtering. These problems will cause the edge information of the steel plate parts after cutting to be weak and unable to accurately locate the position of the steel plate parts. Therefore, a part positioning method that can improve the positioning accuracy of parts is urgently needed. Summary of the invention

[0004] Based on this, it is necessary to provide a part positioning method, device, computer equipment, computer readable storage medium and computer program product that can improve the part positioning accuracy in response to the above technical problems.

[0005] In a first aspect, the present application provides a part positioning method, comprising:

[0006] Obtain a part nesting diagram including a part graphic of the part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0007] Fill the point cloud of the part nesting drawing to obtain the template point cloud;

[0008] Use multiple edge detection algorithms to extract edge features from part point clouds, and fuse the extracted edge features to obtain fused point clouds;

[0009] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0010] In one embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned, including:

[0011] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the template point cloud is adjusted in position and posture based on the first registration relationship;

[0012] The part pose of the part to be positioned is determined based on the pose of the template point cloud after the pose adjustment.

[0013] In one embodiment, determining the part pose of the part to be positioned based on the pose of the template point cloud after the pose adjustment includes:

[0014] Performing a second registration on the template point cloud and the part point cloud after the posture adjustment to obtain a second registration relationship;

[0015] The pose of the template point cloud after pose adjustment is adjusted based on the second registration relationship until the template point cloud after pose adjustment and the fused point cloud overlap, and the pose of the template point cloud after pose adjustment is used as the fused part pose.

[0016] In one embodiment, filling the point cloud of the part nesting drawing to obtain the template point cloud includes:

[0017] Obtain application scenarios of parts to be located;

[0018] Determine the point cloud generation method according to the application scenario, and fill the point cloud of the part nesting diagram according to the point cloud generation method to obtain a point cloud filling diagram;

[0019] Perform point cloud conversion on the point cloud filling map to obtain the template point cloud.

[0020] In one embodiment, multiple edge detection algorithms are used to extract edge features from the part point cloud, including:

[0021] Perform outlier detection on the part point cloud and delete the outlier points from the part point cloud;

[0022] A variety of edge detection algorithms are used to extract edge features from the part point cloud after outliers are removed.

[0023] In one embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned, including:

[0024] De-duplicate the fused point cloud;

[0025] Based on the feature description of the fused point cloud after deduplication, the template point cloud and the fused point cloud after deduplication are registered to obtain the part pose of the part to be located.

[0026] In a second aspect, the present application also provides a parts positioning device, comprising:

[0027] An acquisition module, used to acquire a part nesting diagram including a part graphic of a part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0028] The filling module is used to fill the point cloud of the part nesting drawing to obtain the template point cloud;

[0029] The fusion module is used to extract edge features from the part point cloud using a variety of edge detection algorithms, and fuse the extracted edge features to obtain a fused point cloud;

[0030] The registration module is used to register the template point cloud and the fused point cloud based on the feature description of the fused point cloud to obtain the part pose of the part to be located.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Obtain a part nesting diagram including a part graphic of the part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0033] Fill the point cloud of the part nesting drawing to obtain the template point cloud;

[0034] Use multiple edge detection algorithms to extract edge features from part point clouds, and fuse the extracted edge features to obtain fused point clouds;

[0035] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0037] Obtain a part nesting diagram including a part graphic of the part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0038] Fill the point cloud of the part nesting drawing to obtain the template point cloud;

[0039] Use multiple edge detection algorithms to extract edge features from part point clouds, and fuse the extracted edge features to obtain fused point clouds;

[0040] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] Obtain a part nesting diagram including a part graphic of the part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0043] Fill the point cloud of the part nesting drawing to obtain the template point cloud;

[0044] Use multiple edge detection algorithms to extract edge features from part point clouds, and fuse the extracted edge features to obtain fused point clouds;

[0045] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0046] The above-mentioned part positioning method, device, computer equipment, computer-readable storage medium and computer program product use multiple edge detection algorithms to extract edge features from part point clouds, and fuse the extracted edge features to obtain a fused point cloud, which is beneficial to enhancing the feature description of the fused point cloud, thereby facilitating improving the positioning accuracy of the part; by filling the point cloud of the part nesting diagram containing the part graphics of the part to be positioned with points to obtain a template point cloud, edge features are extracted and features are fused on the part point cloud of the part to be positioned obtained by photographing to obtain a fused point cloud. Since the template point cloud is obtained based on the part nesting diagram, the edge information of the template point cloud is relatively strong. Therefore, the template point cloud and the fused point cloud are aligned based on the feature description of the fused point cloud, which is beneficial to obtaining the accurate part posture of the part to be positioned, thereby improving the part positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A diagram showing an application environment of a parts positioning method in an embodiment;

[0049] Figure 2 A schematic diagram of a process flow of a parts positioning method in one embodiment;

[0050] Figure 3 A schematic diagram of a method for edge feature extraction and fusion in one embodiment;

[0051] Figure 4is a schematic diagram of point cloud registration in one embodiment;

[0052] Figure 5 is a schematic diagram of point cloud filling in one embodiment;

[0053] Figure 6 A schematic diagram of a part image and a part point cloud in one embodiment;

[0054] Figure 7 It is a schematic diagram of the overall process of a part positioning method in one embodiment;

[0055] Figure 8 is a structural block diagram of a part positioning device in one embodiment;

[0056] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The part positioning method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment uses the method applied to the terminal 102 as an example. It can be understood that the method can also be applied to the server 104, and can also be applied to a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. The terminal 102 obtains a part nesting diagram containing a part graphic of the part to be located, and a part point cloud obtained by photographing the part to be located; fills the part nesting diagram with point clouds to obtain a template point cloud; uses a variety of edge detection algorithms to extract edge features from the part point cloud, and fuses the extracted edge features to obtain a fused point cloud; based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are aligned to obtain the part pose of the part to be located. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0059] In an exemplary embodiment, Figure 2 As shown, a part positioning method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, including the following steps 202 to 208. Among them:

[0060] Step 202, obtaining a part nesting diagram including a part graphic of the part to be positioned, and a part point cloud obtained by photographing the part to be positioned.

[0061] The parts to be positioned may be parts in various manufacturing scenarios. For example, they may be steel plate parts cut by laser. According to the market's various customized requirements for steel plate parts, the parts to be positioned may have different shapes. The part graphic is a graphic that has the same shape as the part to be positioned and has a preset ratio to the size of the part to be positioned.

[0062] A part nesting drawing refers to a graphic file containing part graphics. A part nesting drawing may contain at least one part graphic of a part to be positioned. Each part graphic in the part nesting drawing has a clear part edge, and the part nesting drawing may also contain the pose information of each part graphic.

[0063] The part point cloud refers to the point cloud obtained by photographing the part to be located using a depth camera.

[0064] Step 204, fill the point cloud of the part nesting drawing to obtain the template point cloud.

[0065] Among them, point cloud filling is mainly the process of filling point clouds of two-dimensional part nesting drawings according to different templates to obtain three-dimensional point clouds.

[0066] Since the part nesting diagram is a two-dimensional image, laser-cut steel plate parts often have a certain slit and thickness. Therefore, the part nesting diagram can be filled with point clouds to obtain a three-dimensional template point cloud.

[0067] Step 206 , using multiple edge detection algorithms to extract edge features from the part point cloud, and fusing the extracted edge features to obtain a fused point cloud.

[0068] Among them, the edge detection algorithm refers to the algorithm that extracts the edge information of the part from the part point cloud. Common edge detection algorithms include ISS key point detection, normal estimation edge detection, longitude and latitude scanning edge detection, grid division edge detection, Alpha-Shape edge detection, etc. The terminal can use two or more different edge detection algorithms among the above-mentioned multiple edge detection algorithms to extract edge features from the part point cloud respectively. For example, ISS key point detection and normal estimation edge detection can be used to extract edge features from the part point cloud respectively.

[0069] Each edge feature refers to the edge features of the part point cloud extracted by multiple edge detection algorithms. Fusion refers to the process of merging the edge features to obtain a more complete, accurate and dense fused point cloud.

[0070] Since the detection principles of each edge detection algorithm are different, the detected edge features are different. For laser-cut steel plate parts, the gaps between the steel plate parts are small. In addition, the different application scenarios of laser cutting and the influence of the on-site environment will also lead to problems such as part skewness and defects caused by laser sputtering. These problems will cause the edge information of the steel plate parts after cutting to be weak, making it impossible to accurately locate the position of the steel plate parts. Therefore, the fusion of the edge features extracted by multiple edge detection algorithms is beneficial to improving the gap features between the parts to be located, as well as the detection effect of the edge features.

[0071] like Figure 3 FIG. 1 is a schematic diagram of a method for edge feature extraction and fusion in an embodiment. FIG. (a) shows edge features extracted from a part point cloud using ISS key point detection, FIG. (b) shows edge features extracted from a part point cloud using normal estimation edge detection, and FIG. (c) shows a fused point cloud obtained by fusing edge features extracted from two key point detection algorithms.

[0072] Step 208 , based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned.

[0073] The feature description refers to the edge features of the fused point cloud. The terminal extracts edge features from the fused point cloud to obtain the feature description of the fused point cloud.

[0074] Registration refers to the process of fusing the template point cloud and the fused point cloud into a unified coordinate system. Since the part point cloud contains at least one point cloud of the part to be located, the registration can register the template point cloud corresponding to the same part to be located with the corresponding fused point cloud.

[0075] Part pose refers to the position and posture of the part to be positioned. Since the pose of the part graphics of the part to be positioned in the part nesting diagram is known, after registration, the part pose of the part to be positioned can be determined according to the pose of the part to be positioned in the part nesting diagram, thereby realizing the positioning of the part to be positioned.

[0076] In the above-mentioned part positioning method, a variety of edge detection algorithms are used to extract edge features of the part point cloud, and the extracted edge features are fused to obtain a fused point cloud, which is beneficial to enhancing the feature description of the fused point cloud, thereby facilitating improving the positioning accuracy of the part; the template point cloud is obtained by filling the point cloud of the part nesting diagram containing the part graphics of the part to be positioned, and the edge features of the part point cloud to be positioned obtained by shooting are extracted and fused to obtain a fused point cloud. Since the template point cloud is obtained based on the part nesting diagram, the edge information of the template point cloud is relatively strong. Therefore, the template point cloud and the fused point cloud are aligned based on the feature description of the fused point cloud, which is beneficial to obtaining the accurate part posture of the part to be positioned, thereby improving the part positioning accuracy.

[0077] In an exemplary embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned, including: based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the pose of the template point cloud is adjusted based on the first registration relationship; based on the pose of the template point cloud after the pose adjustment, the part pose of the part to be positioned is determined.

[0078] Among them, the first registration is used to find a first registration relationship that makes the posture of the part to be positioned in the template point cloud and the posture of the part to be positioned in the fusion point cloud approximately consistent, which belongs to coarse registration.

[0079] In the case where there are multiple parts to be located, for any part to be located, the current template point cloud corresponding to the current part to be located is determined in the template point cloud, and the current fused point cloud whose similarity with the current part to be located is higher than a preset value is determined in the fused point cloud. The first registration refers to finding a first registration relationship that can approximately align the current template point cloud with the current fused point cloud. In some embodiments, the feature description of the fused point cloud includes point cloud edge features and point cloud normals. The terminal performs a first registration on the template point cloud and the fused point cloud based on the point cloud edge features and the point cloud normals, which refers to finding a first registration relationship so that after the current template point cloud is adjusted in position and posture according to the first registration relationship, the edge features of the current template point cloud after the position and posture adjustment are consistent with the edge features of the current fused point cloud, and the point cloud normal of the current template point cloud after the position and posture adjustment is consistent with the point cloud features of the current fused point cloud.

[0080] The first registration relationship includes a rotation matrix and a translation matrix. The first registration relationship is used to adjust the posture of the current template point cloud, that is, the current template point cloud is rotated and translated. The current template point cloud after posture adjustment is approximately aligned with the current fusion point cloud.

[0081] like Figure 4 FIG. 1 is a schematic diagram of point cloud registration in an embodiment, wherein FIG. (a) shows the template point clouds corresponding to the three parts to be positioned, FIG. (b) shows the fused point clouds, and FIG. (c) shows the registered point clouds.

[0082] Since the pose of each part to be positioned in the template point cloud is known, the terminal can further determine the part pose of the part to be positioned according to the pose of the template point cloud after the pose adjustment.

[0083] In this embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to ensure that the template point cloud after posture adjustment is approximately consistent with the posture of the part to be positioned in the fused point cloud, which is beneficial to determine the part posture of the part to be positioned. This method of first registration improves the accuracy of part positioning.

[0084] In an exemplary embodiment, the part pose of the part to be positioned is determined based on the pose of the template point cloud after pose adjustment, including: performing a second registration on the template point cloud after pose adjustment and the fused point cloud to obtain a second registration relationship; adjusting the pose of the template point cloud after pose adjustment based on the second registration relationship until the template point cloud after pose adjustment and the part point cloud coincide with each other, and using the pose of the template point cloud after pose adjustment as the part pose of the part to be positioned.

[0085] Among them, the second registration is used to find a second registration relationship that makes the posture of the part to be positioned in the template point cloud and the posture of the part to be positioned in the fusion point cloud completely consistent, which belongs to precise registration.

[0086] After the first registration, the template point cloud and the corresponding fused point cloud corresponding to each part to be positioned are roughly aligned. In order to further optimize the spatial pose of the point cloud, a more accurate second registration relationship is obtained to minimize the spatial position difference between the point clouds. The second registration relationship includes a rotation matrix and a translation matrix. In some embodiments, an iterative closest point (ICP) algorithm can be used for the second registration.

[0087] The terminal uses the second registration relationship to adjust the posture of the template point cloud after posture adjustment obtained by using the first registration relationship, and uses the second registration relationship finally found that can make the template point cloud after posture adjustment coincide with the part point cloud as the final second registration relationship.

[0088] Since the template point cloud after posture adjustment obtained according to the second registration relationship coincides with the part point cloud, and the posture of the part to be positioned in the template point cloud is known, the posture of the part to be positioned in the template point cloud after posture adjustment can be determined and used as the part posture of the part to be positioned.

[0089] In this embodiment, after the first configuration, a second registration is further performed to find a second registration relationship so that the template point cloud and the part point cloud after posture adjustment coincide with each other to minimize the spatial position difference between the point clouds. In this way, the template point cloud after posture adjustment obtained by using the second registration relationship can determine the accurate part posture, which is beneficial to improving the positioning accuracy of the part.

[0090] In an exemplary embodiment, a part nesting diagram is filled with point clouds to obtain a template point cloud, including: obtaining an application scenario of the part to be located; determining a point cloud generation method according to the application scenario, and filling the part nesting diagram with point clouds according to the point cloud generation method to obtain a point cloud filling diagram; and performing point cloud conversion on the point cloud filling diagram to obtain a template point cloud.

[0091] The application scenario refers to the actual application scenario of the parts to be positioned, for example, a large slit scenario where the slit is greater than a preset value, a small slit scenario where the slit is not greater than a preset value, a thick part where the thickness is greater than a preset thickness, a thin part where the thickness is not greater than a preset thickness, etc.

[0092] The point cloud generation method refers to the use of different templates to fill in the point cloud. For example, in the small slit scenario, since the slit features of the part point cloud obtained by camera scanning are weak, a template without thickness information is used to fill in the point cloud; in the thick part scenario, a template with thickness information is used; in the large slit scenario, or in the scenario where the part surface features need to be applied, a full information template can be used.

[0093] The terminal has a preset mapping relationship between scenes and generation methods. The terminal searches for the mapping relationship and determines different point cloud generation methods for different application scenarios.

[0094] After the point cloud generation method is determined, the terminal performs point cloud filling on the two-dimensional part nesting diagram according to the point cloud generation method to obtain a three-dimensional point cloud filling diagram. Then, the point cloud filling diagram is converted into a template point cloud. Figure 5 The figure shows a schematic diagram of point cloud filling in an embodiment. It can be seen from the figure that for the application scenario where the internal excess material is removed, the template that retains the internal filling should be used to fill the point cloud of the part nesting diagram; for the application scenario where the internal excess material is not removed, the template that is completely filled should be used to fill the point cloud of the part nesting diagram; for the application scenario where the internal excess material is not removed and the part thickness is greater than the preset thickness, the template that is completely filled and adds thickness information should be used to fill the point cloud of the part nesting diagram.

[0095] In this embodiment, by determining the application scenario of the part to be located, and filling and converting the point cloud of the part nesting diagram according to the point cloud generation method corresponding to the application scenario, the template point cloud is obtained. This is beneficial for flexibly adjusting the point cloud generation method according to different application scenarios, ensuring that the generated template point cloud is closer to the actual part to be located, and is beneficial for improving the accuracy of part positioning during subsequent part positioning.

[0096] In an exemplary embodiment, multiple edge detection algorithms are used to extract edge features from a part point cloud, including: detecting outliers on the part point cloud and deleting outliers from the part point cloud; and extracting edge features from the part point cloud after deleting outliers using multiple edge detection algorithms.

[0097] Among them, the part point cloud of the part to be located obtained by taking a camera often contains a lot of noise. Figure 6 The figure shows a schematic diagram of a part image and a part point cloud in an embodiment. Among them, Figure (a) shows a part image captured by a two-dimensional camera, and Figure (b) shows a part point cloud captured by a three-dimensional camera. For the noise in the part point cloud, a filter can be used to filter out the noise, or an outlier detection algorithm can be used to detect outliers, and then the outliers are deleted from the part point cloud to obtain the part point cloud after the outliers are deleted. Therefore, the terminal can use a variety of edge detection algorithms to extract edge features from the part point cloud after the outliers are deleted.

[0098] In some embodiments, in order to improve the algorithm running speed and alignment accuracy, the terminal can also downsample the template point cloud and the part point cloud after outlier removal, so that the downsampled template point cloud and the downsampled part point cloud are at the same point cloud density.

[0099] In this embodiment, outlier detection is performed, outlier points in the part point cloud are deleted, and edge features are extracted from the part point cloud after the outlier points are deleted, which is beneficial to improving the accuracy of edge features and thus to improving the part positioning accuracy.

[0100] In an exemplary embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are aligned to obtain the part pose of the part to be positioned, including: deduplicating the fused point cloud; based on the feature description of the fused point cloud after deduplication, the template point cloud and the fused point cloud after deduplication are aligned to obtain the part pose of the part to be positioned.

[0101] Among them, in the fused point cloud obtained after point cloud fusion, some point clouds may overlap, resulting in uneven point cloud density affecting the registration accuracy. Therefore, the terminal can use a point cloud deduplication algorithm to perform smooth deduplication operations on the fused point cloud to obtain a fused point cloud after deduplication.

[0102] The terminal can align the template point cloud and the fused point cloud after deduplication based on the feature description of the fused point cloud after deduplication, and obtain the part pose of the part to be located.

[0103] In this embodiment, by performing point cloud deduplication on the fused point cloud, it is helpful to delete duplicate point clouds and avoid uneven point cloud density parts in the fused point cloud, thereby helping to improve the registration accuracy.

[0104] To explain the method and effect of positioning parts in this solution in detail, a most detailed embodiment is described below:

[0105] The following example describes the identification and positioning of steel plate parts after laser cutting. The parts to be positioned are steel plate parts after laser cutting.

[0106] Part positioning methods mainly include part nesting diagram analysis, edge feature extraction, feature fusion, computational feature description, and point cloud registration. Figure 7 Shown is a schematic diagram of the overall process of a part positioning method in one embodiment.

[0107] Among them, in terms of part nesting diagram analysis, the application scenario of the part to be located can be used, and the point cloud generation method can be determined according to the application scenario. The part nesting diagram is filled with point clouds according to the point cloud generation method to obtain a point cloud filling diagram, and the point cloud filling diagram is converted into a template point cloud. In terms of edge feature extraction, outlier point detection is performed on the part point cloud, and outliers are deleted from the part point cloud. A variety of edge detection algorithms are used to extract edge features from the part point cloud after the outliers are deleted. In terms of feature fusion, the extracted edge features are fused to obtain a fused point cloud, and the fused point cloud is deduplicated to obtain a fused point cloud after deduplication. In terms of computational feature description, the feature description of the fused point cloud is calculated. In terms of point cloud registration, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned. Specifically, the point cloud registration includes a first registration and a second registration. Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the pose of the template point cloud is adjusted based on the first registration relationship; the template point cloud and the part point cloud after the pose adjustment are secondly registered to obtain a second registration relationship, and the pose of the template point cloud after the pose adjustment is adjusted based on the second registration relationship until the template point cloud and the fused point cloud after the pose adjustment coincide, and the pose of the template point cloud after the pose adjustment is used as the fused part pose.

[0108] The above-mentioned part positioning method adopts a variety of edge detection algorithms to extract edge features from the part point cloud, and fuses the extracted edge features to obtain a fused point cloud, which is beneficial to enhancing the feature description of the fused point cloud, thereby facilitating improving the positioning accuracy of the part; by filling the point cloud of the part nesting diagram containing the part graphics of the part to be positioned, a template point cloud is obtained, and edge features are extracted and fused on the part point cloud of the part to be positioned obtained by shooting to obtain a fused point cloud. Since the template point cloud is obtained based on the part nesting diagram, the edge information of the template point cloud is relatively strong. Therefore, the template point cloud and the fused point cloud are aligned based on the feature description of the fused point cloud, which is beneficial to obtaining the accurate part posture of the part to be positioned, thereby improving the part positioning accuracy. In addition, the problem of small gaps between steel plate parts after laser cutting, and parts skewness caused by different laser cutting application scenarios and the influence of on-site environment is solved. A variety of edge detection algorithms are used to extract the edge features of the parts, and then the edge features extracted by different algorithms are combined for point cloud fusion. The point cloud deduplication algorithm is used to filter out partially overlapping point clouds caused by feature fusion, which improves the detection effect of gap features and edge features between parts, and greatly improves the accuracy of steel plate part recognition and positioning.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a part positioning device for implementing the part positioning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the part positioning device provided below can refer to the limitations of the part positioning method above, and will not be repeated here.

[0111] In an exemplary embodiment, Figure 8 As shown, a part positioning device 100 is provided, comprising: an acquisition module 120, a filling module 140, a fusion module 160 and a registration module 180, wherein:

[0112] An acquisition module 120 is used to acquire a part nesting diagram including a part graphic of a part to be positioned, and a part point cloud obtained by photographing the part to be positioned;

[0113] A filling module 140 is used to fill the point cloud of the part nesting drawing to obtain a template point cloud;

[0114] A fusion module 160 is used to extract edge features from the part point cloud using a variety of edge detection algorithms, and to fuse the extracted edge features to obtain a fused point cloud;

[0115] The registration module 180 is used to register the template point cloud and the fused point cloud based on the feature description of the fused point cloud to obtain the part pose of the part to be located.

[0116] The above-mentioned part positioning device adopts a variety of edge detection algorithms to extract edge features from the part point cloud, and fuses the extracted edge features to obtain a fused point cloud, which is beneficial to enhancing the feature description of the fused point cloud, thereby facilitating improving the positioning accuracy of the part; by filling the point cloud of the part nesting diagram containing the part graphics of the part to be positioned, a template point cloud is obtained, and edge features are extracted and fused on the part point cloud of the part to be positioned obtained by shooting to obtain a fused point cloud. Since the template point cloud is obtained based on the part nesting diagram, the edge information of the template point cloud is relatively strong. Therefore, the template point cloud and the fused point cloud are aligned based on the feature description of the fused point cloud, which is beneficial to obtaining the accurate part posture of the part to be positioned, thereby improving the part positioning accuracy.

[0117] In one embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned. The registration module 180 is also used to: based on the feature description of the fused point cloud, perform a first registration on the template point cloud and the fused point cloud to obtain a first registration relationship, and adjust the pose of the template point cloud based on the first registration relationship; determine the part pose of the part to be positioned based on the pose of the template point cloud after the pose adjustment.

[0118] In one embodiment, the part pose of the part to be positioned is determined based on the pose of the template point cloud after pose adjustment, and the registration module 180 is also used to: perform a second registration on the template point cloud after pose adjustment and the fused point cloud to obtain a second registration relationship; adjust the pose of the template point cloud after pose adjustment based on the second registration relationship until the template point cloud after pose adjustment and the fused point cloud coincide with each other, and use the pose of the template point cloud after pose adjustment as the part pose of the part to be positioned.

[0119] In one embodiment, the part nesting diagram is filled with point clouds to obtain a template point cloud, and the filling module 140 is also used to: obtain an application scenario of the part to be located; determine a point cloud generation method according to the application scenario, and fill the part nesting diagram with point clouds according to the point cloud generation method to obtain a point cloud filling diagram; perform point cloud conversion on the point cloud filling diagram to obtain a template point cloud.

[0120] In one embodiment, multiple edge detection algorithms are used to extract edge features from the part point cloud. The fusion module 160 is also used to: detect outliers on the part point cloud and delete outliers from the part point cloud; and use multiple edge detection algorithms to extract edge features from the part point cloud after the outliers are deleted.

[0121] In one embodiment, based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned, and the registration module 180 is also used to: deduplicate the fused point cloud; based on the feature description of the fused point cloud after deduplication, the template point cloud and the fused point cloud after deduplication are registered to obtain the part pose of the part to be positioned.

[0122] Each module in the above-mentioned part positioning device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0123] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a part positioning method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0124] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0126] A part nesting diagram containing the part graphics of the part to be located and a part point cloud obtained by photographing the part to be located are obtained; the point cloud of the part nesting diagram is filled to obtain a template point cloud; edge features of the part point cloud are extracted using a variety of edge detection algorithms, and the extracted edge features are fused to obtain a fused point cloud; based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0127] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0128] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the template point cloud is adjusted in posture based on the first registration relationship; the part posture of the part to be positioned is determined based on the posture of the template point cloud after the posture adjustment.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Perform a second registration on the template point cloud and the fused point cloud after the posture adjustment to obtain a second registration relationship; adjust the posture of the template point cloud after the posture adjustment based on the second registration relationship until the template point cloud and the fused point cloud after the posture adjustment coincide with each other, and use the posture of the template point cloud after the posture adjustment as the part posture of the part to be positioned.

[0131] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0132] Obtain the application scenario of the part to be located; determine the point cloud generation method according to the application scenario, and fill the part nesting map with point clouds according to the point cloud generation method to obtain a point cloud filling map; perform point cloud conversion on the point cloud filling map to obtain a template point cloud.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0134] The outliers are detected on the part point cloud and deleted from the part point cloud; a variety of edge detection algorithms are used to extract edge features from the part point cloud after the outliers are deleted.

[0135] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0136] The fused point cloud is deduplicated; based on the feature description of the deduplicated fused point cloud, the template point cloud and the deduplicated fused point cloud are registered to obtain the part pose of the part to be positioned.

[0137] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0138] A part nesting diagram containing the part graphics of the part to be located and a part point cloud obtained by photographing the part to be located are obtained; the point cloud of the part nesting diagram is filled to obtain a template point cloud; edge features of the part point cloud are extracted using a variety of edge detection algorithms, and the extracted edge features are fused to obtain a fused point cloud; based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0140] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the template point cloud is adjusted in posture based on the first registration relationship; the part posture of the part to be positioned is determined based on the posture of the template point cloud after the posture adjustment.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0142] Perform a second registration on the template point cloud and the fused point cloud after the posture adjustment to obtain a second registration relationship; adjust the posture of the template point cloud after the posture adjustment based on the second registration relationship until the template point cloud and the fused point cloud after the posture adjustment coincide with each other, and use the posture of the template point cloud after the posture adjustment as the part posture of the part to be positioned.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] Obtain the application scenario of the part to be located; determine the point cloud generation method according to the application scenario, and fill the part nesting map with point clouds according to the point cloud generation method to obtain a point cloud filling map; perform point cloud conversion on the point cloud filling map to obtain a template point cloud.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] The outliers are detected on the part point cloud and deleted from the part point cloud; a variety of edge detection algorithms are used to extract edge features from the part point cloud after the outliers are deleted.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] The fused point cloud is deduplicated; based on the feature description of the deduplicated fused point cloud, the template point cloud and the deduplicated fused point cloud are registered to obtain the part pose of the part to be positioned.

[0149] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0150] A part nesting diagram containing the part graphics of the part to be located and a part point cloud obtained by photographing the part to be located are obtained; the point cloud of the part nesting diagram is filled to obtain a template point cloud; edge features of the part point cloud are extracted using a variety of edge detection algorithms, and the extracted edge features are fused to obtain a fused point cloud; based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be located.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are first registered to obtain a first registration relationship, and the template point cloud is adjusted in posture based on the first registration relationship; the part posture of the part to be positioned is determined based on the posture of the template point cloud after the posture adjustment.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Perform a second registration on the template point cloud and the fused point cloud after the posture adjustment to obtain a second registration relationship; adjust the posture of the template point cloud after the posture adjustment based on the second registration relationship until the template point cloud and the fused point cloud after the posture adjustment coincide with each other, and use the posture of the template point cloud after the posture adjustment as the part posture of the part to be positioned.

[0155] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0156] Obtain the application scenario of the part to be located; determine the point cloud generation method according to the application scenario, and fill the part nesting map with point clouds according to the point cloud generation method to obtain a point cloud filling map; perform point cloud conversion on the point cloud filling map to obtain a template point cloud.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0158] The outliers are detected on the part point cloud and deleted from the part point cloud; a variety of edge detection algorithms are used to extract edge features from the part point cloud after the outliers are deleted.

[0159] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0160] The fused point cloud is deduplicated; based on the feature description of the deduplicated fused point cloud, the template point cloud and the deduplicated fused point cloud are registered to obtain the part pose of the part to be positioned.

[0161] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0162] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0163] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0164] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A part positioning method, characterized in that: The method comprises: Acquire a part nesting diagram including a part graphic of a part to be positioned, and a part point cloud obtained by photographing the part to be positioned; Filling the part nesting diagram with point clouds to obtain a template point cloud; Using a variety of edge detection algorithms to extract edge features from the part point cloud, and fusing the extracted edge features to obtain a fused point cloud; Based on the feature description of the fused point cloud, the template point cloud and the fused point cloud are registered to obtain the part pose of the part to be positioned.

2. The method according to claim 1, characterized in that The registering the template point cloud and the fused point cloud based on the feature description of the fused point cloud to obtain the part pose of the part to be positioned includes: Based on the feature description of the fused point cloud, perform a first registration on the template point cloud and the fused point cloud to obtain a first registration relationship, and adjust the position and posture of the template point cloud based on the first registration relationship; The part pose of the part to be positioned is determined based on the pose of the template point cloud after the pose adjustment.

3. The method according to claim 2, characterized in that The determining the part pose of the part to be positioned based on the pose of the template point cloud after the pose adjustment comprises: Performing a second registration on the template point cloud after posture adjustment and the fused point cloud to obtain a second registration relationship; The pose of the template point cloud after the pose adjustment is adjusted based on the second registration relationship until the template point cloud after the pose adjustment coincides with the fused point cloud, and the pose of the template point cloud after the pose adjustment is used as the part pose of the part to be positioned.

4. The method according to claim 1, characterized in that: The step of filling the point cloud of the part nesting drawing to obtain a template point cloud includes: Acquire the application scenario of the part to be positioned; Determine a point cloud generation method according to the application scenario, and perform point cloud filling on the part nesting diagram according to the point cloud generation method to obtain a point cloud filling diagram; The point cloud filling map is converted into a template point cloud.

5. The method according to claim 1, characterized in that The edge feature extraction of the part point cloud using a plurality of edge detection algorithms includes: Performing outlier detection on the part point cloud and deleting outlier points from the part point cloud; A variety of edge detection algorithms are used to extract edge features from the part point cloud after outliers are removed.

6. The method according to claim 1, characterized in that The registering the template point cloud and the fused point cloud based on the feature description of the fused point cloud to obtain the part pose of the part to be positioned includes: Deduplication of the fused point cloud; Based on the feature description of the fused point cloud after deduplication of point clouds, the template point cloud and the fused point cloud after deduplication of point clouds are registered to obtain the part pose of the part to be positioned.

7. A parts positioning device, characterized in that: The device comprises: An acquisition module, used to acquire a part nesting diagram including a part graphic of a part to be positioned, and a part point cloud obtained by photographing the part to be positioned; A filling module is used to fill the point cloud of the part nesting diagram to obtain a template point cloud; A fusion module, used to extract edge features from the part point cloud using a variety of edge detection algorithms, and fuse the extracted edge features to obtain a fused point cloud; A registration module is used to register the template point cloud and the fused point cloud based on the feature description of the fused point cloud to obtain the part pose of the part to be located.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images