A multi-part assembly modeling recognition method, device and medium

Abstract

The application discloses a multi-part combined modeling identification method, which comprises the following steps: S1, acquiring a three-dimensional model of a sample part, and constructing a cloud model library of the sample part according to the three-dimensional model of the sample part; S2, collecting image data of all parts to be identified, and matching the image data with the three-dimensional model of the sample part in the cloud model library to identify all the parts to be identified; S3, selecting a plurality of associated parts from all the parts to be identified, the plurality of associated parts can be combined into one or more integral parts, and a three-dimensional model group corresponding to the plurality of associated parts is obtained; and S4, combining the three-dimensional model group corresponding to the plurality of associated parts to obtain a three-dimensional combined model, and the three-dimensional combined model is used for identification comparison with the integral part composed of the plurality of associated parts.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, specifically to a method, device, and medium for modeling and recognizing multi-part combinations. Background Technology

[0002] In the stamping and assembly processes of automotive sheet metal, inspecting whether stamped sheets and components meet specifications and have any visible damage is essential in intelligent automotive factories. Traditional target recognition algorithms use cameras to capture two-dimensional images of components and then identify the component type. Another common approach is a neural network-based target recognition algorithm, which uses training results from two-dimensional images of components to classify and detect targets based on features extracted from those images. Both of these methods rely on recognizing features within two-dimensional images. However, automotive components are diverse and complex in shape, and two-dimensional images lack dimension, making it difficult to represent the full picture of the target. Therefore, improving the accuracy of existing target recognition methods for automotive component inspection is a problem that needs to be solved. This paper proposes a multi-part assembly modeling and recognition method, equipment, and medium to address these issues. Summary of the Invention

[0003] The purpose of this invention is to provide a method, device and medium for modeling and recognizing multi-part combinations. This method is based on visual recognition algorithms and augmented reality projection technology, which can effectively identify objects and assist operators in confirming the recognition results and reducing misjudgments.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0005] A method for modeling and recognizing multi-part assemblies, the method comprising the following steps:

[0006] S1: Obtain the 3D model of the sample parts and construct a cloud model library of the sample parts based on the 3D model of the sample parts;

[0007] S2: Collect image data of all parts to be identified and match them with the 3D models of sample parts in the cloud model library to identify all parts to be identified;

[0008] S3: Select several related parts from all the parts to be identified. The several related parts can be combined into one or more integral parts to obtain a three-dimensional model group corresponding to the several related parts.

[0009] S4: Combine the three-dimensional models corresponding to several related parts to obtain a three-dimensional combined model. The three-dimensional combined model is used to identify and compare with the overall component composed of several related parts.

[0010] Furthermore, the method for obtaining the 3D model when executing S1 includes the following steps:

[0011] Sa: Obtain a sample component image set and digitize the sample component image set to obtain sample component images;

[0012] Sb: Cluster analysis is performed based on the sample component image set and pose information to obtain clustered images;

[0013] Sc: Feature extraction is performed on the sample component images to obtain feature points;

[0014] Sd: Match the feature points of different images to obtain the corresponding points of the sample component images;

[0015] Se: Based on the sample component image set, pose information and the corresponding points, a binding constraint calculation is performed to obtain a sparse point cloud;

[0016] Sf: Perform multi-view stereo matching based on the sparse point cloud and the clustered image to obtain a dense point cloud, and render the dense point cloud to obtain a three-dimensional model of the sample component.

[0017] Furthermore, when executing S1, the method for obtaining the three-dimensional model includes the following steps: performing a three-dimensional scan on the sample parts to obtain the three-dimensional image information of the sample parts, and then using a three-dimensional modeling tool to construct a three-dimensional model, or directly retrieving the three-dimensional model of the standard parts from an existing database.

[0018] Furthermore, the feature points of the part to be identified are compared with the point cloud models in the 3D model comparison library. When the similarity exceeds 95%, the part to be identified is considered to have been successfully identified.

[0019] This invention provides a multi-part assembly modeling and recognition device, comprising:

[0020] The model library creation module is used to acquire the 3D models of sample parts and to build a cloud model library of sample parts based on the 3D models of sample parts.

[0021] The acquisition module is used to acquire image data of the parts to be identified;

[0022] The identification and comparison module is used to collect image data of the parts to be identified and match it with the cloud model library of sample parts to identify the parts to be identified.

[0023] The matching module is used to select several related parts, which can be combined into a whole component to match the corresponding three-dimensional model.

[0024] The combination comparison module is used to combine the matched 3D models to obtain a combined model, which is used to identify and compare with the overall component composed of several related parts.

[0025] This invention provides an electronic device, comprising:

[0026] Memory, used to store computer programs;

[0027] A processor, configured to execute the computer program to implement the steps of the multi-part assembly modeling and recognition method as described in any one of claims 1 to 4.

[0028] The present invention provides a computer-readable storage medium storing a corresponding program for a multi-part assembly modeling and recognition method, wherein when the corresponding program for the augmented reality-based drawing comparison projection recognition method is executed, it implements the multi-part assembly modeling and recognition method as described in any one of claims 1 to 4.

[0029] The beneficial effects of this invention are as follows:

[0030] 1. This invention allows for the identification of an overall component formed by combining various sub-components by simply sampling the images of the sub-components within the overall component. In a simulated point cloud digital environment, the sub-components can be arbitrarily combined. As long as the actual shape matches the combined component, the combined component can be identified. This is convenient and time-saving, and the sub-components can be freely combined to create various identifiable overall components. Attached Figure Description

[0031] Figure 1 This is a flowchart of the process of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0033] Example

[0034] like Figure 1 As shown, a multi-part assembly modeling and recognition method includes the following steps:

[0035] S1: Obtain the 3D model of the sample parts and construct a cloud model library of the sample parts based on the 3D model of the sample parts;

[0036] Specifically, the method for obtaining a 3D model includes the following steps:

[0037] Sa: Obtain a sample component image set and digitize the sample component image set to obtain sample component images;

[0038] Sb: Cluster analysis is performed based on the sample component image set and pose information to obtain clustered images;

[0039] Sc: Feature extraction is performed on the sample component images to obtain feature points;

[0040] Sd: Match the feature points of different images to obtain the corresponding points of the sample component images;

[0041] Se: Based on the sample component image set, pose information and the corresponding points, a binding constraint calculation is performed to obtain a sparse point cloud;

[0042] Sf: Perform multi-view stereo matching based on the sparse point cloud and the clustered image to obtain a dense point cloud, and render the dense point cloud to obtain a three-dimensional model of the sample component.

[0043] In addition, methods for obtaining 3D models include: performing 3D scanning on sample parts to obtain 3D image information of the sample parts, and then using 3D modeling tools to construct 3D models, or directly retrieving 3D models of standard parts from existing databases.

[0044] S2: Collect image data of all parts to be identified and match them with the 3D models of sample parts in the cloud model library to identify all parts to be identified;

[0045] S3: Select several related parts from all the parts to be identified. The several related parts can be combined into one or more integral parts to obtain a three-dimensional model group corresponding to the several related parts.

[0046] S4: Combine the three-dimensional models corresponding to several related parts to obtain a three-dimensional combined model. The three-dimensional combined model is used to identify and compare with the overall component composed of several related parts.

[0047] Specifically, the feature points of the part to be identified are compared with the point cloud model in the 3D model comparison library. When the similarity exceeds 95%, the part to be identified is considered to have been successfully identified.

[0048] This invention provides a multi-part assembly modeling and recognition device, comprising:

[0049] The model library creation module is used to acquire the 3D models of sample parts and to build a cloud model library of sample parts based on the 3D models of sample parts.

[0050] The acquisition module is used to acquire image data of the parts to be identified;

[0051] The identification and comparison module is used to collect image data of the parts to be identified and match it with the cloud model library of sample parts to identify the parts to be identified.

[0052] The matching module is used to select several related parts, which can be combined into a whole component to match the corresponding three-dimensional model.

[0053] The combination comparison module is used to combine the matched 3D models to obtain a combined model, which is used to identify and compare with the overall component composed of several related parts.

[0054] This invention provides an electronic device, comprising:

[0055] Memory, used to store computer programs;

[0056] A processor, configured to execute the computer program to implement the steps of the multi-part assembly modeling and recognition method as described in any one of claims 1 to 4.

[0057] The present invention provides a computer-readable storage medium storing a corresponding program for a multi-part assembly modeling and recognition method, wherein when the corresponding program for the augmented reality-based drawing comparison projection recognition method is executed, it implements the multi-part assembly modeling and recognition method as described in any one of claims 1 to 4.

[0058] It should also be noted that the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for modeling and recognizing multi-part combinations, characterized in that, The method includes the following steps: S1: Obtain the 3D model of the sample component, and construct a cloud model library of the sample component based on the 3D model; the method for obtaining the 3D model of the sample component includes the following steps: Sa: Obtain a sample component image set and digitize the sample component image set to obtain sample component images; Sb: Cluster analysis is performed based on the sample component image set and pose information to obtain clustered images; Sc: Feature extraction is performed on the sample component images to obtain feature points; Sd: Match the feature points of different images to obtain the corresponding points of the sample component images; Se: Based on the sample component image set, pose information and the corresponding points, a binding constraint calculation is performed to obtain a sparse point cloud; Sf: Perform multi-view stereo matching based on the sparse point cloud and the clustered image to obtain a dense point cloud, and render the dense point cloud to obtain a three-dimensional model of the sample component. S2: Collect image data of all parts to be identified and match them with the 3D models of sample parts in the cloud model library to identify all parts to be identified; S3: Select several related parts from all the parts to be identified. The several related parts can be combined into one or more integral parts to obtain a three-dimensional model group corresponding to the several related parts. S4: Combine the three-dimensional models corresponding to several related parts to obtain a three-dimensional combined model. The three-dimensional combined model is used to identify and compare with the overall component composed of several related parts.

2. The multi-part assembly modeling and recognition method according to claim 1, characterized in that, When executing S1, the method for obtaining the three-dimensional model includes the following steps: performing a three-dimensional scan on the sample parts to obtain the three-dimensional image information of the sample parts, and then using a three-dimensional modeling tool to construct the three-dimensional model, or directly retrieving the three-dimensional model of the sample parts from an existing database.

3. The multi-part assembly modeling and recognition method according to claim 1, characterized in that, The cloud model library includes point cloud models of sample components. The feature points of the component to be identified are compared with the point cloud models in the cloud model library. When the similarity exceeds 95%, the component to be identified is determined to be successfully identified.

4. A multi-part assembly modeling and recognition device, characterized in that, include: The model library creation module is used to acquire 3D models of sample parts and construct a cloud model library of sample parts based on these 3D models. The 3D models of sample parts are acquired through the following steps: Sa: Obtain a sample component image set and digitize the sample component image set to obtain sample component images; Sb: Cluster analysis is performed based on the sample component image set and pose information to obtain clustered images; Sc: Feature extraction is performed on the sample component images to obtain feature points; Sd: Match the feature points of different images to obtain the corresponding points of the sample component images; Se: Based on the sample component image set, pose information and the corresponding points, a binding constraint calculation is performed to obtain a sparse point cloud; Sf: Perform multi-view stereo matching based on the sparse point cloud and the clustered image to obtain a dense point cloud, and render the dense point cloud to obtain a three-dimensional model of the sample component. The acquisition module is used to acquire image data of the parts to be identified; The identification and comparison module is used to collect image data of the parts to be identified and match it with the cloud model library of sample parts to identify the parts to be identified. The matching module is used to select several related parts, which can be combined into a whole component to match the corresponding three-dimensional model. The combination comparison module is used to combine the matched 3D models to obtain a combined model, which is used to identify and compare with the overall component composed of several related parts.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the multi-part assembly modeling and recognition method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a corresponding program for a multi-part assembly modeling and recognition method, which, when executed, implements the multi-part assembly modeling and recognition method as described in any one of claims 1 to 3.

Images