A digital factory production line calibration system based on 3D point cloud technology
By adopting a three-dimensional point cloud technology system in a digital factory, combining semantic learning and precise registration algorithms, the interference and noise problems of three-dimensional point cloud extraction in complex workshop environments are solved, and high-precision and high-efficiency production line calibration is achieved.
Patent Information
- Application Number
- CN202311423594.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-10-30
AI Technical Summary
In a complex workshop environment, the three-dimensional point cloud extraction of the target product is disturbed, and there is a lot of noise and incompleteness, resulting in a huge number of three-dimensional point clouds, which in turn reduces the accuracy and efficiency of digital factory production line calibration.
The digital factory production line calibration system based on three-dimensional point cloud technology includes data acquisition, data lightweighting, point cloud segmentation, multi-source point cloud registration and relative position deviation correction module between products. Through semantic learning, feature learning, and bounding box- Corresponding point matching algorithm and improved nearest point iteration algorithm to achieve high-precision three-dimensional point cloud segmentation and multi-source point cloud registration to ensure data consistency and position calibration.
It significantly improves the three-dimensional point cloud segmentation accuracy and multi-source point cloud registration accuracy, reduces errors and losses in the production process, and achieves high-precision and high-efficiency production line calibration.
Smart Images

Figure CN117593345B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial automation and digital manufacturing, and in particular to a digital factory production line calibration system based on three-dimensional point cloud technology. Background Art
[0002] With the introduction of the concepts of intelligent manufacturing and digital twins, robotic operations have gradually replaced the traditional manual manufacturing model and become the main manufacturing industry. In order to ensure the safety of the robot operation process, the use of digital factories to simulate robot operations has become a new trend in the manufacturing industry. However, due to the inevitable gap between the simulation environment and the physical environment in terms of the relative position relationship between products, the simulation results are unreliable, which increases the investment in manpower and material resources. Therefore, in modern manufacturing, factory production line calibration of the simulation environment and the physical environment is an important link, which involves the precise coordination of multiple devices and systems to ensure the smooth operation of the production line and the high quality of the products.
[0003] With the rapid development of 3D point cloud acquisition technology, it is more common to use hardware devices such as laser scanners and depth cameras to obtain environmental information. The main process of digital factory production line calibration based on 3D point cloud is to extract the target product points by segmenting the scanned point cloud, and to realize the position calibration of the corresponding products in the digital factory based on the registration algorithm, which ultimately improves the virtual-real consistency of the relative position relationship between products and the credibility of the simulation results. Therefore, in the calibration process, high-precision digital factory production line calibration is particularly important. First, due to the high complexity of the workshop scene and the variety of products, the 3D point cloud extraction of the target product is seriously disturbed; secondly, due to the limitations of equipment and scenes, a large amount of noise is mixed into the 3D point cloud of the target scene, and the 3D point cloud of the target product is incomplete; finally, due to the large number of 3D point clouds, the program processing efficiency is low. The above three types of technical problems have brought great difficulties to the calibration of digital factory production lines. Summary of the invention
[0004] The purpose of the present invention is to provide a high-precision and high-efficiency production line calibration system.
[0005] To achieve the above objectives, the present invention proposes a digital factory production line calibration system based on three-dimensional point cloud technology, comprising the following modules:
[0006] Data acquisition module: used for the definition and generation of 3D point cloud, digital model information and related configuration files of the target scene;
[0007] Data lightweight module: used to perform lightweight processing on the collected data and convert digital model information into three-dimensional point cloud information;
[0008] Point cloud segmentation module: used for semantic learning of the target scene, instance learning of the same type of 3D point cloud and recognition of small target 3D point cloud, segmenting the 3D point cloud information of the target scene and extracting the 3D point cloud of the required product;
[0009] Multi-source point cloud registration module: used to realize the 3D point cloud registration of different sources, calculate the error information between different sources of the same product, integrate and optimize data, and ensure data consistency;
[0010] Relative position correction module between products: used to update the product position information of the digital factory;
[0011] Data visualization module: used to display the calculation results of each module in graphical form.
[0012] Furthermore, the data acquisition module configures the operating environment of the digital factory production line calibration system by inputting the three-dimensional point cloud, digital model file and configuration file information of the target scene.
[0013] Furthermore, the data acquisition module is preset with information of multiple product digital models and configuration files.
[0014] Furthermore, the data lightweight module converts the digital model information into the three-dimensional point cloud through triangular face division and uniform grid method, thereby achieving data lightweight.
[0015] Furthermore, the point cloud segmentation module adopts three methods: scene semantic segmentation based on feature learning algorithm, instance segmentation based on supervoxel proximity, and small target three-dimensional point cloud recognition based on distributed optimization, so as to achieve the segmentation of the three-dimensional point cloud of the target scene from coarse to fine.
[0016] Furthermore, the multi-source point cloud registration module implements the multi-source three-dimensional point cloud registration by using two methods: coarse registration and fine registration; the coarse registration is based on a bounding box-corresponding point matching algorithm, and the fine registration is based on an improved closest point iterative algorithm.
[0017] Furthermore, the relative position correction module between products is guided by the result of the multi-source point cloud registration module, and realizes the calibration of the relative position between products and the production line calibration of the digital factory by modifying the configuration file;
[0018] Furthermore, the data visualization module is respectively connected to the data acquisition module, the data lightweight module, the point cloud segmentation module, the multi-source point cloud registration module, the relative position correction module between products and the data visualization module to display the three-dimensional point cloud, digital model information, digital model lightweight results, point cloud segmentation results, multi-source point cloud registration results and relative position correction results between products of the target scene; at the same time, the results of each module can also be exported for further analysis.
[0019] Furthermore, the steps for using the digital factory production line calibration system based on three-dimensional point cloud technology are as follows:
[0020] Step 1: The user inputs the information of the three-dimensional point cloud, digital model file and related configuration files of the target scene, so as to configure the operating environment of the calibration system;
[0021] Step 2: After completing the environment configuration, start the fully automatic production line calibration and position calibration process. After each calibration module is completed, the calibration system will prompt the user that the current module has been completed and save the calculation results. The user can decide whether to run the next calibration module.
[0022] Compared with the prior art, the advantages of the present invention are:
[0023] 1. Due to the wide variety of products in general target scenes, the direct registration accuracy between scenes is low. The present invention uses a point cloud segmentation module, based on scene semantic segmentation, robot instance segmentation and small target point cloud recognition, to achieve coarse-to-fine segmentation of three-dimensional point clouds, effectively improve the segmentation accuracy, and greatly improve the efficiency of subsequent product calibration.
[0024] 2. The present invention calculates error information through a multi-source point cloud registration module, adopts a coarse registration based on a bounding box-corresponding point matching algorithm and a fine registration based on an improved nearest point iterative algorithm, and realizes high-precision multi-source three-dimensional point cloud registration. According to the final calculation result, the relative position correction module between products is used to calibrate the relative positions of different products, which significantly reduces errors and losses in the production process and realizes high-precision and high-efficiency production line calibration. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 This is a block diagram of the embodiment of the digital factory production line calibration system based on three-dimensional point cloud technology of the present invention. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be further described below.
[0027] This embodiment proposes a digital factory production line calibration system based on three-dimensional point cloud technology. Figure 1 As shown, it includes a data acquisition module 1, a data lightweight module 2, a point cloud segmentation module 3, a multi-source point cloud registration module 4, a relative position correction module 5 between products and a data visualization module 6. The working principles of each module in this embodiment are as follows:
[0028] Data acquisition module 1: Using this module, the user can interact with the digital factory production line calibration system based on 3D point cloud technology. Through this module, the 3D point cloud and digital model files of the target scene input by the user are received to produce a visualization interface; by receiving the relevant configuration file input by the user, which contains the product coordinate system information in the simulation environment, the system operating environment is configured; after the user completes all inputs, the calibration system of this embodiment can start to correct the digital factory products.
[0029] Data lightweight module 2: This module is connected to the data acquisition module 1. After obtaining the target scene digital model information and configuration file information output by the data acquisition module 1, it converts the scene three-dimensional digital model information into a three-dimensional point cloud by integrating triangular face division and uniform grid method on the basis of the three-dimensional digital model file, and combines the product coordinate system information of the above configuration file to realize the production of the final three-dimensional point cloud of the product.
[0030] Point cloud segmentation module 3: Due to the complex variety of products in the scene and the low accuracy of direct registration between scenes, this embodiment uses a point cloud segmentation module 3 to extract the three-dimensional point cloud of the product to be registered before multi-source point cloud registration. This module is connected to the above-mentioned data lightweight module 2, receives the product digital three-dimensional point cloud information output by the data lightweight module 2, segments the three-dimensional point cloud information of the target scene, and extracts the three-dimensional point cloud of the required product. The point cloud segmentation module 3 is divided into three sub-modules: scene semantic segmentation, robot instance segmentation, and small target point cloud recognition; in the scene semantic segmentation module, a three-dimensional point cloud feature learning model is constructed, which integrates machine learning and deep learning algorithms to improve segmentation accuracy; in the robot instance segmentation module, efficient and high-precision robot group instance segmentation is realized based on the super voxel nearest neighbor algorithm; in the small target point cloud recognition module, a geometric feature calculation model is constructed to achieve detailed segmentation of the three-dimensional point cloud.
[0031] Multi-source point cloud registration module 4: This module is connected to the point cloud segmentation module 3, receives a single product 3D point cloud, calculates the deviation between the 3D point cloud information of the digital model and the scanned 3D point cloud information, and is divided into two sub-modules: coarse registration based on the bounding box-corresponding point matching algorithm and fine registration based on the improved nearest point iteration algorithm; in the coarse registration module, the coordinate system information of the fixture in the configuration file is used, and a specific algorithm is used to perform matrix conversion on the scanned real point cloud data and the digital model point cloud information to obtain a conversion matrix T1 of the digital model relative to the real fixture point cloud; in the fine registration module, the conversion matrix T1 obtained in the above coarse registration process is used to multiply the factory point cloud data by the conversion matrix T1 to obtain a new point cloud data; then, in order to enable the virtual factory to be registered with the real factory, the virtual factory point cloud data is multiplied by the conversion matrix T2 to obtain the new point cloud data of the real factory, at which point the fine registration is completed; in different modules, specific registration algorithms are used to integrate and optimize point cloud data from different sources to achieve high-precision multi-source 3D point cloud registration.
[0032] Relative position correction module 5 between products: This module is connected to the multi-source point cloud registration module 4. It uses the results obtained by the precision registration module of the multi-source point cloud registration module 4 to calibrate the relative positions between products through a high-precision algorithm, thereby reducing errors and losses in the manufacturing process and updating the file information of the virtual factory.
[0033] Data visualization module 6: This module is respectively connected to the above-mentioned data acquisition module 1, data lightweight module 2, point cloud segmentation module 3, multi-source point cloud registration module 4 and relative position correction module 5, and is used to intuitively display the numerical calculation results of the above-mentioned modules in the form of graphics for users to observe and analyze. Specifically, this module can display the three-dimensional point cloud of the target scene, digital model information, digital model lightweight results, point cloud segmentation results, multi-source point cloud registration results and relative position correction results between products; at the same time, the results of each module can also be exported for further analysis.
[0034] The practical steps of the calibration system in this embodiment are as follows:
[0035] First, the user inputs the three-dimensional point cloud, digital model file and related configuration file information of the target scene, thereby configuring the operating environment of the calibration system of this embodiment;
[0036] After completing the environment configuration, the digital factory production line calibration system of this embodiment starts the fully automatic production line calibration and position calibration process. After each calibration module is completed, the user will be prompted that the current module has been completed and the calculation results will be saved. The user can decide whether to run the next calibration module.
[0037] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.
Claims
1. A digital factory production line calibration system based on three-dimensional point cloud technology, characterized in that: Includes the following modules: Data acquisition module: used for the definition and generation of 3D point cloud, digital model information and related configuration files of the target scene; Data lightweight module: used to perform lightweight processing on the collected data and convert digital model information into three-dimensional point cloud information; Point cloud segmentation module: used for semantic learning of the target scene, instance learning of the same type of 3D point cloud and recognition of small target 3D point cloud, segmenting the 3D point cloud information of the target scene and extracting the 3D point cloud of the required product; the point cloud segmentation module adopts three methods: scene semantic segmentation based on feature learning algorithm, instance segmentation of supervoxel proximity and distributed optimization of small target 3D point cloud recognition, so as to achieve the segmentation of the 3D point cloud of the target scene from coarse to fine; Multi-source point cloud registration module: used to realize the 3D point cloud registration of different sources, calculate the error information between different sources of the same product, integrate and optimize the data, and ensure the consistency of the data; the multi-source point cloud registration module adopts two methods: coarse registration and fine registration to realize the multi-source 3D point cloud registration; the coarse registration is based on the bounding box-corresponding point matching algorithm, and the fine registration is based on the improved closest point iterative algorithm; Relative position correction module between products: used to update the product position information of the digital factory; Data visualization module: used to display the calculation results of each module in graphical form.
2. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The data acquisition module configures the operating environment of the digital factory production line calibration system by inputting the three-dimensional point cloud, digital model file and configuration file information of the target scene.
3. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The data acquisition module is preset with information of multiple product digital models and configuration files.
4. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The data lightweight module converts the digital model information into the three-dimensional point cloud through triangular face division and uniform grid method, thereby achieving data lightweight.
5. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The relative position correction module between products is guided by the result of the multi-source point cloud registration module, and realizes the calibration of the relative position between products and the production line calibration of the digital factory by correcting the configuration file.
6. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The data visualization module is respectively connected to the data acquisition module, the data lightweight module, the point cloud segmentation module, the multi-source point cloud registration module, the relative position correction module between products and the data visualization module to display the three-dimensional point cloud, digital model information, digital model lightweight results, point cloud segmentation results, multi-source point cloud registration results and relative position correction results between products of the target scene; at the same time, the results of each module can also be exported for further analysis.
7. The digital factory production line calibration system based on three-dimensional point cloud technology according to claim 1 is characterized in that: The specific steps are as follows: Step 1: The user inputs the information of the three-dimensional point cloud, digital model file and related configuration files of the target scene, so as to configure the operating environment of the calibration system; Step 2: After completing the environment configuration, start the fully automatic production line calibration and position calibration process. After each calibration module is completed, the calibration system will prompt the user that the current module has been completed and save the calculation results. The user can decide whether to run the next calibration module.
