Complex surface-oriented object contour extraction system and method

Abstract

The invention relates to the technical field of computer vision and image processing, in particular to a complex surface-oriented object contour extraction system and method, and the system comprises an object contour extraction center, an image data module, a contour model construction module, a laser construction module and a fusion acquisition module. According to the method, information of a complex surface object can be comprehensively obtained through multi-view image collection, the image quality is improved through preprocessing, meanwhile, the construction precision and accuracy of a visual contour model are improved along with feature fusion analysis and feature point analysis, high-precision point cloud data are obtained through optical scanning, a laser contour model is obtained, and the accuracy of the laser contour model is improved. And meanwhile, the accuracy and integrity of the visual contour model and the laser contour model are ensured by using a multi-dimensional verification mechanism, and the construction of the final contour model is realized through a model overlapping fusion mode, that is, the precise contour extraction of the target object is realized through a laser contour and visual feature combination mode.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a system and method for extracting object contours for complex surfaces. Background Technology

[0002] In many scenarios such as industrial production, cultural relic protection, and reverse design, it is necessary to obtain the contour information of objects. For objects with flat surfaces and regular shapes, contour extraction is relatively easy. However, in reality, many objects have complex surfaces, such as uneven textures, multiple curved surfaces, or holes, which makes contour extraction very difficult.

[0003] However, contour extraction of complex surface objects presents many challenges. The irregular shape, texture and curvature of their surfaces can easily lead to problems such as missing contours and inaccuracies in traditional contour extraction methods. At the same time, existing methods often lack effective verification methods and cannot guarantee the reliability of the extraction results.

[0004] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a system and method for extracting object contours on complex surfaces to address the aforementioned technical deficiencies. This invention achieves the construction of the final contour model through model overlap and fusion, that is, it achieves accurate extraction of the contour of the target object by combining laser contours and visual features.

[0006] The objective of this invention can be achieved through the following technical solution: a method for extracting object contours for complex surfaces, comprising the following steps:

[0007] Step 1: Perform multimodal image acquisition and preprocessing on target objects with complex surfaces;

[0008] Step 2: Perform feature extraction based on the preprocessed multimodal image, accompanied by feature fusion analysis, to obtain the comprehensive feature value F. At the same time, analyze the preliminary contour based on the comprehensive feature value F.

[0009] Step 3: Based on the feature point analysis of the preliminary contour, a visual contour model of the target object is obtained by combining a pre-set dynamic fitting model.

[0010] Step 4: Acquiring, preprocessing, and constructing a laser contour model of the target object's optical point cloud data using laser scanning technology;

[0011] Step 5: Obtain the final contour model of the target object through multi-dimensional verification analysis and fusion of visual contour model and laser contour model.

[0012] Preferably, the analysis process in step one is as follows:

[0013] Multimodal image acquisition: Multimodal images of the target object are acquired simultaneously using a visible light camera, an infrared camera, and a polarization camera. The multimodal images include visible light images, infrared images, and polarization images.

[0014] Preprocess the multimodal images of the acquired target object.

[0015] Preferably, the analysis process in step two is as follows: Based on the preprocessed multimodal images, extract the gradient features and polarization features of each image in the multimodal images;

[0016] The gradient features and polarization features are fused by a pre-set adaptive weight fusion model to obtain a comprehensive feature value F. The comprehensive feature value F is compared with a preset threshold. Pixels with a comprehensive feature value F greater than or equal to the preset threshold are marked as contour pixels. Through 8-neighborhood connectivity analysis, the contour pixels are connected to form a continuous closed region, which is set as the preliminary contour.

[0017] Preferably, the analysis process in step three is as follows:

[0018] S1: Obtaining the basic feature point set;

[0019] S2: Matching pairs are obtained based on the analysis of the basic feature point set after preliminary screening;

[0020] S3: Reverse verification analysis process for matching pairs;

[0021] S4: The process of evaluating the validity of the effective matching point set;

[0022] S5: Based on the qualified signal, the visual contour model of the target object is obtained by combining the effective matching point set with the pre-set dynamic fitting model.

[0023] Preferably, the analysis process in step four is as follows: optical point cloud data of the target object is obtained through laser scanning technology, the optical point cloud data of the target object is preprocessed, a three-dimensional model of the target object is constructed based on the preprocessed optical point cloud data, and a laser contour model is extracted from the constructed three-dimensional model using a pre-set edge detection algorithm.

[0024] Preferably, the analysis process in step five is as follows: Based on the visual contour model and the laser contour model, the feedback results of smoothness verification and actual size consistency verification are obtained. The feedback results include verification qualified and verification unqualified. The number of verification qualified in the feedback results is obtained, and the number of verification qualified in the feedback results is set as the extraction confidence level. The extraction confidence level is then processed. If the extraction confidence level is equal to the preset threshold, a preliminary qualified signal is generated. When the preliminary qualified signal is generated, the visual contour model and the laser contour model are overlapped and fused to obtain the final contour model.

[0025] If the extracted confidence level is not equal to the preset threshold, an update signal is generated, and the S1 operation is executed immediately in response to the update signal.

[0026] Preferably, the smoothness verification involves calculating the curvature change value of the contour lines in the final contour model and processing the curvature change value. If the curvature change value is greater than a preset curvature change value threshold, the verification is deemed unqualified; if the curvature change value is less than or equal to the preset curvature change value threshold, the verification is deemed qualified. The actual size consistency verification involves obtaining a basic size image of the target object and comparing the size of the target object with the final contour model to obtain the size error of the corresponding part. If the size error exceeds a preset range, the verification is deemed unqualified; if the size error does not exceed the preset range, the verification is deemed qualified.

[0027] The beneficial effects of this invention are as follows:

[0028] (1) This invention can comprehensively acquire information about complex surface objects through multi-view image acquisition, improve image quality through preprocessing, and obtain preliminary contours through feature fusion analysis. At the same time, feature point analysis helps to improve the accuracy and precision of visual contour model construction.

[0029] (2) This invention obtains high-precision point cloud data through optical scanning, extracts contours by combining advanced three-dimensional reconstruction and edge detection algorithms, and obtains laser contour models. At the same time, it uses a multi-dimensional verification mechanism to ensure the accuracy and integrity of the visual contour model and the laser contour model, thereby effectively improving the quality and reliability of contour extraction. It meets the high-precision requirements of contour extraction of complex surface objects in various fields, has good application value and promotion prospects, and realizes the construction of the final contour model through model overlap and fusion, that is, the precise extraction of the contour of the target object is achieved by combining laser contour and visual features. Attached Figure Description

[0030] The invention will now be further described with reference to the accompanying drawings;

[0031] Figure 1 This is a reference diagram of the method of the present invention;

[0032] Figure 2 This is a partial analytical diagram of Embodiment 2 of the present invention;

[0033] Figure 3 This is a flowchart of the system of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments;

[0036] Example 1:

[0037] Please see Figures 1 to 2 As shown, this invention provides a method for extracting object contours for complex surfaces, comprising the following steps:

[0038] Step 1: Perform multimodal image acquisition and preprocessing on target objects with complex surfaces;

[0039] Step 2: Perform feature extraction based on the preprocessed multimodal image, accompanied by feature fusion analysis, to obtain the comprehensive feature value F. At the same time, analyze the preliminary contour based on the comprehensive feature value F.

[0040] Step 3: Based on the feature point analysis of the preliminary contour, a visual contour model of the target object is obtained by combining a pre-set dynamic fitting model.

[0041] Step 4: Acquiring, preprocessing, and constructing a laser contour model of the target object's optical point cloud data using laser scanning technology;

[0042] Step 5: Based on the multi-dimensional verification analysis and fusion of the visual contour model and the laser contour model, the final contour model of the target object is obtained;

[0043] The analysis process of step one, which involves multimodal image acquisition and preprocessing of target objects with complex surfaces, is as follows:

[0044] Multimodal image acquisition: Multimodal images of the target object are acquired simultaneously using a visible light camera, an infrared camera, and a polarization camera. The multimodal images include visible light images, infrared images, and polarization images, including polarization images at 45°, 90°, etc.

[0045] The acquired multimodal images of the target object are preprocessed, including alignment and noise reduction.

[0046] In this embodiment of the invention, the alignment process is as follows: using the visible light image as a reference, the corresponding feature points of the infrared image, polarization image and visible light image are found through ORB feature matching, the affine transformation matrix is ​​calculated, and the infrared image and polarization image are scaled, rotated and translated to achieve pixel-level alignment with the visible light image.

[0047] The denoising process includes: guided filtering for visible light images, wavelet thresholding for infrared images, and polarization-based weighted average denoising for polarized images.

[0048] The analysis process in step two is as follows:

[0049] Based on the preprocessed multimodal images, the gradient features and polarization features of each image in the multimodal images are extracted;

[0050] The gradient features and polarization features are fused by a pre-set adaptive weight fusion model to obtain a comprehensive feature value F. The comprehensive feature value F is compared with a preset threshold. Pixels with a comprehensive feature value F greater than or equal to the preset threshold are marked as contour pixels. Through 8-neighborhood connectivity analysis, the contour pixels are connected to form a continuous closed region, which is set as the preliminary contour.

[0051] There are many connected component labeling algorithms, and they are relatively mature, so they will not be elaborated on here.

[0052] Example 2:

[0053] The process of obtaining and analyzing the visual contour model is as follows:

[0054] S1: Acquisition of basic feature point set: Based on the preliminary contour, the SIFT algorithm is used to extract feature points, and the temperature gradient at the location of the feature point is calculated in combination with the infrared image. The temperature gradient is judged to see if it is greater than the preset temperature gradient threshold. If it is, the corresponding feature point is determined to be a rejection point. If not, the corresponding feature point is determined to be a basic point. A basic feature point set is constructed based on the basic points.

[0055] S2: Based on the analysis of the basic feature point set after preliminary screening, the matching pair is obtained: For the basic feature point set after preliminary screening, the Euclidean distance between the point to be matched in the preliminary contour and the target point in the visible light image is calculated, and the Euclidean distance is discriminated. If the Euclidean distance is less than the preset Euclidean distance threshold, the point to be matched is determined to be a valid matching point, and the valid matching point and the target point are set as a matching pair.

[0056] S3: The reverse verification analysis process of the matching pairs, that is, to perform reverse verification on the obtained matching pairs: calculate the cosine similarity between the target point and the point to be matched, and perform discrimination processing on the cosine similarity. If the cosine similarity is greater than the preset cosine similarity threshold, the corresponding matching pair is determined to be a valid matching pair. If the cosine similarity is less than or equal to the preset cosine similarity threshold, the corresponding matching pair is determined to be a defective matching pair. Based on the valid matching pairs, the valid matching points are obtained, and the set of valid matching points is constructed.

[0057] S4: Validity evaluation process of valid matching point set: Based on the valid matching point set, the number of valid matching points is obtained, and the number of standard contour points of the target object is also obtained. The ratio between the number of valid matching points and the number of standard contour points of the target object is set as the matching integrity coefficient. The matching integrity coefficient is judged to see if it is greater than the preset matching integrity coefficient threshold. If it is, a qualified signal is generated; if not, an unqualified signal is generated. In response to the unqualified signal, the S1 operation is executed again.

[0058] S5: Based on the qualified signal, the visual contour model of the target object is obtained by combining the effective matching point set and the pre-set dynamic fitting model.

[0059] The process of obtaining the laser contour model is as follows:

[0060] Optical point cloud data of the target object is obtained through laser scanning technology;

[0061] For example, when using a laser line scanner to scan a target object, if the minimum feature size of the target object is 5mm, the scanning interval is set to 0.5mm. Since the minimum feature size of the handicraft is 5mm, the scanning interval must meet the requirement that it is no more than 1 / 10 of the minimum feature size of the object. During the scanning process, the laser line scanner emits a laser beam and obtains the three-dimensional coordinate information of the target object's surface based on the laser reflection signal, thereby forming optical point cloud data.

[0062] The optical point cloud data of the target object is preprocessed, and a three-dimensional model of the target object is constructed based on the preprocessed optical point cloud data. The laser contour model is extracted from the constructed three-dimensional model using a pre-set edge detection algorithm.

[0063] For example, the Canny edge detection algorithm is used to identify edge points on the surface of a 3D model. This algorithm finds the position of the edge points by calculating the gradient magnitude and direction, and then connects these edge points in a certain order to form a contour line, thereby obtaining the laser contour model of the target object.

[0064] The process of obtaining the final contour model is as follows:

[0065] Feedback results for smoothness verification and actual size consistency verification are obtained based on the visual contour model and the laser contour model. The feedback results include verification qualified and verification unqualified. The number of verification qualified in the feedback results is obtained and set as the extraction confidence level. The extraction confidence level is then processed. If the extraction confidence level is equal to the preset threshold, a preliminary qualified signal is generated. When the preliminary qualified signal is generated, the visual contour model and the laser contour model are overlapped and fused to obtain the final contour model.

[0066] If the extracted confidence level is not equal to the preset threshold, an update signal is generated, and the S1 operation is executed immediately in response to the update signal to ensure the effectiveness and reliability of the final contour model construction.

[0067] Smoothness verification: Calculate the curvature change value of the contour line in the final contour model, and perform discrimination processing on the curvature change value. If the curvature change value is greater than the preset curvature change value threshold, it is judged as unqualified. If the curvature change value is less than or equal to the preset curvature change value threshold, it is judged as qualified.

[0068] If the curvature change exceeds our preset threshold, such as a sudden large bend appearing on a smooth contour line, it indicates that the contour may be incorrect and the verification will fail.

[0069] Actual size consistency verification: Obtain the basic size image of the target object, compare the size of the target object with the final contour model, and obtain the size error of the corresponding part. If the size error exceeds the preset range, it is judged as unqualified. If the size error does not exceed the preset range, it is judged as qualified.

[0070] Example 3:

[0071] A system for extracting object contours on complex surfaces includes an object contour extraction center, an image data module, a contour model construction module, a laser construction module, and a fusion acquisition module. The object contour extraction center and the image data module have a bidirectional communication connection, the object contour extraction center and the contour model construction module have a bidirectional communication connection, the object contour extraction center and the laser construction module have a unidirectional communication connection, the laser construction module and the fusion acquisition module have a unidirectional communication connection, and the fusion acquisition module and the object contour extraction center have a unidirectional communication connection.

[0072] The image data module is internally equipped with a data acquisition unit, a data processing unit, and an extraction and analysis unit.

[0073] The data acquisition unit is used to acquire multimodal images of the target object;

[0074] The data processing unit is used to preprocess multimodal images.

[0075] The extraction and analysis unit is used to extract and fuse features from the preprocessed multimodal image to obtain a preliminary contour.

[0076] The contour model building module is used to obtain the visual contour model of the target object and send it to the object contour extraction center;

[0077] The laser construction module is used to construct a laser contour model of a target object using laser scanning technology;

[0078] The fusion acquisition module retrieves the visual contour model from the object contour extraction center, performs multi-dimensional verification analysis and fusion with the laser contour model to obtain the final contour model of the target object, and sends the final contour model to the object contour extraction center.

[0079] In summary, this invention comprehensively acquires information about complex surface objects through multi-view image acquisition, improves image quality through preprocessing, and obtains preliminary contours through feature fusion analysis. Furthermore, feature point analysis helps improve the accuracy and precision of visual contour model construction. High-precision point cloud data is acquired through optical scanning, and contours are extracted using advanced 3D reconstruction and edge detection algorithms to obtain a laser contour model. A multi-dimensional verification mechanism ensures the accuracy and integrity of both the visual and laser contour models, effectively improving the quality and reliability of contour extraction. This meets the high-precision requirements for contour extraction of complex surface objects in various fields, demonstrating good application value and promising prospects. Finally, the final contour model is constructed through model overlap and fusion, achieving precise contour extraction of the target object by combining laser contours and visual features.

[0080] The threshold is set for comparative analysis of results to determine whether they are good or bad. The value of the threshold is determined by a combination of large-scale model analysis of sample data and human experience. It can also be adjusted appropriately based on seasonal or common-sense influencing factors.

[0081] The size of the coefficient is a specific value obtained by quantifying each parameter to facilitate subsequent comparison. The size of the coefficient depends on the amount of sample data and the corresponding operating coefficient initially set by those skilled in the art for each set of sample data; as long as it does not affect the proportional relationship between the parameter and the quantified value.

[0082] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for extracting object contours on complex surfaces, characterized in that, Includes the following steps: Step 1: Perform multimodal image acquisition and preprocessing on target objects with complex surfaces; Step 2: Perform feature extraction based on the preprocessed multimodal image, accompanied by feature fusion analysis, to obtain the comprehensive feature value F. At the same time, analyze the preliminary contour based on the comprehensive feature value F. Step 3: Based on the feature point analysis of the preliminary contour, a visual contour model of the target object is obtained by combining a pre-set dynamic fitting model. Step 4: Acquiring, preprocessing, and constructing a laser contour model of the target object's optical point cloud data using laser scanning technology; Step 5: Based on the multi-dimensional verification analysis and fusion of the visual contour model and the laser contour model, the final contour model of the target object is obtained.

2. The method for extracting object contours for complex surfaces according to claim 1, characterized in that, The analysis process in step one is as follows: Multimodal image acquisition: Multimodal images of the target object are acquired simultaneously using a visible light camera, an infrared camera, and a polarization camera. The multimodal images include visible light images, infrared images, and polarization images. Preprocess the multimodal images of the acquired target object.

3. The method for extracting object contours for complex surfaces according to claim 2, characterized in that, Step two analysis process is as follows: Based on the preprocessed multimodal images, extract the gradient features and polarization features of each image in the multimodal images; The gradient features and polarization features are fused by a pre-set adaptive weight fusion model to obtain a comprehensive feature value F. The comprehensive feature value F is compared with a preset threshold. Pixels with a comprehensive feature value F greater than or equal to the preset threshold are marked as contour pixels. Through 8-neighborhood connectivity analysis, the contour pixels are connected to form a continuous closed region, which is set as the preliminary contour.

4. The method for extracting object contours for complex surfaces according to claim 1, characterized in that, The analysis process in step three is as follows: S1: Obtaining the basic feature point set; S2: Matching pairs are obtained based on the analysis of the basic feature point set after preliminary screening; S3: Reverse verification analysis process for matching pairs; S4: The process of evaluating the validity of the effective matching point set; S5: Based on the qualified signal, the visual contour model of the target object is obtained by combining the effective matching point set with the pre-set dynamic fitting model.

5. The method for extracting object contours for complex surfaces according to claim 1, characterized in that, The analysis process in step four is as follows: optical point cloud data of the target object is obtained through laser scanning technology, the optical point cloud data of the target object is preprocessed, a three-dimensional model of the target object is constructed based on the preprocessed optical point cloud data, and the laser contour model is extracted from the constructed three-dimensional model using a pre-set edge detection algorithm.

6. The method for extracting object contours for complex surfaces according to claim 4, characterized in that, Step 5 analysis process is as follows: Based on the visual contour model and laser contour model, the feedback results of smoothness verification and actual size consistency verification are obtained. The feedback results include verification qualified and verification unqualified. The number of verification qualified in the feedback results is obtained, and the number of verification qualified in the feedback results is set as the extraction confidence level. The extraction confidence level is then processed. If the extraction confidence level is equal to the preset threshold, a preliminary qualified signal is generated. When the preliminary qualified signal is generated, the visual contour model and laser contour model are overlapped and fused to obtain the final contour model. If the extracted confidence level is not equal to the preset threshold, an update signal is generated, and the S1 operation is executed immediately in response to the update signal.

7. The method for extracting object contours for complex surfaces according to claim 6, characterized in that, Smoothness verification: Calculate the curvature change value of the contour line in the final contour model, and perform discrimination processing on the curvature change value. If the curvature change value is greater than the preset curvature change value threshold, it is judged as unqualified. If the curvature change value is less than or equal to the preset curvature change value threshold, it is judged as qualified. Actual size consistency verification: Obtain the basic size image of the target object, compare the size of the target object with the final contour model, and obtain the size error of the corresponding part. If the size error exceeds the preset range, it is judged as unqualified. If the size error does not exceed the preset range, it is judged as qualified.

8. A system for extracting object contours for complex surfaces, wherein the system is applied to the method for extracting object contours for complex surfaces according to any one of claims 1-7, characterized in that, It includes an object contour extraction center, an image data module, a contour model construction module, a laser construction module, and a fusion acquisition module. The image data module is internally equipped with a data acquisition unit, a data processing unit, and an extraction and analysis unit.

Images