A method for quantitative detection of pits on container surfaces based on vision and point cloud fusion

Abstract

This invention relates to the field of detection technology and discloses a method for quantitative detection of surface pits on containers based on vision and point cloud fusion. Through intelligent data acquisition and analysis technology, it effectively overcomes the bottlenecks of traditional measurement, simplifies the inspection process, and supports the reliability of container equipment. The specific steps are as follows: 1) Acquire a two-dimensional grayscale image and three-dimensional point cloud data of the container to be inspected, constructing an image-point cloud data pair for subsequent collaborative processing; 2) Input the preprocessed two-dimensional grayscale image into the FastSAM model for instance-level target segmentation to obtain all pit masks, and filter out mask regions that meet the area requirements and represent real pits; 3) Perform operations on all acquired pit masks; 4) When operating on each pit mask, first perform morphological edge expansion operations on the pit mask to generate an expanded region mask; 5) Then, by calculating the difference between the expanded region mask and the original mask, obtain a ring mask surrounding the edge of the original pit.

Description

Technical Field

[0001] This invention relates to the field of detection technology. Specifically, it is a method for quantitative detection of surface pits on containers based on the fusion of vision and point cloud data. Background Technology

[0002] Existing technologies for detecting and measuring the dimensions of pressure vessel dents can be categorized as follows, with the core steps outlined below:

[0003] 1. Manual inspection and traditional measuring instrument techniques:

[0004] Based on manual visual observation, combined with traditional measuring tools such as vernier calipers and micrometers, the location of the dent and its size measurement are completed by relying on the operator's experience.

[0005] Main steps:

[0006] 1. Inspectors shall conduct a comprehensive visual inspection of the pressure vessel surface to identify potential pitting areas;

[0007] 2. Mark the suspected pitted areas to clearly define the measurement targets;

[0008] 3. Select appropriate measuring tools (calipers, micrometers, etc.) and manually fit them to the edge and bottom of the recess;

[0009] 4. Read and record the key geometric parameters of the pit, including its depth, length, and width;

[0010] 5. After the actual inspection is completed, the measurement information is then entered into the corresponding system.

[0011] 2. 3D Measurement Technology:

[0012] Three-dimensional data acquisition equipment such as laser scanning and structured light imaging are used to acquire three-dimensional point cloud data of the container surface, and pit identification and size quantification are realized based on point cloud modeling and analysis.

[0013] Specific steps:

[0014] 1. Deploy laser scanners or structured light imaging equipment, and complete equipment calibration and parameter settings;

[0015] 2. Perform a full-area scan of the pressure vessel surface to collect 3D point cloud data;

[0016] 3. Preprocess the point cloud data, including denoising, registration, and simplification, to eliminate environmental interference and data redundancy;

[0017] 4. Suspected pit areas are selected after modeling the point cloud data;

[0018] 5. Calculate the geometric dimensions of the pit area, such as depth, volume, and maximum diameter.

[0019] However, existing technologies for detecting and measuring the dimensions of pressure vessel dents have the following problems:

[0020] 1. Manual inspection is inefficient and time-consuming.

[0021] 2. Due to the strong subjectivity of human measurement, when the boundaries of the pit are blurred or the shape is irregular, the measurement accuracy may be inaccurate.

[0022] 3. 3D measurement lacks precise two-dimensional position guidance, requiring a 3D scan of the entire pressure vessel before further processing, resulting in a huge amount of data and low processing efficiency.

[0023] 4. 3D scanning equipment is complex to deploy and relies heavily on offline data processing, which cannot meet the real-time measurement needs of industrial sites. Summary of the Invention

[0024] The purpose of this invention is to provide a quantitative detection method for surface pits on containers based on vision and point cloud fusion. This method uses binocular stereo vision technology to acquire surface images and point cloud data, enabling automatic identification and dimensional measurement of pit defects. This replaces traditional manual visual and contact-based measurements, improving detection efficiency and accuracy. In terms of inspection, it can be applied to the periodic maintenance of containers, allowing for efficient investigation and quantitative analysis of pit defects without disassembling the equipment, providing reliable data support for equipment safety assessments. Through intelligent data acquisition and analysis technology, it effectively overcomes traditional measurement bottlenecks, simplifies the inspection process, and supports the reliability of container equipment use.

[0025] This invention is achieved through the following technical solution: a method for quantitative detection of surface pits on containers based on vision and point cloud fusion, comprising the following specific steps:

[0026] 1) Obtain the two-dimensional grayscale image of the container to be detected and the corresponding three-dimensional point cloud data (the acquisition of the two-dimensional grayscale image and the corresponding three-dimensional point cloud data is a public method), and construct an image-point cloud data pair for subsequent collaborative processing;

[0027] 2) Input the preprocessed 2D grayscale image into the FastSAM (Fast Segment Anything Model) model to perform instance-level target segmentation, obtain all pit masks, and filter out the mask regions that meet the area requirements and represent real pits.

[0028] 3) Obtain the corresponding ring mask for all the pit masks obtained by the following operation: When operating on each pit mask, first perform a morphological edge expansion operation on the pit mask to generate an expanded region mask that covers the original mask and the surrounding local background; then, by calculating the difference between the expanded region mask and the original mask, obtain a ring mask that surrounds the edge of the original pit.

[0029] 4) The acquired 3D point cloud data is filtered for outliers based on coordinate thresholds, and then the ring point cloud data corresponding to the ring mask is extracted. Subsequently, the RANSAC (Random Sample Consensus) algorithm is used to perform robust plane fitting on the ring point cloud data to obtain the reference plane equation and the unit normal vector of the reference plane representing the container surface in the pit-free state in this region. The reference plane represents the surface of the container when it is in the original pit-free state.

[0030] 5) Based on the 3D point cloud data after anomaly filtering, the corresponding pit interior point cloud data is extracted by combining the obtained pit mask. Then, the height of each pit interior point relative to the reference plane is calculated by combining the dot product of the unit normal vector of the reference plane and the distance parameter. The maximum value is taken as the maximum depth of the pit.

[0031] 6) Calculate the unit vector using the unit normal vector of the reference plane that is not parallel to the unit normal vector, thereby establishing a local orthogonal coordinate system with the reference plane as the reference; complete the principal vector analysis of the point cloud inside the pit through the local orthogonal coordinate system, and project the point cloud data inside the pit onto the principal direction coordinate axis to calculate the length and width of the pit.

[0032] 7) Construct Delaunay triangulation based on the set of two-dimensional projection points of the pit on the new local orthogonal coordinate system. Multiply the area of ​​each triangular facet by the average height of the three vertices (take non-negative) to obtain the corresponding local volume. Summate all local volumes to obtain the total volume of the pit.

[0033] 8) The pit mask obtained in step 2) is superimposed and fused onto the initially acquired two-dimensional grayscale image, and different colors are used to represent pits at different locations to visualize the pit locations on the initially acquired two-dimensional grayscale image; at the same time, combined with the corresponding parameters obtained from the point cloud analysis of each pit, a final output image containing the identifiers and parameters of all pits is generated.

[0034] To further improve the realization of the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following setup is adopted: In step 1), a two-dimensional grayscale image of the container to be detected and the corresponding three-dimensional point cloud data are acquired simultaneously using a binocular structured light camera.

[0035] To further improve the implementation of the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following configuration is specifically adopted: Step 2) includes the following specific steps:

[0036] 2.1) The two-dimensional grayscale image obtained in step 1) is preprocessed by median filtering and contrast adjustment, and then input into the FastSAM (Fast Segment Anything Model). The FastSAM (Fast Segment Anything Model) model identifies all potential pit regions in the preprocessed two-dimensional grayscale image and generates a fine pit mask (binarization mask) for each identified pit region. This mask accurately outlines the pixel-level contour of the pit in the image.

[0037] 2.2) After obtaining the pit mask, the pit mask is further filtered by area screening. During the filtering, a lower limit of the area threshold is set, and then the pixel area of ​​each pit mask is calculated and compared with the lower limit of the area threshold. Masks that do not meet the requirements are removed, and mask areas that meet the area requirements and represent real pits are retained.

[0038] To further improve the implementation of the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following configuration is specifically adopted: Step 6) includes the following specific steps:

[0039] 6.1) Based on the unit normal vector of the reference plane, select a standard coordinate axis direction in the global coordinate system where the point cloud data is located as an auxiliary vector, and the auxiliary vector is not collinear with the unit normal vector; for example, use the X-axis (1,0,0) or the Y-axis (0,1,0) as the auxiliary direction; the X-axis (1,0,0) is preferred as the auxiliary direction; when the unit normal vector of the reference plane is close to the X-axis (the absolute value of the X component is ≥0.9), the Y-axis (0,1,0) is used instead.

[0040] 6.2) Calculate two unit vectors orthogonal to the unit normal vector and to each other by cross product using the unit normal vector and auxiliary vector of the reference plane; construct a local orthogonal coordinate system using the calculated unit vectors, and then transform the point cloud inside the pit into the local orthogonal coordinate system while preserving the two-dimensional coordinate position;

[0041] 6.3) For the point cloud inside the pit transformed into a local orthogonal coordinate system, calculate the mean coordinate of all points in this local orthogonal coordinate system, and use this mean to perform a global translation of the point cloud inside the pit to center it. Then, perform singular value decomposition on the centered point cloud data inside the pit to extract the two orthogonal principal component directions on the local plane. Project the centered point cloud inside the pit onto the principal axis coordinate system formed by the two orthogonal principal component directions, and calculate the maximum and minimum coordinate values ​​along each principal axis. Use the difference between the two as the span of the corresponding direction. The maximum span is defined as the pit length, and the minimum span is defined as the pit width, thereby obtaining the size parameters of the pit region.

[0042] To further improve the implementation of the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following configuration is specifically adopted: Step 7) includes the following specific steps:

[0043] 7.1) Construct a new local orthogonal coordinate system based on the method in step 6), and project the point cloud inside the pit into the new local orthogonal coordinate system to obtain the corresponding two-dimensional projection point set;

[0044] 7.2) After step 7.1), a Delaunay triangulation is constructed based on the two-dimensional projection point set in the new local orthogonal coordinate system (a public method that can generate a mesh with a better shape and avoid narrow triangles). A set of non-overlapping triangular patches are generated to form a triangular mesh covering the distribution range of the two-dimensional projection point set, which is used to support the subsequent calculation of the pit volume.

[0045] 7.3) Based on the triangular mesh, for a single triangular facet, first calculate its geometric area on the two-dimensional plane; simultaneously, based on the height values ​​of the three vertices of the triangular facet relative to the reference plane, take the arithmetic mean of the three height values ​​as the overall height of the triangular facet, and apply a non-negative constraint to the overall height (i.e., if the average value is negative or zero, it is considered as having no depression); finally, multiply the area of ​​the triangular facet by the overall height of the triangular facet to obtain the volume corresponding to the triangular facet.

[0046] 7.4) Traverse all triangular faces generated by Delaunay triangulation, sum the volumes calculated for each face, and thus obtain the total volume of the pit.

[0047] To further improve the implementation of the container surface pit quantization detection method based on vision and point cloud fusion as described in this invention, the following settings are adopted: the corresponding parameters for each pit include pit length, pit width, total pit volume, maximum pit depth, and pixel area of ​​each pit mask.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] This invention acquires 2D images (two-dimensional grayscale images) and 3D point cloud data (three-dimensional point cloud data) simultaneously using a binocular camera, ensuring the synchronization of the two sets of data and improving the overall stability of the method.

[0050] This invention addresses the issues of low efficiency and long processing time associated with manual inspection methods by employing an AI algorithm that integrates 2D image data and 3D point cloud data to measure container dents, thus meeting the requirements for accuracy and real-time performance in field applications.

[0051] This invention addresses pit measurement by using only a portion of point cloud data for calculation, thereby reducing the computational load while ensuring the accuracy of the calculation results.

[0052] This invention locates pits using 2D images and then measures the corresponding 3D point cloud data, achieving fully automated inspection throughout the entire process. Attached Figure Description

[0053] Figure 1 This is the overall flowchart of the present invention.

[0054] Figure 2 This is a grayscale image of a local surface of the container to be inspected.

[0055] Figure 3 This is a local point cloud image of the container to be tested.

[0056] Figure 4 This is a segmented view of the pits on a local surface of the container to be inspected.

[0057] Figure 5 This is a ring-shaped mask image surrounding the edge of the original pit. Detailed Implementation

[0058] The present invention will be further described in detail below with reference to embodiments, but the implementation of the present invention is not limited thereto.

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0061] Example 1:

[0062] This invention presents a method for quantitatively detecting surface pits on containers based on the fusion of vision and point cloud data. It utilizes binocular stereo vision technology to acquire surface images and point cloud data, enabling automatic identification and dimensional measurement of pit defects. This replaces traditional manual visual and contact-based measurements, improving detection efficiency and accuracy. In terms of inspection, it can be applied to the periodic maintenance of containers, enabling efficient investigation and quantitative analysis of pit defects without disassembling the equipment, providing reliable data support for equipment safety assessments. Through intelligent data acquisition and analysis technology, it effectively overcomes traditional measurement bottlenecks, simplifies the inspection process, and supports the reliability of container equipment use. The specific steps include:

[0063] 1) Obtain the two-dimensional grayscale image of the container to be detected and the corresponding three-dimensional point cloud data (the acquisition of the two-dimensional grayscale image and the corresponding three-dimensional point cloud data is a public method), and construct an image-point cloud data pair for subsequent collaborative processing;

[0064] 2) Input the preprocessed 2D grayscale image into the FastSAM (Fast Segment Anything Model) model to perform instance-level target segmentation, obtain all pit masks, and filter out the mask regions that meet the area requirements and represent real pits.

[0065] 3) Obtain the corresponding ring mask for all the pit masks obtained by the following operation: When operating on each pit mask, first perform a morphological edge expansion operation on the pit mask to generate an expanded region mask that covers the original mask and the surrounding local background; then, by calculating the difference between the expanded region mask and the original mask, obtain a ring mask that surrounds the edge of the original pit.

[0066] 4) The acquired 3D point cloud data is filtered for outliers based on coordinate thresholds, and then the ring point cloud data corresponding to the ring mask is extracted. Subsequently, the RANSAC (Random Sample Consensus) algorithm is used to perform robust plane fitting on the ring point cloud data to obtain the reference plane equation and the unit normal vector of the reference plane representing the container surface in the pit-free state in this region. The reference plane represents the surface of the container when it is in the original pit-free state.

[0067] 5) Based on the 3D point cloud data after anomaly filtering, the corresponding pit interior point cloud data is extracted by combining the obtained pit mask. Then, the height of each pit interior point relative to the reference plane is calculated by combining the dot product of the unit normal vector of the reference plane and the distance parameter. The maximum value is taken as the maximum depth of the pit.

[0068] 6) Calculate the unit vector using the unit normal vector of the reference plane that is not parallel to the unit normal vector, thereby establishing a local orthogonal coordinate system with the reference plane as the reference; complete the principal vector analysis of the point cloud inside the pit through the local orthogonal coordinate system, and project the point cloud data inside the pit onto the principal direction coordinate axis to calculate the length and width of the pit.

[0069] 7) Construct Delaunay triangulation based on the set of two-dimensional projection points of the pit on the new local orthogonal coordinate system. Multiply the area of ​​each triangular facet by the average height of the three vertices (take non-negative) to obtain the corresponding local volume. Summate all local volumes to obtain the total volume of the pit.

[0070] 8) The pit mask obtained in step 2) is superimposed and fused onto the initially acquired two-dimensional grayscale image, and different colors are used to represent pits at different locations to visualize the pit locations on the initially acquired two-dimensional grayscale image; at the same time, combined with the corresponding parameters obtained from the point cloud analysis of each pit, a final output image containing the identifiers and parameters of all pits is generated.

[0071] Example 2:

[0072] This embodiment is a further optimization based on the above embodiment. The parts that are the same as those in the aforementioned technical solutions will not be repeated here. In order to better realize the container surface pit quantification detection method based on vision and point cloud fusion of the present invention, the following setting is adopted: Step 1) The two-dimensional grayscale image of the container to be detected and the three-dimensional point cloud data corresponding to the two-dimensional grayscale image are acquired simultaneously by a binocular structured light camera (public method).

[0073] Example 3:

[0074] This embodiment is a further optimization based on any of the above embodiments. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following setting is specifically adopted: Step 2) includes the following specific steps:

[0075] 2.1) The two-dimensional grayscale image obtained in step 1) is preprocessed by median filtering and contrast adjustment, and then input into the FastSAM (Fast Segment Anything Model) model. The FastSAM model identifies all potential pit regions in the preprocessed two-dimensional grayscale image and generates a fine pit mask (binarization mask) for each identified pit region. This mask accurately outlines the pixel-level contour of the pit in the image.

[0076] 2.2) After obtaining the pit mask, the pit mask is further filtered by area screening. During the filtering, a lower limit of the area threshold is set, and then the pixel area of ​​each pit mask is calculated and compared with the lower limit of the area threshold. Masks that do not meet the requirements are removed, and mask areas that meet the area requirements and represent real pits are retained.

[0077] Example 4:

[0078] This embodiment is a further optimization based on any of the above embodiments. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following setting is specifically adopted: Step 6) includes the following specific steps:

[0079] 6.1) Based on the unit normal vector of the reference plane, select a standard coordinate axis direction in the global coordinate system where the point cloud data is located as an auxiliary vector, and the auxiliary vector is not collinear with the unit normal vector; for example, use the X-axis (1,0,0) or the Y-axis (0,1,0) as the auxiliary direction; the X-axis (1,0,0) is preferred as the auxiliary direction; when the unit normal vector of the reference plane is close to the X-axis (the absolute value of the X component is ≥0.9), the Y-axis (0,1,0) is used instead.

[0080] 6.2) Calculate two unit vectors orthogonal to the unit normal vector and to each other by cross product using the unit normal vector and auxiliary vector of the reference plane; construct a local orthogonal coordinate system using the calculated unit vectors, and then transform the point cloud inside the pit into the local orthogonal coordinate system while preserving the two-dimensional coordinate position;

[0081] 6.3) For the point cloud inside the pit transformed into a local orthogonal coordinate system, calculate the mean coordinate of all points in this local orthogonal coordinate system, and use this mean to perform a global translation of the point cloud inside the pit to center it. Then, perform singular value decomposition on the centered point cloud data inside the pit to extract the two orthogonal principal component directions on the local plane. Project the centered point cloud inside the pit onto the principal axis coordinate system formed by the two orthogonal principal component directions, and calculate the maximum and minimum coordinate values ​​along each principal axis. Use the difference between the two as the span of the corresponding direction. The maximum span is defined as the pit length, and the minimum span is defined as the pit width, thereby obtaining the size parameters of the pit region.

[0082] Example 5:

[0083] This embodiment is a further optimization based on any of the above embodiments. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the container surface pit quantification detection method based on vision and point cloud fusion described in this invention, the following setting is specifically adopted: Step 7) includes the following specific steps:

[0084] 7.1) Construct a new local orthogonal coordinate system based on the method in step 6), and project the point cloud inside the pit into the new local orthogonal coordinate system to obtain the corresponding two-dimensional projection point set;

[0085] 7.2) After step 7.1), a Delaunay triangulation is constructed based on the two-dimensional projection point set in the new local orthogonal coordinate system (a public method that can generate a mesh with a better shape and avoid narrow triangles). A set of non-overlapping triangular patches are generated to form a triangular mesh covering the distribution range of the two-dimensional projection point set, which is used to support the subsequent calculation of the pit volume.

[0086] 7.3) Based on the triangular mesh, for a single triangular facet, first calculate its geometric area on the two-dimensional plane; simultaneously, based on the height values ​​of the three vertices of the triangular facet relative to the reference plane, take the arithmetic mean of the three height values ​​as the overall height of the triangular facet, and apply a non-negative constraint to the overall height (i.e., if the average value is negative or zero, it is considered as having no depression); finally, multiply the area of ​​the triangular facet by the overall height of the triangular facet to obtain the volume corresponding to the triangular facet.

[0087] 7.4) Traverse all triangular faces generated by Delaunay triangulation, sum the volumes calculated for each face, and thus obtain the total volume of the pit.

[0088] Example 6:

[0089] This embodiment is a further optimization based on any of the above embodiments. The parts that are the same as those in the foregoing technical solutions will not be repeated here. In order to better realize the container surface pit quantification detection method based on vision and point cloud fusion of the present invention, the following settings are adopted: the corresponding parameters of each pit include pit length, pit width, total pit volume, maximum pit depth and pixel area of ​​each pit mask.

[0090] Example 7:

[0091] A method for quantitative detection of surface pits on containers based on vision and point cloud fusion, such as Figure 1 As shown, the specific steps include the following:

[0092] S1. Binocular Camera, Data Preprocessing: A binocular structured light camera synchronously acquires a 2D grayscale image of the container to be detected and its corresponding 3D point cloud data, constructing image-point cloud data pairs for subsequent collaborative processing, such as... Figure 2 , Figure 3 As shown;

[0093] S2, FastSAM pit segmentation:

[0094] S2.1 The two-dimensional grayscale image obtained in step 1) is preprocessed by median filtering and contrast adjustment, and then input into the FastSAM (Fast Segment Anything Model). The FastSAM (Fast Segment Anything Model) model identifies all potential pit regions in the preprocessed two-dimensional grayscale image and generates a fine pit mask (binarization mask) for each identified pit region. This mask accurately outlines the pixel-level contour of the pit in the image.

[0095] S2.2 After obtaining the pit mask, the pit mask is further filtered by area screening. During filtering, a lower limit for the area threshold is set, and then the pixel area of ​​each pit mask is calculated and compared with the lower limit for the area threshold. Masks that do not meet the requirements are removed, and mask areas that meet the area requirements and represent real pits are retained, such as... Figure 4 As shown.

[0096] S3. Ring Strip Extraction: For all the acquired pit masks, the corresponding ring strip masks are obtained through the following operations: When operating on each pit mask, firstly, a morphological edge expansion operation is performed on the pit mask to generate an expanded region mask covering the original mask and the surrounding local background; then, by calculating the difference between this expanded region mask and the original mask, a ring strip mask surrounding the edge of the original pit is obtained, such as... Figure 5 As shown.

[0097] S4. Reference Plane Fitting: The acquired 3D point cloud data is filtered for outliers based on coordinate thresholds, and then the ring point cloud data corresponding to the ring mask is extracted. Subsequently, the RANSAC (Random Sample Consensus) algorithm is used to perform robust plane fitting on the ring point cloud data to obtain the reference plane equation and the unit normal vector of the reference plane representing the container surface in the pit-free state in this region. The reference plane represents the surface of the container when the original surface is pit-free.

[0098] S5. Height Calculation: Based on the 3D point cloud data after anomaly filtering, the corresponding internal point cloud data of the pit is extracted by combining the obtained pit mask. Then, the height of each internal point of the pit relative to the reference plane is calculated by combining the dot product of the unit normal vector of the reference plane and the distance parameter. The maximum value is taken as the maximum depth of the pit.

[0099] S6, Local Coordinate System:

[0100] S6.1. Based on the unit normal vector of the reference plane, select a standard coordinate axis direction in the global coordinate system where the point cloud data is located as an auxiliary vector, and this auxiliary vector is not collinear with the unit normal vector; for example, use the X-axis (1,0,0) or the Y-axis (0,1,0) as the auxiliary direction; the X-axis (1,0,0) is preferred as the auxiliary direction; when the unit normal vector of the reference plane is close to the X-axis (the absolute value of the X component is ≥0.9), the Y-axis (0,1,0) is used instead.

[0101] S6.2. Calculate two unit vectors orthogonal to the unit normal vector and to each other by cross product using the unit normal vector and auxiliary vector of the reference plane; construct a local orthogonal coordinate system using the calculated unit vectors, and then transform the point cloud inside the pit into the local orthogonal coordinate system while preserving the two-dimensional coordinate position.

[0102] S7. Point Cloud Principal Vector Analysis: For the point cloud inside the pit transformed into a local orthogonal coordinate system, calculate the mean coordinate of all points in this local orthogonal coordinate system, and use this mean to perform a global translation of the point cloud inside the pit to center it. Then, perform singular value decomposition on the centered point cloud data inside the pit to extract the two orthogonal principal component directions on the local plane. Project the centered point cloud inside the pit onto the principal axis coordinate system formed by the two orthogonal principal component directions, and calculate the maximum and minimum coordinate values ​​along each principal axis. Use the difference between the two as the span of the corresponding direction. The maximum span is defined as the pit length, and the minimum span is defined as the pit width, thereby obtaining the size parameters of the pit region.

[0103] S8, Triangular Network Volume Aggregation:

[0104] S8.1. Based on the method in step S6, construct a new local orthogonal coordinate system and project the point cloud inside the pit into the new local orthogonal coordinate system to obtain the corresponding two-dimensional projection point set;

[0105] S8.2 After step S8.1, a Delaunay triangulation is constructed based on the two-dimensional projection point set in the new local orthogonal coordinate system (a public method that can generate a mesh with better shape and avoid narrow triangles). A set of non-overlapping triangular patches are generated to form a triangular mesh covering the distribution range of the two-dimensional projection point set, which is used to support the subsequent calculation of the pit volume.

[0106] S8.3. Based on the triangular mesh, for a single triangular facet, first calculate its geometric area on the two-dimensional plane; simultaneously, based on the height values ​​of the three vertices of the triangular facet relative to the reference plane, take the arithmetic mean of the three height values ​​as the overall height of the triangular facet, and apply a non-negative constraint to the overall height (i.e., if the average value is negative or zero, it is considered as having no depression); finally, multiply the area of ​​the triangular facet by the overall height of the triangular facet to obtain the volume corresponding to the triangular facet.

[0107] S8.4. Traverse all triangular faces generated by Delaunay triangulation, sum the calculated volumes of each face to obtain the total volume of the pit.

[0108] S9. Result Integration: The pit mask obtained in step S2 is superimposed and fused onto the initially acquired two-dimensional grayscale image, and different colors are used to represent pits at different locations, so as to realize the visualization of pit locations on the initially acquired two-dimensional grayscale image; at the same time, combined with the corresponding parameters obtained from the point cloud analysis of each pit, a final result image containing the labels and parameters of all pits is output.

[0109] In terms of pit detection, this invention utilizes effective pit detection and pit mask filtering in 2D images to improve the accuracy and efficiency of subsequent processing.

[0110] In terms of reference plane construction, the key point of this invention is to perform extended calculations on the mask obtained from pit detection to obtain the annular mask related to the original mask, and at the same time combine 3D point cloud data to construct the annular reference plane, which serves as the basis for the geometric feature analysis of pit point cloud.

[0111] This invention employs point cloud principal vector analysis technology and overcomes the singularity problem of the reference plane orientation by adaptively switching auxiliary vectors, ensuring the numerical stability of the local coordinate system construction, thereby achieving accurate and robust extraction of the essential length and width dimensions of the pit.

[0112] This invention designs a triangular mesh volume aggregation: by discretizing the surface of the pit into a series of triangular prism micro-elements with independent vertex heights and performing integration, it breaks through the accuracy limitation of traditional methods that use average height, and realizes high-precision and refined calculation of the volume of irregular pits.

[0113] This invention integrates 2D image detection results and 3D point cloud geometric feature analysis results for output, ensuring that the output information is accurate and complete.

[0114] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for quantitative detection of surface pits on containers based on vision and point cloud fusion, characterized in that: The specific steps include the following: 1) Obtain the two-dimensional grayscale image of the container to be detected and the corresponding three-dimensional point cloud data, and construct an image-point cloud data pair; 2) Input the preprocessed 2D grayscale image into the FastSAM model to perform instance-level target segmentation, obtain all pit masks, and filter out the mask regions that meet the area requirements and represent real pits. 3) Obtain the corresponding ring mask by performing the following operations on all the pit masks; when operating on each pit mask, first perform a morphological edge expansion operation on the pit mask to generate an expanded region mask that covers the original mask and the surrounding local background. Subsequently, by calculating the difference between the extended region mask and the original mask, a ring mask surrounding the edge of the original pit is obtained. 4) The acquired 3D point cloud data is filtered for outliers based on coordinate thresholds, and then the annular point cloud data corresponding to the annular mask is extracted from it. Subsequently, the RANSAC algorithm was used to perform robust plane fitting on the ring point cloud data to obtain the reference plane equation and the unit normal vector of the reference plane characterizing the container surface in the region without pits. 5) Based on the 3D point cloud data after anomaly filtering, the corresponding pit interior point cloud data is extracted by combining the obtained pit mask. Then, the height of each pit interior point relative to the reference plane is calculated by combining the dot product of the unit normal vector of the reference plane and the distance parameter. The maximum value is taken as the maximum depth of the pit. 6) Calculate the unit vector using the unit normal vector of the reference plane that is not parallel to the unit normal vector, thereby establishing a local orthogonal coordinate system with the reference plane as the reference; complete the principal vector analysis of the point cloud inside the pit through the local orthogonal coordinate system, and project the point cloud data inside the pit onto the principal direction coordinate axis to calculate the length and width of the pit. 7) Construct Delaunay triangulation based on the set of two-dimensional projection points of the pit on the new local orthogonal coordinate system. Multiply the area of ​​each triangular facet by the average height of the three vertices to obtain the corresponding local volume. Summate all local volumes to obtain the total volume of the pit. 8) The pit mask obtained in step 2) is superimposed and fused onto the initially acquired two-dimensional grayscale image, and different colors are used to represent pits at different locations to visualize the pit locations on the initially acquired two-dimensional grayscale image; at the same time, combined with the corresponding parameters obtained from the point cloud analysis of each pit, a final output image containing the identifiers and parameters of all pits is generated.

2. The method for quantitative detection of surface pits on containers based on vision and point cloud fusion according to claim 1, characterized in that: Step 1) synchronously acquires a two-dimensional grayscale image of the container to be detected and the corresponding three-dimensional point cloud data using a binocular structured light camera.

3. The method for quantitative detection of surface pits on containers based on vision and point cloud fusion according to claim 1, characterized in that: Step 2) includes the following specific steps: 2.1) The two-dimensional grayscale image obtained in step 1) is preprocessed by median filtering and contrast adjustment, and then input into the FastSAM model. The FastSAM model identifies all potential pit regions in the preprocessed two-dimensional grayscale image and generates a fine pit mask for each identified pit region. 2.2) After obtaining the pit mask, the pit mask is further filtered by area screening. During the filtering, a lower limit of the area threshold is set, and then the pixel area of ​​each pit mask is calculated and compared with the lower limit of the area threshold. Masks that do not meet the requirements are removed, and mask areas that meet the area requirements and represent real pits are retained.

4. The method for quantitative detection of surface pits on containers based on vision and point cloud fusion according to claim 1, characterized in that: Step 6) includes the following specific steps: 6.1) Based on the unit normal vector of the reference plane, select a standard coordinate axis direction in the global coordinate system where the point cloud data is located as an auxiliary vector, and this auxiliary vector is not collinear with the unit normal vector; 6.2) Calculate two unit vectors orthogonal to the unit normal vector and to each other by cross product using the unit normal vector and auxiliary vector of the reference plane; construct a local orthogonal coordinate system using the calculated unit vectors, and then transform the point cloud inside the pit into the local orthogonal coordinate system while preserving the two-dimensional coordinate position; 6.3) For the point cloud inside the pit transformed into a local orthogonal coordinate system, calculate the mean coordinate of all points in this local orthogonal coordinate system, and use this mean to perform a global translation of the point cloud inside the pit to center it. Then, perform singular value decomposition on the centered point cloud data inside the pit to extract the two orthogonal principal component directions on the local plane. Project the centered point cloud inside the pit onto the principal axis coordinate system formed by the two orthogonal principal component directions, and calculate the maximum and minimum coordinate values ​​along each principal axis. Use the difference between the two as the span of the corresponding direction. The maximum span is defined as the pit length, and the minimum span is defined as the pit width, thereby obtaining the size parameters of the pit region.

5. The method for quantitative detection of surface pits on containers based on vision and point cloud fusion according to claim 1, characterized in that: Step 7) includes the following specific steps: 7.1) Construct a new local orthogonal coordinate system based on the method in step 6), and project the point cloud inside the pit into the new local orthogonal coordinate system to obtain the corresponding two-dimensional projection point set; 7.2) After step 7.1), Delaunay triangulation is constructed based on the two-dimensional projection point set in the new local orthogonal coordinate system, generating a set of non-overlapping triangular patches to form a triangular mesh covering the distribution range of the two-dimensional projection point set. 7.3) Based on the triangular mesh, for a single triangular facet, first calculate the geometric area on the two-dimensional plane; at the same time, according to the height values ​​of the three vertices of the triangular facet relative to the reference plane, take the arithmetic mean of the three height values ​​as the overall height of the triangular facet, and apply non-negative constraints to the overall height; finally, multiply the area of ​​the triangular facet by the overall height of the triangular facet to obtain the volume corresponding to the triangular facet. 7.4) Traverse all triangular faces generated by Delaunay triangulation, sum the volumes calculated for each face, and thus obtain the total volume of the pit.

6. The method for quantitative detection of surface pits on containers based on vision and point cloud fusion according to claim 1, characterized in that: The parameters for each pit include pit length, pit width, total pit volume, maximum pit depth, and pixel area of ​​each pit mask.

Images