Glass profile extraction method, device, equipment and computer readable storage medium

Abstract

The application discloses a glass contour extraction method, device and equipment and a computer readable storage medium, and belongs to the technical field of vehicle manufacturing and detection. The method comprises the following steps: acquiring position information of each reference point in the glass whose contour is to be extracted; acquiring a first tangent vector and a second tangent vector of each reference point according to the position information of the reference points; projecting the neighborhood points of each reference point to the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point; determining the maximum angle difference of each reference point according to the polar angle of the projection points corresponding to the neighborhood points of each reference point; determining the edge point of the glass in the reference points according to the maximum angle difference of each reference point and the number of the neighborhood points of each reference point; and extracting the contour of the glass according to the edge point of the glass. The contour of the glass extracted by the method is more accurate.

Description

Technical Field

[0001] This application relates to the field of vehicle manufacturing and testing technology, and in particular to a method, apparatus, device and computer-readable storage medium for extracting glass contours. Background Technology

[0002] In the vehicle manufacturing process, glass is a key component of the vehicle body and interior space. The size, cut shape and edge quality of the glass have a significant impact on assembly accuracy and sealing performance. In order to ensure that the glass meets the design requirements, it is necessary to extract the glass contour.

[0003] In related technologies, images of glass are captured by a camera, and edge recognition is performed on the images to obtain the outline of the glass.

[0004] However, due to the high transparency and strong reflectivity of glass, the acquired images are easily affected by lighting conditions, resulting in blurred images and poor accuracy of the extracted glass contours, which in turn affects the sealing performance of the glass assembly. Summary of the Invention

[0005] This application provides a method, apparatus, device, and computer-readable storage medium for extracting glass contours, which can be used to solve the problem that the accuracy of the glass contours extracted in related technologies is poor, thus affecting the sealing performance of glass assembly. The technical solution is as follows.

[0006] On one hand, embodiments of this application provide a method for extracting glass contours, the method comprising: Obtain the position information of each reference point in the glass containing the contour to be extracted; Based on the position information of each reference point, obtain the first tangent vector and the second tangent vector of each reference point, wherein the first tangent vector and the second tangent vector are perpendicular; The neighboring points of each reference point are projected onto the tangent plane of each reference point to obtain the projection points corresponding to the neighboring points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point. The maximum angle difference between each reference point is determined based on the polar angle of the projection point corresponding to the neighboring points of each reference point. Based on the maximum angular difference between each reference point and the number of neighboring points of each reference point, the edge point of the glass is determined among each reference point; The outline of the glass is extracted based on the edge points of the glass.

[0007] In one possible implementation, before obtaining the position information of each reference point in the glass whose contour to be extracted, the method further includes: Acquire three-dimensional point cloud data of the glass, wherein the three-dimensional point cloud data includes the position information of each point in the glass; Based on the location information of each point, the reference point is determined among each point.

[0008] In one possible implementation, determining the reference point among the points based on the location information of each point includes: Based on the location information of each point, a voxel downsampling operation is performed on each point to obtain a set of voxel units after downsampling. The set of voxel units includes multiple voxels, and each voxel includes at least one point. Based on the points included in each voxel, determine the first point corresponding to each voxel; Determine the neighborhood points of each first point; The first point is filtered based on the number of its neighboring points, and the reference point is determined based on the filtering results.

[0009] In one possible implementation, determining the neighborhood points of each first point includes: For any one of the aforementioned first points, a first region is determined based on the location information of that first point; a second point located within the first region is taken as a neighboring point of that first point, where the second point is any point among the aforementioned points other than the first point; or... Based on the location information of any first point and the location information of the second point, the distance between any first point and the second point is determined; if the distance between any first point and the second point is less than a distance threshold, the second point is taken as a neighboring point of any first point.

[0010] In one possible implementation, obtaining the first tangent vector and the second tangent vector of each reference point based on the position information of each reference point includes: For any one of the reference points, the normal vector of any one reference point is estimated based on its position information to obtain the normal vector of any one reference point. The normal vector of any one reference point is perpendicular to the first tangent vector and the second tangent vector of any one reference point. Based on the normal vector of any reference point, obtain the first tangent vector and the second tangent vector of any reference point.

[0011] In one possible implementation, the step of estimating the normal vector of any reference point based on its position information to obtain the normal vector of that reference point includes: Based on the location information of any reference point, determine the neighboring points of any reference point; The location information of the target point is determined based on the location information of the neighboring points of any reference point. Based on the position information of the neighboring points of any reference point and the position information of the target point, determine the deviation vector of the neighboring points of any reference point relative to the target point; Construct the covariance matrix of any reference point based on the deviation vector of its neighboring points relative to the target point; The covariance matrix of any reference point is decomposed into eigenvalues ​​to obtain the normal vector of any reference point.

[0012] In one possible implementation, before determining the maximum angular difference between the reference points based on the polar angles of the projection points corresponding to the neighboring points of each reference point, the method further includes: For any neighboring point of any of the reference points, the polar angle of the projection point corresponding to any neighboring point of any of the reference points is determined based on the first tangent vector, the second tangent vector of any reference point, and the position information of any neighboring point.

[0013] In one possible implementation, determining the maximum angular difference between the reference points based on the polar angles of the projection points corresponding to the neighboring points of each reference point includes: For any one of the reference points, sort the polar angles of the projection points corresponding to the neighboring points of any one reference point to obtain a polar angle sequence; Determine the absolute value of the difference between two adjacent polar angles in the polar angle sequence, and calculate the circumference interval between the first and last polar angles in the polar angle sequence; The absolute value of the difference between two adjacent polar angles in the polar angle sequence and the maximum value in the surrounding angle interval are taken as the maximum angle difference of any reference point.

[0014] In one possible implementation, determining the edge point of the glass among the reference points based on the maximum angular difference between the reference points and the number of neighboring points of each reference point includes: The reference point whose maximum angle difference is not less than the angle difference threshold and whose number of neighboring points is not less than the second quantity threshold is taken as the edge point of the glass.

[0015] In one possible implementation, extracting the contour of the glass based on its edge points includes: The edge points of the glass are connected according to their spatial proximity to obtain a spatial reference graphic; The outline of the glass is determined based on the spatial reference drawing.

[0016] In one possible implementation, determining the outline of the glass based on the spatial reference drawing includes: Use the spatial reference graphic as the outline of the glass; or, The number of edge points contained in each polyline in the spatial reference graphic is counted, and polylines with an edge point count not exceeding a threshold are removed to obtain the target graphic, which is then used as the outline of the glass.

[0017] On the other hand, embodiments of this application provide a glass contour extraction device, the device comprising: The acquisition module is used to acquire the position information of each reference point in the glass whose contour is to be extracted; The acquisition module is further configured to acquire a first tangent vector and a second tangent vector of each reference point based on the position information of each reference point, wherein the first tangent vector and the second tangent vector are perpendicular; The projection module is used to project the neighborhood points of each reference point onto the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point. The determination module is used to determine the maximum angle difference between the reference points based on the polar angle of the projection points corresponding to the neighboring points of each reference point. The determining module is further configured to determine the edge point of the glass among the reference points based on the maximum angle difference between the reference points and the number of neighboring points of the reference points. The extraction module is used to extract the outline of the glass based on the edge points of the glass.

[0018] In one possible implementation, the acquisition module is further configured to acquire three-dimensional point cloud data of the glass, the three-dimensional point cloud data including the position information of each point in the glass; The determining module is further configured to determine the reference point among the points based on the location information of each point.

[0019] In one possible implementation, the determining module is configured to perform voxel downsampling on each point based on the position information of each point to obtain a set of downsampled voxel units, wherein the set of voxel units includes multiple voxels, and each voxel includes at least one point; determine a first point corresponding to each voxel based on the points included in each voxel; determine the neighboring points of each first point; filter the first point based on the number of neighboring points of the first point; and determine the reference point based on the filtering results.

[0020] In one possible implementation, the determining module is configured to, for any one of the first points, determine a first region based on the location information of the first point; and designate a second point located in the first region as a neighboring point of the first point, wherein the second point is a point other than the first point among the points; or, determine the distance between the first point and the second point based on the location information of the first point and the location information of the second point; and, if the distance between the first point and the second point is less than a distance threshold, designate the second point as a neighboring point of the first point.

[0021] In one possible implementation, the acquisition module is configured to, for any one of the reference points, perform normal estimation on the reference point based on the position information of the reference point to obtain the normal vector of the reference point, wherein the normal vector of the reference point is perpendicular to the first tangent vector and the second tangent vector of the reference point; and acquire the first tangent vector and the second tangent vector of the reference point based on the normal vector of the reference point.

[0022] In one possible implementation, the acquisition module is configured to: determine the neighboring points of the arbitrary reference point based on the location information of the arbitrary reference point; determine the location information of the target point based on the location information of the neighboring points of the arbitrary reference point; determine the deviation vector of the neighboring points of the arbitrary reference point relative to the target point based on the location information of the neighboring points of the arbitrary reference point and the location information of the target point; construct the covariance matrix of the arbitrary reference point based on the deviation vector of the neighboring points of the arbitrary reference point relative to the target point; and perform eigenvalue decomposition on the covariance matrix of the arbitrary reference point to obtain the normal vector of the arbitrary reference point.

[0023] In one possible implementation, the determining module is further configured to, for any neighboring point of any of the reference points, determine the polar angle of the projection point corresponding to any neighboring point of any of the reference points, based on the first tangent vector, the second tangent vector of any of the reference points, and the position information of any neighboring point.

[0024] In one possible implementation, the determining module is configured to, for any one of the reference points, sort the polar angles of the projection points corresponding to the neighboring points of the reference point to obtain a polar angle sequence; determine the absolute value of the difference between two adjacent polar angles in the polar angle sequence, and calculate the circumferential angle interval between the first and last polar angles in the polar angle sequence; and take the maximum value of the absolute value of the difference between two adjacent polar angles in the polar angle sequence and the circumferential angle interval as the maximum angle difference of the reference point.

[0025] In one possible implementation, the determining module is used to select the reference points among the reference points whose maximum angle difference is not less than an angle difference threshold and whose number of neighboring points is not less than a second quantity threshold as the edge points of the glass.

[0026] In one possible implementation, the extraction module is used to connect the edge points of the glass according to spatial proximity to obtain a spatial reference graphic; and to determine the outline of the glass based on the spatial reference graphic.

[0027] In one possible implementation, the extraction module is used to use the spatial reference graphic as the outline of the glass; or, it counts the number of edge points contained in each polyline in the spatial reference graphic, removes polylines with an edge point count not greater than a threshold, obtains a target graphic, and uses the target graphic as the outline of the glass.

[0028] On the other hand, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores at least one piece of program code, which is loaded and executed by the processor to enable the electronic device to implement any of the glass contour extraction methods described above.

[0029] On the other hand, a computer-readable storage medium is also provided, wherein at least one piece of program code is stored therein, the at least one piece of program code being loaded and executed by a processor to enable an electronic device to implement any of the glass contour extraction methods described above.

[0030] On the other hand, a computer program or computer program product is also provided, wherein the computer program or computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for extracting glass contours.

[0031] The technical solution provided in this application has at least the following beneficial effects: The technical solution provided in this application constructs tangent planes for each reference point by obtaining two mutually perpendicular tangent vectors for each reference point. By projecting neighboring points of each reference point onto these tangent planes, interference from irrelevant directions in three-dimensional space can be effectively eliminated, focusing on the feature distribution within the local plane. Simultaneously, by combining the polar angles of the projection points of neighboring points of each reference point, the maximum angular difference between each reference point is determined, accurately capturing the difference between edge and non-edge points of the glass. The increased number of neighboring points further filters out noise and outliers, reducing the risk of misidentifying points as edges, thus achieving precise positioning of glass edge points and extracting a more reliable and accurate glass contour. Furthermore, this method has good adaptability to glass with different shapes and surface conditions, improving the versatility and accuracy of glass contour extraction. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the implementation environment of a glass contour extraction method provided in an embodiment of this application; Figure 2 This is a flowchart of a method for extracting glass contours provided in an embodiment of this application; Figure 3 This is a schematic diagram of three-dimensional point cloud data of glass provided in an embodiment of this application; Figure 4 This is a schematic diagram of the outline of a glass provided in an embodiment of this application; Figure 5 This is a flowchart of a method for extracting glass contours provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a glass contour extraction device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0036] Figure 1 This is a schematic diagram illustrating the implementation environment of a glass contour extraction method provided in this application embodiment, such as... Figure 1 As shown, the implementation environment includes an electronic device 101, which can be a terminal device or a server; this application embodiment does not limit this. The electronic device 101 is used to execute the glass contour extraction method provided in this application embodiment.

[0037] Optionally, electronic device 101 is a terminal device. A terminal device can be any electronic device product that can interact with a user through one or more methods such as a keyboard, touchpad, remote control, voice interaction, or handwriting device. Examples include PCs (Personal Computers), mobile phones, smartphones, PDAs (Personal Digital Assistants), wearable devices, PPCs (Pocket PCs), tablet computers, and smart car systems.

[0038] A terminal device can refer to one of multiple terminal devices; this embodiment uses only one terminal device as an example. Those skilled in the art will understand that the number of terminal devices can be more or less. For example, there may be only one terminal device, or there may be dozens or hundreds, or even more. This application embodiment does not limit the number or type of terminal devices.

[0039] Optionally, the electronic device 101 is a server, which may be a single server, a server cluster consisting of multiple servers, or any of the following: a cloud computing platform and a virtualization center. This embodiment of the application does not limit this. The server and the terminal device communicate directly or indirectly via wired or wireless communication. The server has data receiving, data processing, and data sending functions. Of course, the server may also have other functions, which this embodiment of the application does not limit.

[0040] Those skilled in the art should understand that the above-described terminal devices and servers are merely illustrative examples. Other existing or future terminal devices and servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0041] This application provides a method for extracting glass contours, which can be applied to the above-mentioned... Figure 1 The implementation environment shown is as follows: Figure 2 The flowchart shown in this embodiment of the present application illustrates a method for extracting glass contours. This method can be implemented by... Figure 1 The electronic device 101 in the system performs the operation. For example... Figure 2 As shown, the method includes the following steps 201 to 206.

[0042] In step 201, the position information of each reference point in the glass whose contour is to be extracted is obtained.

[0043] In the exemplary embodiments of this application, the glass can be the triangular window glass of a vehicle, the windshield of a vehicle, the rear windshield of a vehicle, the sunroof of a vehicle, the window glass of a vehicle, or other movable or immovable glass in a vehicle. The embodiments of this application do not limit the scope of the glass.

[0044] In one possible implementation, before obtaining the position information of each reference point in the glass whose contour to be extracted, it is necessary to first determine each reference point. This application does not limit the process of determining each reference point.

[0045] For example, the process of determining each reference point includes: acquiring three-dimensional point cloud data of the glass, which includes the positional information of each point in the glass; determining reference points among each point based on the positional information of each point. Determining reference points among each point based on the positional information of each point includes, but is not limited to, removing outliers (NaN / Inf) from each point. The point cloud is then lightly truncated to suppress the impact of a small number of outliers on subsequent processing. For example, the 1% quantile and 99% quantile are calculated for the point cloud in three coordinate dimensions to obtain the span, and upper and lower limits are constructed accordingly. Then, the coordinates of each point are truncated dimension-by-dimensionally without changing the number of points. This process does not change the number of points but significantly reduces the interference of extreme outliers on the bounding box scale, voxel downsampling parameters, and normal estimation stability. Afterwards, voxel downsampling is performed, for example, using bisection downsampling.

[0046] Among them, the position information of each point in the 3D point cloud data of the glass is used to determine the reference point, which can ensure that the reference point is accurately anchored to the actual spatial position of the glass. This provides a unified and reliable benchmark for the subsequent edge extraction of the glass, effectively avoids the detection error caused by benchmark deviation, lays the foundation for the subsequent processing, and improves the reliability and accuracy of edge extraction.

[0047] The process of acquiring 3D point cloud data of glass includes, but is not limited to: acquiring 3D point cloud data of glass using a laser scanner or structured light equipment. For example... Figure 3 This is a schematic diagram of three-dimensional point cloud data of glass provided in an embodiment of this application.

[0048] Optionally, the process of determining a reference point among the points based on their location information includes: performing voxel downsampling on each point based on its location information to obtain a set of downsampled voxel units, wherein the set of voxel units includes multiple voxels, and each voxel includes at least one point; determining the first point corresponding to each voxel based on the points included in each voxel; determining the neighboring points of each first point; filtering the first points based on the number of neighboring points of the first point, and determining the reference point based on the filtering results.

[0049] The first point is filtered based on the number of its neighboring points, and a reference point is determined based on the filtering results. This includes, but is not limited to: if the number of neighboring points of the first point is greater than a first threshold, the first point is used as a reference point; or, if the number of neighboring points of the first point is not greater than the first threshold, and the first point is located within a reference region, the first point is used as a reference point. The reference region is a region with significant geometric feature changes. For example, curvature is used to determine regions with significant geometric feature changes. For instance, curvature is used to determine regions with significant geometric feature changes, and regions where the geometric feature changes exceed a threshold are considered regions with significant changes and are used as reference regions. Alternatively, if the number of neighboring points of the first point is not greater than the first threshold and satisfies preset geometric features, the first point is used as a reference point.

[0050] Before performing voxel downsampling on each point, the process includes: adaptively determining voxel downsampling parameters based on the spatial range of the glass's 3D point cloud data and the preset number of target points. Correspondingly, performing voxel downsampling on each point includes: performing voxel downsampling on each point according to the determined voxel downsampling parameters. For example, the determined voxel downsampling parameters include the voxel size.

[0051] In related technologies, voxel downsampling typically involves manually specifying a voxel size. However, the number of points in a glass point cloud can vary greatly depending on the context, ranging from hundreds of thousands (e.g., from scanned point cloud data) to millions (e.g., from a point cloud template generated by CAD). If the voxel size is fixed directly, parameter adjustments are required when switching to different types of point clouds. This application utilizes methods from related technologies to determine voxel downsampling parameters, but requires adjusting the matching parameters for the point cloud; that is, adaptively adjusting the fixed voxel size based on the point cloud.

[0052] Optionally, this application employs a binary downsampling method. For example, before downsampling, a target point range is set, such as 150,000 to 250,000 points. This target point range includes, but is not limited to, being determined based on device performance and the complexity of subsequent algorithms. Then, based on the bounding box size of the glass point cloud, an initial voxel size is estimated, such that, ideally, each voxel retains approximately only one representative point. The voxel size is then adjusted iteratively. For example, if the number of points after downsampling at the current voxel size is greater than the target point range, it indicates the voxel is too small, and the voxel size is increased. If the number of points after downsampling is less than the target point range, it indicates the voxel is too large, and the voxel size is decreased. This embodiment of the application gradually approximates the target point range in this manner until the number of points after downsampling falls within the preset range.

[0053] By using voxel downsampling to select a representative first point from the various points included in the 3D point cloud data of the glass, and then determining a reference point based on the number of neighboring points of the first point, the computational workload in determining the reference point is significantly reduced. A threshold for the number of neighboring points is used to select the first point with a dense distribution of surrounding points and strong stability as the reference point. Furthermore, by also including first points in areas where the number of neighboring points does not meet the quantity requirement but which are located in regions with significant changes in the geometric features of the glass as reference points, the determination of the reference point is more comprehensive, resulting in a more complete set of reference points.

[0054] In this process, voxel downsampling is performed on each point, and the number of voxels included in the resulting set of downsampled voxel units is the target number. This target number is set based on experience or can be flexibly adjusted according to the implementation environment; this embodiment does not limit this. A first quantity threshold is also set based on experience or can be flexibly adjusted according to the implementation environment; this embodiment does not limit this either. For example, the first quantity threshold can be set according to point cloud density or scanning accuracy, such as 30.

[0055] Optionally, if the number of neighboring points of the first point is not greater than a first quantity threshold and the first point is not located in the reference region, the first point is not used as a reference point.

[0056] Optionally, the process of determining the first point corresponding to each voxel based on the points included in each voxel includes: for any voxel in each voxel, taking any point among the points included in any voxel as the first point corresponding to any voxel; or, taking the center point among the points included in any voxel as the first point corresponding to any voxel; or, determining first average position information based on the position information of the points included in any voxel, and taking the point corresponding to the first average position information as the first point corresponding to any voxel.

[0057] In one possible implementation, the process of determining the neighborhood points of each first point includes: for any first point among the first points, determining a first region based on the location information of any first point; taking a second point located in the first region as a neighborhood point of any first point, wherein the second point is a point other than the first point among the points; or, determining the distance between any first point and the second point based on the location information of any first point and the location information of the second point; if the distance between any first point and the second point is less than a distance threshold, taking the second point as a neighborhood point of any first point.

[0058] In this implementation, the neighborhood points of each first point are determined in two ways, making the determination of the neighborhood points of each first point more flexible and convenient.

[0059] Optionally, the process of determining the first region based on the location information of any first point includes: determining a target spatial neighborhood region with the location indicated by the location information of any first point as the center and the target length as the radius; and taking the area covered by the target spatial neighborhood region as the first region. The target spatial neighborhood region can be a three-dimensional spherical region.

[0060] The target length is set based on experience, or can be flexibly adjusted according to the implementation environment; this application embodiment does not limit this. For example, the target length is 0.05 meters.

[0061] Optionally, the process of determining the distance between any first point and the second point based on the location information of any first point and the location information of the second point includes: taking the Euclidean distance between any first point and the second point as the distance between any first point and the second point.

[0062] In one possible implementation, after determining each reference point, the location information of each reference point is obtained.

[0063] In step 202, based on the position information of each reference point, the first tangent vector and the second tangent vector of each reference point are obtained, and the first tangent vector and the second tangent vector are perpendicular.

[0064] In one possible implementation, the normal vector is estimated based on the position information of each reference point, and the first tangent vector and the second tangent vector of each reference point are obtained based on the normal vector. For example, the process of obtaining the first tangent vector and the second tangent vector of each reference point based on the position information includes: for any reference point, estimating the normal vector of any reference point based on its position information to obtain the normal vector of that reference point, wherein the normal vector of any reference point is perpendicular to the first tangent vector and the second tangent vector of that reference point; and obtaining the first tangent vector and the second tangent vector of any reference point based on the normal vector of that reference point. The normal vector estimation includes, but is not limited to, tasks performed by Open3D's built-in interfaces.

[0065] In this implementation, the normal vector of the reference point is obtained by first estimating the normal vector of the reference point, and then the first and second tangent vectors that are perpendicular to each other are obtained based on the normal vector. The perpendicular relationship between the normal vector and the tangent vector can be used to accurately anchor the directional features of the local three-dimensional space of the reference point, ensuring that the two constructed tangent vectors are orthogonal and fit the geometry of the glass surface, avoiding deviations in the direction of the tangent plane determined based on the two tangent vectors. At the same time, this method of determining the tangent vector based on the normal vector can provide a stable and accurate planar reference for subsequent steps such as neighborhood point projection and polar angle calculation, reducing errors caused by improper setting of the tangent plane and improving the reliability of glass contour extraction.

[0066] In one possible implementation, the process of estimating the normal vector of any reference point based on its position information includes: determining the neighboring points of any reference point based on its position information; determining the position information of a target point based on the position information of its neighboring points; determining the deviation vectors of the neighboring points of any reference point relative to the target point based on the position information of the neighboring points and the target point; constructing the covariance matrix of any reference point based on the deviation vectors of its neighboring points relative to the target point; and performing eigenvalue decomposition on the covariance matrix of any reference point to obtain its normal vector.

[0067] In this implementation, by first determining the neighborhood points of the reference point and then determining the target point based on these neighborhood points, the deviation vector of the neighborhood points relative to the target point is calculated. This weakens the interference of single-point position errors of the reference point and focuses on the geometric distribution characteristics within the neighborhood. Constructing a covariance matrix based on the deviation vector and performing eigenvalue decomposition is a normal estimation method based on statistical properties. This method can accurately fit the true orientation of the local surface of the reference point, and the obtained normal vector has high accuracy and stability. At the same time, this method relies entirely on the position information of the reference point to complete the calculation, without the need to introduce complex prior parameters. It has strong adaptability and can provide a reliable directional reference for subsequent operations such as obtaining the first and second tangent vectors based on the normal vector and constructing the tangent plane, thereby improving the accuracy of glass contour extraction.

[0068] In one possible implementation, the process of determining the neighboring points of any reference point based on the location information of any reference point is similar to the process of determining the neighboring points of each first point described above. For example, it can also be implemented using Euclidean distance. The embodiments of this application will not be described in detail here.

[0069] In one possible implementation, the process of determining the location information of the target point based on the location information of the neighboring points of any reference point includes: determining second average location information based on the location information of the neighboring points of any reference point; and using the second average location information as the location information of the target point.

[0070] In one possible implementation, the deviation vector of the neighboring points of any reference point relative to the target point is determined according to the following formula (1) based on the position information of the neighboring points of any reference point and the position information of the target point.

[0071] (1) In the above formula (1), Let be the deviation vector of the j-th neighboring point of any reference point relative to the target point. For the location information of the j-th neighboring point of any reference point, This refers to the location information of the target point.

[0072] In one possible implementation, the covariance matrix of any reference point is constructed according to the following formula (2) based on the deviation vector of the neighborhood points of any reference point relative to the target point.

[0073] (2) In the above formula (2), Let be the covariance matrix of any reference point. The number of neighborhood points of any reference point. Let be the deviation vector of the j-th neighboring point of any reference point relative to the target point. for The transpose of .

[0074] In one possible implementation, the process of performing eigenvalue decomposition on the covariance matrix of any reference point to obtain the normal vector of any reference point includes: performing eigenvalue decomposition on the covariance matrix of any reference point to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; and taking the eigenvector corresponding to the smallest eigenvalue among the multiple eigenvalues ​​as the normal vector of any reference point.

[0075] For example, eigenvalue decomposition is performed on the covariance matrix of any reference point to obtain three eigenvalues ​​( , , ) and the eigenvectors corresponding to each eigenvalue ( Due to eigenvalues The smallest of the three eigenvalues ​​is the eigenvalue; therefore, the eigenvalue is... Corresponding feature vector The normal vector of any reference point.

[0076] In one possible implementation, after determining the normal vector of any reference point, a first tangent vector of that reference point is determined based on the normal vector, and this first tangent vector is perpendicular to the normal vector of the reference point. Then, a second tangent vector of that reference point is determined based on the normal vector and the first tangent vector, and this second tangent vector is perpendicular to both the normal vector and the first tangent vector. Optionally, the normal vector, the first tangent vector, and the second tangent vector are normalized to make them unit vectors.

[0077] Optionally, obtaining the first and second tangent vectors can be achieved by selecting a reference vector e that is not parallel to the normal vector, for example... Time to take Otherwise take ,in For the preset threshold, or Let be the normal vector of the reference point; calculate the first tangent vector. ,when Switch reference vector when the value is less than a preset threshold To avoid degradation; then calculate the second tangent vector. Thus, we obtain the first tangent vector and the second tangent vector, which are orthogonal to the normal vector and mutually orthogonal.

[0078] In step 203, the neighborhood points of each reference point are projected onto the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point.

[0079] In one possible implementation, before projecting the neighboring points of each reference point onto the tangent plane of each reference point, it is necessary to first determine the neighboring points and the tangent plane of each reference point. The process of determining the neighboring points of each reference point is similar to the process of determining the neighboring points of each first point described above, and will not be repeated here. The process of determining the tangent plane of each reference point includes: for any reference point, determining the tangent plane of that reference point based on the first tangent vector and the second tangent vector of that reference point.

[0080] Optionally, after determining the neighborhood points and tangent planes of each reference point, the neighborhood points of each reference point are projected onto the tangent planes of each reference point to obtain the projected points corresponding to the neighborhood points of each reference point. For example, the displacement vector of the neighborhood point relative to the reference point is orthogonally decomposed in the directions of the first and second tangent vectors to obtain the components along the directions of the first and second tangent vectors, respectively. This orthogonal decomposition is mathematically equivalent to orthogonally projecting the neighborhood points onto the tangent plane.

[0081] In one possible implementation, the first step is to check whether the number of neighboring points of the reference point reaches a second threshold. That is, if the number of neighboring points of the reference point is insufficient, the calculation of the maximum angular difference of the reference point is not performed. In other words, if the number of neighboring points of the reference point does not reach the second threshold, step 204 is not executed. Only when the number of neighboring points of the reference point reaches the second threshold, i.e., the number of neighboring points of the reference point is sufficient, is step 204 executed to determine the maximum angular difference of the reference point.

[0082] In step 204, the maximum angular difference between each reference point is determined based on the polar angle of the projection point corresponding to the neighboring points of each reference point.

[0083] In one possible implementation, the maximum angle difference between reference points is determined based on the distribution of polar angles of the projection points corresponding to the neighboring points of each reference point, for example, by determining the angle coverage within the neighborhood of each reference point. Optionally, before determining the maximum angle difference between reference points based on the polar angles of the projection points corresponding to the neighboring points of each reference point, the polar angles of the projection points corresponding to the neighboring points of each reference point also need to be determined. This application does not limit the method for determining the polar angles of the projection points corresponding to the neighboring points of each reference point. For example, the process of determining the polar angles of the projection points corresponding to the neighboring points of each reference point includes: for any neighboring point of any reference point, determining the polar angle of the projection point corresponding to any neighboring point of any reference point based on the first tangent vector, the second tangent vector of any reference point, and the position information of any neighboring point.

[0084] In this implementation, the first and second tangent vectors of the reference point are mutually perpendicular and serve as the orthogonal basis of the tangent plane of the reference point. The polar angle of the projection point corresponding to the neighboring point is calculated by combining the position information of the neighboring points. This can accurately quantify the orientation distribution characteristics of the projection point relative to the reference point in the tangent plane, providing a standardized and comparable quantitative indicator for the subsequent calculation of the maximum angle difference of the reference point. This lays a reliable data foundation for the subsequent accurate differentiation between edge points and non-edge points of the glass.

[0085] Optionally, based on the first tangent vector, the second tangent vector of any reference point and the position information of any neighboring point, the polar angle of the projection point corresponding to any neighboring point of any reference point is determined according to the following formula (3).

[0086] (3) Formula (3) above can also be expressed in other forms with the same meaning. For example, formula (3) can be expressed as: .

[0087] In the above formula (3), Let be the polar angle of the projection point corresponding to the j-th neighboring point of any reference point, or . It can also be called the polar angle of a neighboring point relative to a reference point in the tangent plane. Let be the displacement vector of the j-th neighboring point of any reference point relative to that reference point. Let be the second tangent vector of any reference point. Let be the first tangent vector of any reference point. For the location information of the j-th neighboring point of any reference point, For the location information of any reference point, Represents the dot product of vectors. ( ) is a bivariate arctangent function.

[0088] In one possible implementation, after determining the polar angles of the projection points corresponding to the neighboring points of each reference point, the process of determining the maximum angular difference between each reference point based on the polar angles of the projection points corresponding to the neighboring points includes: for any reference point, sorting the polar angles of the projection points corresponding to the neighboring points of that reference point to obtain a polar angle sequence; determining the absolute value of the difference between two adjacent polar angles in the polar angle sequence; calculating the circumferential angle interval between the first and last polar angles in the polar angle sequence; and taking the maximum value among the absolute value of the difference between two adjacent polar angles and the circumferential angle interval as the maximum angular difference of any reference point. In other words, the polar angle sequence is sorted in ascending order, and the difference between adjacent polar angles and the difference between the first and last circumferential angles are calculated. The maximum value among the above differences is taken as the maximum angular difference of the reference point. The unit of angle is radians or degrees, which is not limited in this application.

[0089] Optionally, taking angles as the unit of measurement, the method for calculating the circumferential angular interval between the first and last polar angles in the polar angle sequence includes, but is not limited to, determining the first difference between the first polar angle (the initial polar angle) and 360 degrees, and determining the second difference between the last polar angle (the final polar angle) and 360 degrees. The difference between the first and second differences is taken as the circumferential angular interval between the first and last polar angles in the polar angle sequence. For example, if the first polar angle (the initial polar angle) in the polar angle sequence is -170 degrees and the last polar angle (the final polar angle) is 170 degrees, since the first difference between -170 degrees and 360 degrees is 170 degrees, and the second difference between 170 degrees and 360 degrees is 190 degrees, the circumferential angular interval between the first and last polar angles in the polar angle sequence is 190 degrees - 170 degrees = 20 degrees.

[0090] Alternatively, the method for calculating the circumferential angular interval between the first and last polar angles in a polar angle sequence includes, but is not limited to, determining the difference between the first polar angle (the initial polar angle) and the last polar angle (the tail polar angle) in the polar angle sequence, and using the absolute value of this difference and the difference of a full circle of 360 degrees as the circumferential angular interval between the first and last polar angles in the polar angle sequence. For example, if the first polar angle (the initial polar angle) in the polar angle sequence is -170 degrees and the last polar angle (the tail polar angle) is 170 degrees, since the absolute value of the difference between -170 degrees and 170 degrees is 340 degrees, and the difference between 340 degrees and 360 degrees is 20 degrees, then the circumferential angular interval between the first and last polar angles in the polar angle sequence is 20 degrees.

[0091] By sorting the polar angles of the projection points corresponding to the neighboring points of the reference point, calculating the absolute value of the difference between two adjacent polar angles, and taking the maximum value among the absolute value of the difference between two adjacent polar angles and the surrounding angle interval, the degree of angular distribution dispersion of the projection points in the neighborhood of the reference point can be accurately quantified. This provides a key and reliable quantitative indicator for subsequent angular feature-based analysis and helps to quickly capture the angular change characteristics in the neighborhood.

[0092] In one possible implementation, the process of sorting the polar angles of the projection points corresponding to the neighboring points of any reference point to obtain a polar angle sequence includes: sorting the polar angles of the projection points corresponding to the neighboring points of any reference point according to the target order to obtain a polar angle sequence. The target order can be either ascending or descending; this embodiment of the application does not limit this.

[0093] In step 205, the edge point of the glass is determined among the reference points based on the maximum angular difference between each reference point and the number of neighboring points of each reference point.

[0094] In one possible implementation, the process of determining the glass edge points among the reference points based on the maximum angular difference between each reference point and the number of neighboring points of each reference point includes: selecting reference points whose maximum angular difference is not less than an angular difference threshold and whose number of neighboring points is not less than a second quantity threshold as glass edge points. For example, first determine whether the number of neighboring points of a reference point is not less than, i.e., greater than or equal to, the second quantity threshold. If the number of neighboring points of a reference point is not less than the second quantity threshold, then further determine whether the maximum angular difference between the reference points is not less than the angular difference threshold, thereby selecting reference points whose number of neighboring points is not less than the second quantity threshold and whose maximum angular difference is not less than the angular difference threshold as glass edge points.

[0095] The angle difference threshold is set based on experience, or can be flexibly adjusted according to the implementation environment; this embodiment does not limit this. For example, the angle difference threshold is 100 degrees. The second quantity threshold is set based on experience, or can be flexibly adjusted according to the implementation environment; this embodiment also does not limit this. For example, the second quantity threshold is 12.

[0096] Optionally, before designating a reference point whose maximum angle difference is not less than an angle difference threshold and whose number of neighboring points is not less than a second quantity threshold as the edge point of the glass, it is necessary to first determine the maximum angle difference of each reference point and the number of neighboring points of each reference point. The process of determining the maximum angle difference of each reference point has been described in step 204 above, and will not be repeated here in this embodiment. Before determining the number of neighboring points of each reference point, it is necessary to first determine the neighboring points of each reference point. The process of determining the neighboring points of each reference point is similar to the process of determining the neighboring points of the first point above, and this embodiment does not limit this process. After determining the neighboring points of each reference point, the number of neighboring points of each reference point can be determined based on the neighboring points of each reference point.

[0097] By setting dual screening conditions—the maximum angle difference being no less than the angle difference threshold and the number of neighboring points being no less than the second threshold—the edge points of the glass can be determined. This method can accurately distinguish between valid edge points and noise points, isolated points, and smooth area points. It captures the angle change features at the edge through a sufficiently large maximum angle difference to lock the edge position, and ensures that the selected edge points have reliable neighborhood support by using a sufficient number of neighboring points, avoiding misjudging sparse interference points as edge points. This significantly improves the accuracy, robustness, and anti-interference ability of glass edge detection.

[0098] This method can effectively identify points with large-angle gaps in the neighborhood distribution, while avoiding dependence on curvature or dual-scale normal difference in the edge determination stage. Therefore, it can still maintain high-precision edge recognition even when the surface is curved, the point cloud is missing, or the density is uneven.

[0099] In step 206, the outline of the glass is extracted based on the edge points of the glass.

[0100] In one possible implementation, the process of extracting the glass profile based on the glass edge points includes: connecting the glass edge points according to the spatial proximity of the edge points to obtain a spatial reference graphic; and determining the glass profile based on the spatial reference graphic.

[0101] Optionally, the spatial reference figure is a closed figure.

[0102] By first connecting the selected glass edge points sequentially according to their spatial proximity to obtain a spatial reference graphic, and then determining the glass contour based on this spatial reference graphic, the discrete and double-selected effective edge points can be transformed into a reference graphic with a preliminary shape framework. This avoids problems such as edge breaks and shape distortion that may occur when directly extracting the contour, and improves the integrity, continuity and shape accuracy of the determined glass contour.

[0103] In one possible implementation, the process of determining the glass outline based on a spatial reference graphic includes: using the spatial reference graphic as the glass outline; or, determining the length of each line segment in the spatial reference graphic, removing line segments in the spatial reference graphic whose length is not greater than a length threshold to obtain a target graphic, and using the target graphic as the glass outline.

[0104] In one possible implementation, the process of determining the glass outline based on the spatial reference graphic includes: using the spatial reference graphic as the glass outline; or, counting the number of edge points contained in each polyline in the spatial reference graphic, removing polylines with an edge point count not exceeding a threshold to obtain the target graphic, and using the target graphic as the glass outline.

[0105] The target graphic can be a closed shape. The length threshold is set based on experience, or it can be flexibly adjusted according to the implementation environment; this application embodiment does not limit this. For example, the length threshold is 2 centimeters.

[0106] By providing several flexible solutions for obtaining the target image and using it as the glass outline, such as directly using a spatial reference graphic as the glass outline, or removing line segments with a length not exceeding a length threshold from the spatial reference graphic, or removing polylines in the spatial reference graphic whose number of edge points does not exceed a threshold, this approach can efficiently obtain the glass outline by adapting to ideal scenarios with regular edge point distribution and no redundant interference. It can also filter out redundant details such as burrs, gaps, and noise caused by glass edge reflections and textures by removing short line segments, avoiding distortion and irregularity in the glass outline. This significantly improves the regularity, usability, and adaptability of the glass outline. Furthermore, both solutions have clear logic and adjustable parameters, allowing for flexible selection based on the edge quality requirements of different glass inspection scenarios, providing an accurate and reliable outline foundation for subsequent tasks such as glass size measurement and defect detection.

[0107] like Figure 4 This is a schematic diagram of the outline of a glass provided in an embodiment of this application. It should be noted that... Figure 4 The outline of the glass shown is represented by dots, that is... Figure 4 The lines shown may be dots, but if they are unclear or appear as lines due to image resolution or display issues, they are not intended to limit the way the glass outline is displayed in this application.

[0108] This application enables high-precision and robust edge extraction from 3D point cloud data of glass, demonstrating significant technical effects and advantages. Firstly, it utilizes the notch criterion of tangent plane angle coverage to identify glass edge points, eliminating the need for curvature calculations, dual-scale normal differences, or mesh topology. This simplifies the edge determination calculation process, reduces reliance on parameter adjustments, and improves the method's applicability and engineering feasibility.

[0109] Secondly, it can effectively address the problems of curved glass surfaces and incomplete point clouds. Through local tangent plane projection and neighborhood angle analysis, it can maintain the accuracy of edge determination in curved areas and stably identify edge points in areas with missing point clouds or uneven neighborhood distribution, thereby ensuring the integrity and reliability of edge extraction results.

[0110] Furthermore, by connecting the edge points of the glass according to their spatial proximity, not only can the efficient conversion from edge points to structured contours be achieved, but it can also be directly used for industrial applications such as dimensional measurement, assembly inspection, and CAD (Computer-Aided Design) comparison, significantly improving the availability of data and the efficiency of subsequent processing.

[0111] Furthermore, it boasts excellent computational performance and industrial applicability. Its time complexity primarily consists of neighborhood search and polar angle sorting, and is related to the number of reference points N and the number of neighboring points k. This ensures both efficiency in large-scale point cloud processing and meets the requirements for high-precision edge extraction. Through reasonable parameter settings, robust edge detection can be achieved on point clouds of glass with different shapes, curvatures, and sampling densities, providing reliable technical support for the precision assembly and quality control of glass parts in vehicle manufacturing.

[0112] The aforementioned method constructs tangent planes for each reference point by obtaining two mutually perpendicular tangent vectors. By projecting neighboring points of each reference point onto these tangent planes, interference from irrelevant directions in 3D space is effectively eliminated, focusing on the feature distribution within the local plane. Simultaneously, by combining the polar angles of the projection points of neighboring points, the maximum angular difference between each reference point is determined, accurately capturing the difference between edge and non-edge points of the glass. The increased number of neighboring points further filters out noise and outliers, reducing the risk of misidentification as edge points and achieving precise edge point localization. This results in a more reliable and accurate glass contour extraction. Furthermore, this method exhibits good adaptability to glass with different shapes and surface conditions, improving the versatility and accuracy of glass contour extraction.

[0113] Figure 5 This is a flowchart of a glass contour extraction method provided in an embodiment of this application, as shown below. Figure 5 As shown, the method includes the following steps 501 to 510.

[0114] In step 501, the three-dimensional point cloud data of the glass whose contour is to be extracted is obtained. The three-dimensional point cloud data includes the position information of each point in the glass.

[0115] In one possible implementation, the process of acquiring the three-dimensional point cloud data of the glass whose contour to be extracted has been described in step 201 above, and will not be repeated here in the embodiments of this application.

[0116] In step 502, a reference point is determined for each point based on its location information.

[0117] In one possible implementation, the process of determining a reference point among the points based on the location information of each point has been described in step 201 above, and will not be repeated here in the embodiments of this application.

[0118] In step 503, the normal vector of each reference point is obtained based on the position information of each reference point.

[0119] In one possible implementation, the process of obtaining the normal vector of each reference point based on the position information of each reference point has been described in step 202 above, and will not be repeated here in the embodiments of this application.

[0120] In step 504, the first tangent vector and the second tangent vector of each reference point are obtained based on the normal vector of each reference point.

[0121] In this context, the normal vector, the first tangent vector, and the second tangent vector of any reference point are mutually perpendicular.

[0122] In one possible implementation, the process of obtaining the first tangent vector and the second tangent vector of each reference point based on the normal vector of each reference point has been described in step 202 above, and will not be repeated here in the embodiments of this application.

[0123] In step 505, the neighborhood points of each reference point are determined.

[0124] In one possible implementation, the process of determining the neighboring points of each reference point has been described in step 203 above, and will not be repeated here in the embodiments of this application.

[0125] In step 506, the neighboring points of each reference point are projected onto the tangent plane of each reference point to obtain the projection points corresponding to the neighboring points of each reference point.

[0126] The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point.

[0127] In one possible implementation, the process of projecting the neighborhood points of each reference point onto the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point has been described in step 203 above, and will not be repeated here in the embodiments of this application.

[0128] In step 507, the polar angle of the projection point corresponding to the neighboring point of each reference point is determined.

[0129] In one possible implementation, the process of determining the polar angle of the projection point corresponding to the neighboring points of each reference point has been described in step 204 above, and will not be repeated here in the embodiments of this application.

[0130] In step 508, the maximum angular difference between each reference point is determined based on the polar angle of the projection point corresponding to the neighboring points of each reference point.

[0131] In one possible implementation, the process of determining the maximum angular difference between each reference point based on the polar angle of the projection point corresponding to the neighboring points of each reference point has been described in step 204 above, and will not be repeated here in the embodiments of this application.

[0132] In step 509, the edge point of the glass is determined among the reference points based on the maximum angular difference between each reference point and the number of neighboring points of each reference point.

[0133] In one possible implementation, the process of determining the edge point of the glass among each reference point based on the maximum angular difference between each reference point and the number of neighboring points of each reference point has been described in step 205 above, and will not be repeated here in the embodiments of this application.

[0134] In step 510, the outline of the glass is extracted based on the edge points of the glass.

[0135] In one possible implementation, the process of extracting the glass contour based on the glass edge points has been described in step 206 above, and will not be repeated here in the embodiments of this application.

[0136] Figure 6 The diagram shown is a structural schematic of a glass contour extraction device provided in an embodiment of this application. Figure 6 As shown, the device includes: The acquisition module 601 is used to acquire the position information of each reference point in the glass whose contour is to be extracted; The acquisition module 601 is also used to acquire the first tangent vector and the second tangent vector of each reference point based on the position information of each reference point, wherein the first tangent vector and the second tangent vector are perpendicular. The projection module 602 is used to project the neighborhood points of each reference point onto the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point. The determination module 603 is used to determine the maximum angle difference between each reference point based on the polar angle of the projection point corresponding to the neighboring points of each reference point. The determination module 603 is also used to determine the edge point of the glass in each reference point based on the maximum angular difference between each reference point and the number of neighboring points of each reference point; Extraction module 604 is used to extract the outline of the glass based on the edge points of the glass.

[0137] In one possible implementation, the acquisition module 601 is also used to acquire three-dimensional point cloud data of the glass, the three-dimensional point cloud data including the position information of each point in the glass; The determination module 603 is also used to determine a reference point among the points based on the location information of each point.

[0138] In one possible implementation, the determining module 603 is used to perform voxel downsampling on each point based on the position information of each point to obtain a set of voxel units after downsampling. The set of voxel units includes multiple voxels, and each voxel includes at least one point. Based on the points included in each voxel, the first point corresponding to each voxel is determined. The neighboring points of each first point are determined. The first point is filtered based on the number of neighboring points of the first point, and a reference point is determined based on the filtering results.

[0139] In one possible implementation, the determining module 603 is used to determine a first region based on the location information of any first point among the various first points; to take a second point located in the first region as a neighboring point of any first point, wherein the second point is a point other than the first point among the various points; or, to determine the distance between any first point and the second point based on the location information of any first point and the location information of the second point; and to take the second point as a neighboring point of any first point if the distance between any first point and the second point is less than a distance threshold.

[0140] In one possible implementation, the acquisition module 601 is used to estimate the normal vector of any reference point based on its position information, and obtain the normal vector of any reference point. The normal vector of any reference point is perpendicular to the first tangent vector and the second tangent vector of any reference point. Based on the normal vector of any reference point, the first tangent vector and the second tangent vector of any reference point are obtained.

[0141] In one possible implementation, the acquisition module 601 is used to: determine the neighboring points of any reference point based on the position information of any reference point; determine the position information of a target point based on the position information of the neighboring points of any reference point; determine the deviation vector of the neighboring points of any reference point relative to the target point based on the position information of the neighboring points of any reference point and the position information of the target point; construct the covariance matrix of any reference point based on the deviation vector of the neighboring points of any reference point relative to the target point; and perform eigenvalue decomposition on the covariance matrix of any reference point to obtain the normal vector of any reference point.

[0142] In one possible implementation, the determining module 603 is further configured to, for any neighboring point of any reference point among the various reference points, determine the polar angle of the projection point corresponding to any neighboring point of any reference point based on the first tangent vector, the second tangent vector of any reference point and the position information of any neighboring point.

[0143] In one possible implementation, the determining module 603 is used to sort the polar angles of the projection points corresponding to the neighboring points of any reference point for any reference point among the various reference points to obtain a polar angle sequence; determine the absolute value of the difference between two adjacent polar angles in the polar angle sequence, and calculate the circumferential angle interval between the first and last polar angles in the polar angle sequence; and take the maximum value of the absolute value of the difference between two adjacent polar angles in the polar angle sequence and the circumferential angle interval as the maximum angle difference of any reference point.

[0144] In one possible implementation, the determining module 603 is used to identify reference points among the reference points whose maximum angle difference is not less than an angle difference threshold and whose number of neighboring points is not less than a second quantity threshold as edge points of the glass.

[0145] In one possible implementation, the extraction module 604 is used to connect the edge points of the glass according to spatial proximity to obtain a spatial reference graphic; and to determine the outline of the glass based on the spatial reference graphic.

[0146] In one possible implementation, the extraction module 604 is used to use the spatial reference graphic as the outline of the glass; or, it counts the number of edge points contained in each polyline in the spatial reference graphic, removes polylines with an edge point count not exceeding a threshold, obtains the target graphic, and uses the target graphic as the outline of the glass.

[0147] The aforementioned device constructs tangent planes for each reference point by acquiring two mutually perpendicular tangent vectors. By projecting neighboring points of each reference point onto these tangent planes, it effectively eliminates interference from irrelevant directions in three-dimensional space, focusing on the feature distribution within the local plane. Simultaneously, by combining the polar angles of the projection points of neighboring points, the maximum angular difference between each reference point is determined, accurately capturing the difference between edge and non-edge points of the glass. The increased number of neighboring points further filters out noise and outliers, reducing the risk of misidentifying points as edges, thus achieving precise edge point localization and extracting a more reliable and accurate glass contour. Furthermore, it exhibits good adaptability to glass with different shapes and surface conditions, improving the versatility and accuracy of glass contour extraction.

[0148] It should be understood that the above-described apparatus is only illustrated by the division of the functional modules described above when implementing its functions. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0149] Figure 7This illustration shows a structural block diagram of an electronic device 700 provided in an exemplary embodiment of this application. The electronic device 700 can be any electronic device product capable of human-computer interaction with a user through one or more methods such as a keyboard, touchpad, remote control, voice interaction, or handwriting device. Examples include PCs (Personal Computers), mobile phones, smartphones, PDAs (Personal Digital Assistants), wearable devices, PPCs (Pocket PCs), tablet computers, and smart vehicle systems.

[0150] Typically, electronic device 700 includes a processor 701 and a memory 702.

[0151] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 701 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0152] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 is used to store at least one instruction, which is executed by the processor 701 to implement the glass contour extraction method provided in the method embodiments of this application.

[0153] In some embodiments, the electronic device 700 may optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, and a power supply 708.

[0154] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0155] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminal devices through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0156] Display screen 705 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, disposed on the front panel of electronic device 700; in other embodiments, there may be at least two display screens, disposed on different surfaces of electronic device 700 or in a folded design; in still other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of electronic device 700. Furthermore, display screen 705 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0157] The camera assembly 706 is used to acquire images or videos. Optionally, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device 700, and the rear-facing camera is located on the back of the electronic device 700. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0158] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the electronic device 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0159] Power supply 708 is used to supply power to various components in electronic device 700. Power supply 708 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 708 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0160] In some embodiments, the electronic device 700 further includes one or more sensors 709.

[0161] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the electronic device 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0162] Figure 8 This is a schematic diagram of the server structure provided in the embodiments of this application. The server 800 can vary considerably due to different configurations or performance. It may include one or more processors (Central Processing Units, CPUs) 801 and one or more memories 802. The one or more memories 802 store at least one line of program code, which is loaded and executed by the one or more processors 801 to implement the glass contour extraction method provided in the various method embodiments described above. Of course, the server 800 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 800 may also include other components for implementing device functions, which will not be elaborated here.

[0163] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one piece of program code that is loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for extracting glass contours.

[0164] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0165] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer instruction that is loaded and executed by a processor to enable an electronic device to implement any of the above-described methods for extracting glass contours.

[0166] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the 3D point cloud data of the glass involved in this application was obtained with full authorization.

[0167] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0168] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for extracting glass contours, characterized in that, The method includes: Obtain the position information of each reference point in the glass containing the contour to be extracted; Based on the position information of each reference point, obtain the first tangent vector and the second tangent vector of each reference point, wherein the first tangent vector and the second tangent vector are perpendicular; The neighboring points of each reference point are projected onto the tangent plane of each reference point to obtain the projection points corresponding to the neighboring points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point. The maximum angle difference between each reference point is determined based on the polar angle of the projection point corresponding to the neighboring points of each reference point. Based on the maximum angular difference between each reference point and the number of neighboring points of each reference point, the edge point of the glass is determined among each reference point; The outline of the glass is extracted based on the edge points of the glass.

2. The method according to claim 1, characterized in that, Before obtaining the position information of each reference point in the glass containing the contour to be extracted, the method further includes: Acquire three-dimensional point cloud data of the glass, wherein the three-dimensional point cloud data includes the position information of each point in the glass; Based on the location information of each point, the reference point is determined among each point.

3. The method according to claim 2, characterized in that, The step of determining the reference point among the points based on the location information of each point includes: Based on the location information of each point, a voxel downsampling operation is performed on each point to obtain a set of voxel units after downsampling. The set of voxel units includes multiple voxels, and each voxel includes at least one point. Based on the points included in each voxel, determine the first point corresponding to each voxel; Determine the neighborhood points of each first point; The first point is filtered based on the number of its neighboring points, and the reference point is determined based on the filtering results.

4. The method according to claim 3, characterized in that, Determining the neighborhood points of each first point includes: For any one of the aforementioned first points, a first region is determined based on the location information of that first point; a second point located within the first region is taken as a neighboring point of that first point, where the second point is any point among the aforementioned points other than the first point; or... Based on the location information of any first point and the location information of the second point, the distance between any first point and the second point is determined; if the distance between any first point and the second point is less than a distance threshold, the second point is taken as a neighboring point of any first point.

5. The method according to claim 1, characterized in that, The step of obtaining the first tangent vector and the second tangent vector of each reference point based on the position information of each reference point includes: For any one of the reference points, the normal vector of any one reference point is estimated based on its position information to obtain the normal vector of any one reference point. The normal vector of any one reference point is perpendicular to the first tangent vector and the second tangent vector of any one reference point. Based on the normal vector of any reference point, obtain the first tangent vector and the second tangent vector of any reference point.

6. The method according to claim 5, characterized in that, The step of estimating the normal vector of any reference point based on its position information to obtain the normal vector of the reference point includes: Based on the location information of any reference point, determine the neighboring points of any reference point; The location information of the target point is determined based on the location information of the neighboring points of any reference point. Based on the position information of the neighboring points of any reference point and the position information of the target point, determine the deviation vector of the neighboring points of any reference point relative to the target point; Construct the covariance matrix of any reference point based on the deviation vector of its neighboring points relative to the target point; The covariance matrix of any reference point is decomposed into eigenvalues ​​to obtain the normal vector of any reference point.

7. The method according to any one of claims 1 to 6, characterized in that, Before determining the maximum angular difference between the reference points based on the polar angles of the projection points corresponding to the neighboring points of each reference point, the method further includes: For any neighboring point of any of the reference points, the polar angle of the projection point corresponding to any neighboring point of any of the reference points is determined based on the first tangent vector, the second tangent vector of any reference point, and the position information of any neighboring point.

8. The method according to any one of claims 1 to 6, characterized in that, Determining the maximum angle difference among the reference points based on the polar angles of the projection points corresponding to the neighboring points of each reference point includes: For any one of the reference points, sort the polar angles of the projection points corresponding to the neighboring points of any one reference point to obtain a polar angle sequence; Determine the absolute value of the difference between two adjacent polar angles in the polar angle sequence, and calculate the circumference interval between the first and last polar angles in the polar angle sequence; The absolute value of the difference between two adjacent polar angles in the polar angle sequence and the maximum value in the surrounding angle interval are taken as the maximum angle difference of any reference point.

9. The method according to any one of claims 1 to 6, characterized in that, The step of determining the edge point of the glass among the reference points based on the maximum angular difference between the reference points and the number of neighboring points of each reference point includes: The reference point whose maximum angle difference is not less than the angle difference threshold and whose number of neighboring points is not less than the second quantity threshold is taken as the edge point of the glass.

10. The method according to any one of claims 1 to 6, characterized in that, The step of extracting the contour of the glass based on its edge points includes: The edge points of the glass are connected according to their spatial proximity to obtain a spatial reference graphic; The outline of the glass is determined based on the spatial reference drawing.

11. The method according to claim 10, characterized in that, Determining the outline of the glass based on the spatial reference drawing includes: Use the spatial reference graphic as the outline of the glass; or, The number of edge points contained in each polyline in the spatial reference graphic is counted, and polylines with an edge point count not exceeding a threshold are removed to obtain the target graphic, which is then used as the outline of the glass.

12. A device for extracting glass contours, characterized in that, The device includes: The acquisition module is used to acquire the position information of each reference point in the glass whose contour is to be extracted; The acquisition module is further configured to acquire a first tangent vector and a second tangent vector of each reference point based on the position information of each reference point, wherein the first tangent vector and the second tangent vector are perpendicular; The projection module is used to project the neighborhood points of each reference point onto the tangent plane of each reference point to obtain the projection points corresponding to the neighborhood points of each reference point. The tangent plane of any reference point is determined based on the first tangent vector and the second tangent vector of any reference point. The determination module is used to determine the maximum angle difference between the reference points based on the polar angle of the projection points corresponding to the neighboring points of each reference point. The determining module is further configured to determine the edge point of the glass among the reference points based on the maximum angle difference between the reference points and the number of neighboring points of the reference points. The extraction module is used to extract the outline of the glass based on the edge points of the glass.

13. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one piece of program code, which is loaded and executed by the processor to enable the electronic device to implement the glass contour extraction method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to enable the electronic device to implement the glass contour extraction method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable the electronic device to implement the glass contour extraction method as described in any one of claims 1 to 11.

Images