A glass detection method combining two-dimensional laser point cloud and ultrasonic wave

Abstract

The application provides a glass detection method combining two-dimensional laser point cloud and ultrasonic waves, and is characterized in that the method comprises a laser radar and two ultrasonic sensors, and further comprises the following steps: processing laser radar sensor data and classifying laser point cloud data; analyzing basic features of the point cloud through an Euclidean distance method, screening the classified point cloud according to the feature that the point cloud of the glass direction is dense and high in intensity, and determining the position of the clustered laser point cloud; and simultaneously, the laser radar point is determined through ultrasonic assistance. The application clusters the laser point cloud, uses the clustering feature of the laser point cloud in the normal direction of the glass as a premise for judging the existence of the glass, and improves the processing speed of the algorithm.

Description

Technical Field

[0001] This invention belongs to the field of optical technology, specifically relating to a glass inspection method that combines two-dimensional laser point clouds and ultrasonic waves. Background Technology

[0002] With the advancement of industrial and intelligent technologies, an increasing number of intelligent products are emerging in the Industry 4.0 era, with intelligent mobile robots being a prime example. The deployment of intelligent mobile robots requires effective integration with displayed activity scenarios, such as warehouses, supermarkets, and outdoor inspection routes. In the intelligent deployment of robot products, glass inspection is an unavoidable aspect. Glass is typically a high-transparency, high-light-transmittance material, whose applications enrich architectural design and optimize aesthetics. However, for optical sensors in mobile robots that require reflected light data for measurement, such as lasers, monoculars, binoculars, and depth cameras, glass significantly impacts measurement accuracy and mapping results.

[0003] Regarding lasers, laser point clouds mostly pass through glass, making it impossible to detect their presence. Consequently, mobile robots cannot effectively avoid glass obstacles, hindering their deployment in environments with numerous glass surfaces, such as hotels and shopping malls. Uniquely, laser point clouds converge perpendicular to the glass, exhibiting high intensity and density, while point clouds in other directions diffuse and pass through the glass. Based on this unique point cloud characteristic, glass recognition technology uses it as an initial screening criterion, incorporating convolutional image inpainting to attempt to identify glass contours, determine glass presence, and mark it on a map. However, in shopping mall scenarios with many glass surfaces, contours cannot be inferred from walls, making it difficult to effectively construct a loss function to inversely adjust convolution parameters. Therefore, glass detection still faces many limitations. Summary of the Invention

[0004] This invention provides a glass inspection method that combines two-dimensional laser point cloud and ultrasonic waves. By using point cloud clustering and processing a small amount of point cloud data, it saves a lot of computing power and can be more effectively implemented in actual machines.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A glass inspection method combining two-dimensional laser point cloud and ultrasonic waves includes a lidar and two ultrasonic sensors, and further includes the following steps:

[0007] Step S1: Process the lidar sensor data and classify the lidar point cloud data;

[0008] Step S2: Analyze the basic features of the point cloud using the Euclidean distance method. Based on the characteristics of dense and high-intensity clustering of point clouds in the glass direction, filter the classified point clouds to determine the location of the clustered laser point clouds.

[0009] Step S3: Simultaneously with step S2, the point is determined by ultrasonic-assisted lidar, using the following formula:

[0010]

[0011]

[0012]

[0013] and The minimum distance obtained when measuring the glass wall with ultrasonic wave 1 and ultrasonic wave 2 is the radius of the inscribed circle of the ultrasonic arc measurement model and the glass wall, respectively. The distance information between the feature points on the glass wall and the lidar is obtained from point cloud clustering information; This represents the distance measured by the lidar relative to the straight line between two symmetrical ultrasonic radars in a mobile robot. This indicates the angle information between ultrasonic wave 1 and the central axis of the mobile robot. This information is determined by the design and installation position of the sensor. This indicates the angle between the mobile robot and the glass wall; , These represent the measured distances of ultrasonic waves 1 and 2 at the laser radar incident point on the glass wall, respectively.

[0014] Given that the characteristics of the glass laser point cloud are clearly defined, the solution to be obtained is... , and Based on the characteristics of the glass laser point cloud, the extent of the glass boundary and its coordinates in the world coordinate system can be calculated.

[0015] Furthermore, step S1 is detailed as follows:

[0016] The lidar data undergoes preliminary processing using Euclidean clustering, specifically classifying the lidar point cloud using Euclidean distance. The basic clustering formula is as follows:

[0017]

[0018] In the formula, Represents the clustering function. Indicates the first The pose of each laser point. Indicates the number of clusters. This is the maximum value.

[0019] Furthermore, step S2 is detailed as follows:

[0020] The data features of each cluster of point clouds include the centroid pose, average intensity, and cluster radius parameter.

[0021] The method for calculating the centroid pose of a point cloud is as follows:

[0022]

[0023] In the formula, This indicates the centroid pose of the point cloud. This represents a summary of cluster points. Indicates the first The laser point cloud pose of the point;

[0024] The method for calculating the average intensity of point clouds is as follows:

[0025]

[0026] In the formula, This represents the average intensity value of the point cloud. This represents a summary of cluster points. Indicates the first The intensity value of the laser point cloud at the point;

[0027] The method for calculating the cluster radius of point clouds is as follows:

[0028]

[0029] In the formula, Represents the point cloud clustering radius. This represents a summary of cluster points. Indicates the first The second normal form of the laser point cloud pose of a point to the centroid of the point cloud is given, and the distance between the two points is recorded.

[0030] Furthermore, the specific formula for step S3 is as follows:

[0031]

[0032]

[0033] and The minimum distance obtained when measuring the glass wall with ultrasonic wave 1 and ultrasonic wave 2 is the radius of the inscribed circle of the ultrasonic arc measurement model and the glass wall, respectively. The distance information between the feature points on the glass wall and the lidar is obtained from point cloud clustering information; This represents the distance measured by the lidar relative to the straight line between two symmetrical ultrasonic radars in a mobile robot. This indicates the angle information between ultrasonic wave 1 and the central axis of the mobile robot. This information is determined by the design and installation position of the sensor. This indicates the angle between the mobile robot and the glass wall; , These represent the measured distances of ultrasonic waves 1 and 2 at the laser radar incident point on the glass wall, respectively.

[0034] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0035] 1. This invention improves the accuracy of glass inspection by jointly constructing a measurement model using ultrasonic waves and lidar;

[0036] 2. This invention improves the processing speed of the algorithm by clustering laser point clouds and using the clustering characteristics of laser point clouds in the glass normal direction as a prerequisite for judging the presence of glass.

[0037] Compared to manual calibration methods, the method proposed in this invention has lower labor costs and is more suitable for large-scale scenarios;

[0038] 3. Compared with image-based detection methods, the method proposed in this invention is more efficient, requires less computing power, and has lower requirements for model training data, which is more conducive to the rapid deployment and verification of products. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the glass wall measurement model;

[0040] Figure 2 This is a flowchart of laser point cloud clustering analysis;

[0041] Figure 3 This is a schematic diagram of ultrasonic and lidar combined measurement. Detailed Implementation

[0042] The present invention is described below based on embodiments, but the invention is not limited to these embodiments. In the detailed description of the invention below, certain specific details are described in detail. Those skilled in the art will fully understand the invention even without these details. To avoid obscuring the essence of the invention, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0043] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0044] Furthermore, it should be understood that in the following description, "circuit" refers to a conductive loop consisting of at least one element or sub-circuit connected by electrical or electromagnetic connections. When an element or circuit is said to be "connected" to another element or "connected" between two nodes, it can be directly coupled or connected to another element, or there may be intermediate elements. The connection between elements can be physical, logical, or a combination thereof. Conversely, when an element is said to be "directly coupled to" or "directly connected" to another element, it means that there are no intermediate elements between them.

[0045] Unless the context explicitly requires it, the words "comprising," "including," and similar terms throughout the specification and claims should be interpreted as encompassing rather than being exclusive or exhaustive; that is, meaning "including but not limited to."

[0046] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0047] The proposed geometry model includes a lidar and two ultrasonic sensors, such as... Figure 1 As shown. First, the lidar sensor data should be processed to construct an ultrasonic collaborative measurement model.

[0048] like Figure 2 As shown, the lidar data undergoes preliminary processing using clustering methods, classifying the lidar point clouds based on Euclidean distance. Next, the data characteristics of each clustered point cloud are calculated: point cloud location, average point cloud intensity, and normal distribution parameters of point cloud intensity. Then, based on the characteristic of dense and high-intensity clustering of point clouds along the glass direction, the classified point clouds are filtered to determine the location of the clustered lidar point clouds.

[0049] Measurement models for ultrasound and lidar are as follows: Figure 3 As shown in the diagram, and The minimum distance obtained when ultrasonic waves 1 and 2 measure the glass wall is the radius of the inscribed circle of the ultrasonic arc measurement model and the glass wall, respectively; l represents the distance information of the feature points of the glass wall from the lidar, which is obtained from point cloud clustering information; This represents the distance measured by the lidar relative to the straight line between two symmetrical ultrasonic radars in a mobile robot. This indicates the angle information between ultrasonic wave 1 and the central axis of the mobile robot. This information is determined by the design and installation position of the sensor. This indicates the angle between the mobile robot and the glass wall; , These represent the measured distances of ultrasonic waves 1 and 2 at the laser radar incident point on the glass wall, respectively.

[0050] The measurement model shows that, when the features of the glass laser point cloud are clear, it is only necessary to solve the problem based on the model. , , By analyzing the glass laser point cloud features, the extent of the glass boundary and its coordinates in the world can be easily calculated, and then applied to maps for navigation and obstacle avoidance.

[0051] From the relationship of the projection of the shaded triangle, we can obtain:

[0052]

[0053] .

Claims

1. A glass inspection method combining two-dimensional laser point cloud and ultrasonic waves, characterized in that, It includes a lidar and an ultrasonic sensor, wherein the number of ultrasonic sensors is two, and further includes the following steps: Step S1: Process the lidar sensor data and classify the lidar point cloud data; Step S2: Analyze the basic features of the point cloud using the Euclidean distance method. Based on the characteristics of dense and high-intensity clustering of point clouds in the glass direction, filter the classified point clouds to determine the location of the clustered laser point clouds. Step S3: Simultaneously with step S2, the point is determined by ultrasonic-assisted lidar, using the following formula: and The minimum distance obtained when measuring the glass wall with ultrasonic wave 1 and ultrasonic wave 2 is the radius of the inscribed circle of the ultrasonic arc measurement model and the glass wall, respectively. The distance information between the feature points on the glass wall and the lidar is obtained from point cloud clustering information; This represents the distance measured by the lidar relative to the straight line between two symmetrical ultrasonic radars in a mobile robot. This indicates the angle information between ultrasonic wave 1 and the central axis of the mobile robot. This information is determined by the design and installation position of the sensor. This indicates the angle between the mobile robot and the glass wall; , These represent the measured distances of ultrasonic waves 1 and 2 at the laser radar incident point on the glass wall, respectively. Given that the characteristics of the glass laser point cloud are clearly defined, the solution to be obtained is... , and Based on the characteristics of the glass laser point cloud, the extent of the glass boundary and its coordinates in the world coordinate system can be calculated.

2. The glass inspection method combining two-dimensional laser point cloud and ultrasonic waves according to claim 1, characterized in that, Step S1 is as follows: The lidar data undergoes preliminary processing using Euclidean clustering, specifically classifying the lidar point cloud using Euclidean distance. The basic clustering formula is as follows: In the formula, Represents the clustering function. Indicates the first The pose of each laser point. Indicates the number of clusters. This is the maximum value.

3. The glass inspection method combining two-dimensional laser point cloud and ultrasonic waves according to claim 1, step S2 is as follows: The data features of each cluster of point clouds include the centroid pose, average intensity, and cluster radius parameter. The method for calculating the centroid pose of a point cloud is as follows: In the formula, This indicates the centroid pose of the point cloud. This represents a summary of cluster points. Indicates the first The laser point cloud pose of the point; The method for calculating the average intensity of point clouds is as follows: In the formula, This represents the average intensity value of the point cloud. This represents a summary of cluster points. Indicates the first The intensity value of the laser point cloud at the point; The method for calculating the cluster radius of point clouds is as follows: In the formula, Represents the point cloud clustering radius. This represents a summary of cluster points. Indicates the first The second normal form of the laser point cloud pose of a point to the centroid of the point cloud is given, and the distance between the two points is recorded.

Images