A hole position calibration method, terminal device and storage medium for rock drill

Abstract

The present application provides a hole position verification method, terminal device and storage medium for a rock drill, the method comprising: synchronously collecting image data and point cloud data from the rock and the rock drill at multiple time points; projecting the laser points of the point cloud data on the ground onto the pixel points of the image data to obtain a joint point; separating a first target point representing a reflective sheet and a candidate point representing the rock drill from the joint point according to the coordinates and reflection intensity; performing semantic segmentation on the candidate point according to the color component and the coordinates to obtain a second target point representing the drill rod in the rock drill; planning a guided drilling path from the second target point to the first target point; using multiple second target points to generate a real-time drilling path; if the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, performing an attenuation operation on the parameters controlling the drill rod in the rock drill. Reduce the probability of deviation of the rock drill when drilling, improve the accuracy of drilling, and ensure the progress of the project.

Description

Technical Field

[0001] The embodiments of the present application belong to the technical field of rock drills, and in particular, relate to a hole position calibration method, terminal equipment, and storage medium for rock drills. Background Art

[0002] Rock drills are used for high-intensity mining and drilling operations. They use a multi-degree-of-freedom robotic arm equipped with a drill rod to drill holes in rocks.

[0003] The outdoor environment of the rock drill is relatively complex, which accelerates the aging of the rock drill. In addition, the hydraulic system of the robotic arm has poor stability and is easily affected by various outdoor environmental factors, resulting in abnormalities such as oil leakage and component damage.

[0004] In these cases, the rock drill mainly relies on its own sensors to verify whether the drilling is accurate. However, the sensors themselves also have certain errors, and are affected by the vibration during drilling for a long time, which may cause abnormalities during drilling, resulting in drilling deviations and affecting the progress of the project. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a hole position calibration method, a terminal device and a storage medium for a rock drill, so as to improve the accuracy of the rock drill when drilling.

[0006] A first aspect of an embodiment of the present application provides a hole position calibration method for a rock drill, comprising:

[0007] Image data and point cloud data are collected synchronously from the rock and the rock drill at multiple time points; reflective sheets are attached to the set hole positions on the rock; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity;

[0008] Projecting the laser points of the point cloud data above the ground onto the pixel points of the image data to obtain joint points;

[0009] Separating a first target point representing the reflective sheet and a candidate point representing the rock drill from the joint points according to the coordinates and the reflection intensity;

[0010] Performing semantic segmentation on the candidate point according to the color component and the coordinates to obtain a second target point representing a drill rod in the rock drill;

[0011] Mark the current time point as a first monitoring point, and mark the time point after the first monitoring point as a second monitoring point;

[0012] At the first monitoring point, planning a guided drilling path from the second target point to the first target point;

[0013] At the plurality of the second monitoring points, generating a real-time drilling path using the plurality of the second target points;

[0014] If the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, an alarm operation is performed, and a decay operation is performed on the parameter of the rock drill that controls the drill rod, and the process of marking the current time point as a first monitoring point and marking the time point after the first monitoring point as a second monitoring point is returned to execution;

[0015] If the first target point overlaps with the second target point, it is determined that the position of the hole drilled by the drill rod on the rock is aligned with the hole position.

[0016] A second aspect of an embodiment of the present application provides a hole position verification device for a rock drill, comprising:

[0017] A perception module is used to synchronously collect image data and point cloud data from the rock and the rock drill at multiple time points; the hole positions set on the rock are pasted with reflective sheets; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity;

[0018] A point cloud projection module, used to project the laser points of the point cloud data above the ground onto the pixel points of the image data to obtain a joint point;

[0019] A joint point separation module, used for separating a first target point representing the reflective sheet and a candidate point representing the rock drill from the joint points according to the coordinates and the reflection intensity;

[0020] A semantic segmentation module, configured to perform semantic segmentation on the candidate point according to the color component and the coordinates to obtain a second target point representing a drill rod in the rock drill;

[0021] A time point marking module, used to mark the current time point as a first monitoring point, and mark the time point after the first monitoring point as a second monitoring point;

[0022] A guided drilling path planning module, used for planning a guided drilling path from the second target point to the first target point at the first monitoring point;

[0023] A real-time drilling path generation module, used to generate a real-time drilling path at a plurality of the second monitoring points using a plurality of the second target points;

[0024] a deviation processing module, configured to, if the deviation between the real-time drilling path and the guided drilling path is greater than a drilling threshold, perform an alarm operation, and perform an attenuation operation on a parameter of the rock drill for controlling the drill rod, and return to execute the time point marking module;

[0025] The alignment determination module is used to determine that the position of the hole drilled by the drill rod on the rock is aligned with the hole position if the first target point overlaps with the second target point.

[0026] A third aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the hole position verification method for a rock drill as described in the first aspect above is implemented.

[0027] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the hole position verification method of a rock drill as described in the first aspect above is implemented.

[0028] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the hole position verification method for a rock drill as described in the first aspect.

[0029] In this embodiment, image data and point cloud data are synchronously collected from the rock and the rock drill at multiple time points; reflective sheets are pasted on the set hole positions on the rock; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity; the laser points of the point cloud data above the ground are projected onto the pixel points of the image data to obtain joint points; the first target point representing the reflective sheet and the candidate point representing the rock drill are separated from the joint point based on the coordinates and the reflection intensity; semantic segmentation is performed on the candidate points based on the color components and the coordinates to obtain the second target point representing the drill rod in the rock drill. point; mark the current time point as the first monitoring point, and mark the time point after the first monitoring point as the second monitoring point; at the first monitoring point, plan a guided drilling path from the second target point to the first target point; at multiple second monitoring points, use multiple second target points to generate a real-time drilling path; if the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, perform an alarm operation, and perform an attenuation operation on the parameters of the control drill rod in the rock drill; if the first target point overlaps with the second target point, determine that the position of the drill rod drilling on the rock is aligned with the hole position. This embodiment uses external image data and point cloud data to observe the drilling operation of the rock drill, visually and tactilely locate the hole position and the drill rod of the rock drill, and the positioning accuracy is high, thereby realizing a method that is independent of the rock drill and supervises the drilling operation of the rock drill, which can effectively track and calibrate the drilling operation of the rock drill, reduce the probability of deviation of the rock drill when drilling, improve the accuracy of drilling, and ensure the progress of the project. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 is a schematic diagram of a hole position calibration method for a rock drill provided in an embodiment of the present application;

[0032] Figure 2 is a schematic diagram of an intelligent rock drilling system architecture provided in an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a semantic segmentation model provided in an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of a hole position verification device for a rock drill provided in an embodiment of the present application;

[0035] Figure 5It is a schematic diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.

[0037] The technical solution of the present application is described below through specific embodiments.

[0038] Reference Figure 1 , shows a schematic diagram of a hole position calibration method for a rock drill provided in an embodiment of the present application, which may specifically include the following steps:

[0039] Step 101: synchronously collect image data and point cloud data from rocks and rock drills at multiple time points.

[0040] This embodiment can be applied to the intelligent rock drilling system architecture, such as Figure 2 As shown in the figure, the intelligent rock drilling system architecture includes a mobile station acquisition system, a positioning system and a display system. The acquisition system is mainly used to collect data generated by the rock drilling operation, which includes a GNSS (Global Navigation Satellite System) antenna, an industrial-grade flat panel, a tilt sensor and an encoder (to monitor the drilling depth). The positioning system provides positioning support for the rock drilling operation, which includes a global satellite positioning system (such as Beidou, GPS (Global Positioning System), GLONASS (Global Orbiting Navigation Satellite System), etc.) and a CORS (Continuously Operating Reference Station) base station, which can support differential correction data. The display system transmits the data generated by the rock drilling operation to the user's (such as construction, supervision, owner) terminal (such as mobile phone, computer, command center, etc.) through the Internet, cloud servers and other channels for display for remote supervision by the user.

[0041] Engineers can configure the operating parameters of the drilling operation in the industrial-grade tablet according to the needs of the rock drilling operation, such as coordinates, hole positions, differential signals, etc.

[0042] In this embodiment, a monitoring device can be set up on the side (left or right) of the rock drill and the rock. A laser radar and a camera are installed in the monitoring device. Reflective sheets matching the laser radar are pasted on the holes set on the rock. The reflective sheets are disposable equipment and have a high reflectivity to the laser signal reflected by the laser radar.

[0043] Multiple time points are set on the time axis, and the time points are sparser than the sampling frequency of the lidar and the sampling frequency of the camera, so that the point cloud data collected by the lidar and the image data collected by the camera can be aligned at the same time point. Then, at multiple time points, the camera can be synchronously driven to collect image data from the rock and the rock drill, and the lidar can be driven to collect point cloud data from the rock and the rock drill.

[0044] The pixel points of the image data have color components (such as RGB (red, green, and blue)), and the laser points of the point cloud data have coordinates (such as X coordinate, Y coordinate, and Z coordinate) and reflection intensity (Intensity). The reflection intensity is related to the surface material and roughness of the rock drill and rock, the laser incident angle, the laser wavelength, and the energy density of the lidar.

[0045] Step 102: Project the laser points of the point cloud data above the ground onto the pixel points of the image data to obtain joint points.

[0046] In the monitoring instrument, the height of the laser radar from the ground is a priori knowledge. Therefore, in the coordinate system of the laser radar, a plane above the ground is set, and the point cloud data located above the plane is filtered out from all the point cloud data. The laser point of the point cloud data is projected onto the pixel point of the image data to obtain the joint point, and the pixel points that are not projected are ignored. At this time, the joint point has both color components and coordinates and reflection intensity.

[0047] Generally, the rock drilling operation of a rock drill takes place on the ground, and the calibration of the rock drilling operation has nothing to do with the ground points. The joint points eliminate the interference of the ground points, which can effectively reduce the amount of calculation and improve the calculation speed.

[0048] Exemplarily, projecting the laser points of the point cloud data to the pixel points of the image data includes the following steps:

[0049] S1. Query the camera parameters: The camera parameters include internal parameters and external parameters. Internal parameters include the focal length, optical center and other parameters of the camera, which are used to describe the imaging model of the camera. External parameters describe the position and posture of the camera in the world coordinate system.

[0050] S2. Convert the laser points in the point cloud data to the camera's coordinate system: The laser points in the point cloud data are usually expressed in the world coordinate system. According to the conversion matrix pre-calibrated for the laser radar and camera in the instrument, convert them to the camera's coordinate system.

[0051] S3. Mapping the coordinates of the laser points of the point cloud data to the coordinates of the pixel points according to the intrinsic parameters of the camera: Projecting the laser points of the point cloud data onto the imaging plane of the camera using the intrinsic parameters of the camera, and mapping the three-dimensional coordinates to the coordinates of the pixel points.

[0052] S4. Mapping the reflection intensity of the laser points of the point cloud data to the image data.

[0053] Step 103: Separate the first target point representing the reflective sheet and the candidate point representing the rock drill from the joint points according to the coordinates and the reflection intensity.

[0054] In practical applications, the reflection intensity of the reflective sheet is usually significantly higher than that of other objects, and the positions of the luminous sheet and the rock drill are relative to each other. Therefore, the first target point representing the reflective sheet and the candidate point representing the rock drill can be synchronously separated in the joint point based on the coordinates and reflection intensity information in the joint point.

[0055] In one embodiment of the present application, step 103 may include the following steps:

[0056] Step 1031: Cluster the joint points using the coordinates to obtain two first point clusters.

[0057] In this embodiment, the point cloud data on the ground mainly represents rocks and rock drills. The point cloud data of rocks and rock drills are relatively concentrated in coordinates. Therefore, clustering algorithms such as K-means (K means) and DBSCAN (Density Based Spatial Clustering of Applications with Noise) can be called to use the coordinates to perform preliminary clustering of the joint points to obtain two first point clusters, one of which mainly represents rocks, and the other first point cluster mainly represents rock drills.

[0058] Of course, in addition to rocks and rock drills, there may be other equipment on site, and the point cloud data of other equipment can be classified into any first point cluster.

[0059] Step 1032: In each first point cluster, cluster the joint points using the reflection intensity and the coordinates to obtain a plurality of second point clusters.

[0060] In practical applications, the performance of the reflective sheet is relatively balanced, so the reflection intensity of its point cloud data is relatively balanced.

[0061] In each first point cluster, clustering algorithms such as K-means and DBSCAN can be called to perform secondary clustering on the joint points in combination with the reflection intensity and coordinates to obtain multiple second point clusters.

[0062] The reflection intensity and coordinates jointly constrain the secondary clustering, so that the joint points with similar coordinates and similar reflection intensities are clustered in the second point cluster.

[0063] In one embodiment of the present application, step 1032 may include the following steps:

[0064] Step 10321: Initialize multiple second point clusters in the first point cluster.

[0065] In this embodiment, there may be crystalline rock masses on the rock, and the reflectivity of these rock masses is also high, which may interfere with the identification of the reflector. Therefore, multiple second point clusters are initialized in the first point cluster, and the number of second point clusters is large to accommodate the interference of the rock masses.

[0066] Each second point cluster has a central point, and the central point can be generated by random or other methods.

[0067] Step 10322: Encode the coordinates and reflection intensity of each joint point into a vector.

[0068] Traverse each joint point, perform vectorization operations on its coordinates and reflection intensity, and encode them together as a vector.

[0069] Step 10323: Calculate the Euclidean distance between the vector of the joint point and each center point.

[0070] Traverse each joint point, calculate the Euclidean distance between its vector and the center point of each second cluster, and measure the similarity between the vector of the joint point and the center point of each second cluster.

[0071] Step 10324: Divide the joint point into the second point cluster with the smallest Euclidean distance.

[0072] Traverse each joint point, compare the Euclidean distance between its vector and the center point of each second cluster, screen out the second point cluster with the smallest Euclidean distance, and divide it into the second point cluster with the smallest Euclidean distance.

[0073] Step 10325: In each second point cluster, assign the average value between the vectors of the joint points as the center point.

[0074] In each second point cluster, the average value between the vectors of all joint points in the current second point cluster is calculated, and the average value is assigned as the center point to achieve the update of the center point.

[0075] Step 10326: In all second point clusters, add the sum of squared errors between the vector of the joint point and the center point to obtain a loss value.

[0076] In each second point cluster, the sum of the squared errors (SSE) between the vectors of all joint points in the current second point cluster and the center point is calculated, and the sum of the squared errors of all second point clusters is added to obtain the loss value of the current iteration.

[0077] Step 10327: determine whether the change between the current loss value and the previous loss value is less than the training threshold; if not, return to step 10323; if so, execute step 10328.

[0078] Take the absolute value of the difference between the loss value of the current iteration and the loss value of the previous iteration to get the change in the loss value between the two iterations.

[0079] The magnitude of the change is compared with a preset training threshold.

[0080] If the change amplitude is greater than or equal to the training threshold, the process returns to step 10323 to step 10326 to enter the next round of iteration.

[0081] If the change amplitude is less than the training threshold, step 10328-step 10329 is ready to be executed to end the secondary clustering.

[0082] Step 10328: In each second point cluster, perform a corrosion operation on the joint points according to the coordinates. When the corrosion operation is completed, calculate the average value of the reflection intensity of the joint points to obtain the average intensity.

[0083] In each second point cluster, the corrosion operation can be performed on the joint points according to the coordinates to eliminate the interference of some noise points. Although the corrosion operation may eliminate some joint points on the edge of the reflector, there is basically no change to the joint points of the reflector as a whole, and it has basically no impact on subsequent operations.

[0084] When the corrosion operation is completed, the average value of the reflection intensities of all the joint points in the same second point cluster can be calculated and recorded as the average intensity.

[0085] Step 10329: If the difference between the average intensities of two adjacent second point clusters is less than the deviation threshold, the two adjacent second point clusters are merged.

[0086] In this embodiment, since the number of second point clusters is large during secondary clustering and the reflective sheet may be bent, the reflection intensity of the joint points of a part of the reflective sheets is slightly higher than the reflection intensity of the joint points of another part of the reflective sheets as a whole, so that the joint points of the same reflective sheet are divided into two or more second point clusters. At this time, a merging operation can be performed on the second point clusters to obtain the joint points of a complete reflective sheet.

[0087] In a specific implementation, two second point clusters whose edges intersect can be recorded as two adjacent second point clusters, and the absolute value of the difference between the average intensities of the two adjacent second point clusters is taken to obtain the difference between the average intensities of the two adjacent second point clusters, and the difference between the average intensities of the two adjacent second point clusters is compared with a preset deviation threshold. If the difference between the average intensities of the two adjacent second point clusters is less than the deviation threshold, the two second point clusters with adjacent positions are merged, and each second point cluster is traversed in this way to obtain the final second point cluster.

[0088] Step 1033: Match the geometric features of the second point cluster with the identification conditions set for the reflective sheet.

[0089] In this embodiment, identification conditions can be set in advance for the reflective sheet in the geometric dimension. The identification conditions are used to identify whether it is a reflective sheet. Geometric features are extracted from the second point cluster, and the geometric features of the second point cluster are matched with the identification conditions set for the reflective sheet. The matching operation of the geometric features is relatively simple, the matching calculation amount is low, and the matching results can be obtained quickly to meet the real-time requirements.

[0090] In a specific implementation, a three-dimensional coordinate system may be constructed; the coordinate system includes an X-axis, a Y-axis, and a Z-axis.

[0091] The second point cluster is sorted in descending order according to the average intensity, that is, the higher the average intensity, the higher the ranking, and vice versa, the lower the average intensity, the lower the ranking.

[0092] Considering that the monitoring instrument is located on the side of the rock and the rock drill, the joint points of the highest-ranked second point clusters can be projected onto the base plane composed of the X-axis and the Z-axis, excluding the second point clusters that obviously belong to the rock mass with lower reflectivity, thereby reducing the amount of calculation.

[0093] In the base plane, a minimum bounding rectangle is added to the joint points in the same second point cluster, and a normalization operation is performed on the minimum bounding rectangle according to the scale of the camera to convert it to a standard scale.

[0094] Since the reflective sheet is a graphic of inherent size such as a circle or square, even if there is a certain degree of distortion when collecting image data and point cloud data, the distortion is relatively limited. Therefore, a first identification range and a second identification range under a standard ratio can be set for the reflective sheet as identification conditions to accommodate the influence of distortion. At this time, the ratio between the length and the width in the minimum circumscribed rectangle can be calculated, and the area of ​​the minimum circumscribed rectangle can be counted as geometric features, and the ratio between the length and the width in the minimum circumscribed rectangle can be compared with the preset first identification range, and the area of ​​the minimum circumscribed rectangle can be compared with the second identification range.

[0095] If the ratio between the length and the width of the minimum circumscribed rectangle is within the first identification range and the area of ​​the minimum circumscribed rectangle is within the second identification range, it is determined that the geometric features of the second point cluster successfully match the identification conditions set for the reflective sheeting.

[0096] If the ratio between the length and the width of the minimum circumscribed rectangle is outside the first identification range and / or the area of ​​the minimum circumscribed rectangle is outside the second identification range, it is determined that the geometric features of the second point cluster fail to match the identification conditions set for the reflective sheeting.

[0097] Step 1034: If the match is successful, determine the second point cluster as the first target point representing the reflective sheet.

[0098] If the geometric features of a second point cluster successfully match the identification conditions set for the reflective sheeting, the joint point of the second point cluster can be identified as the first target point representing the reflective sheeting.

[0099] Step 1035: Using the first target point as a reference, construct a target plane representing the rock according to the first point cluster.

[0100] In this embodiment, the first target point may be used as a reference object, and a target plane representing the rock may be constructed according to the first point cluster to which the first target point belongs.

[0101] In a specific implementation, within the range of the first target point, a plurality of original planes passing through the first target point are constructed.

[0102] Traverse each original plane, in the first point cluster (mainly representing rocks) where the first target point is located, calculate the distance of each joint point when projected to the original plane, and calculate the average value of the distance between each joint point and the original plane to obtain the cluster distance of the original plane.

[0103] The cluster distances of the original planes are compared, the original plane with the smallest cluster distance is screened out, and the original plane with the smallest cluster distance is determined as the target plane representing the rock.

[0104] Step 1036: Taking the first target point as a reference, construct a three-dimensional space that is compatible with the rock drill in front of the target plane.

[0105] Step 1037: Mark the first point cluster located in the three-dimensional space and constituting a connected region as candidate points representing the rock drill.

[0106] According to prior knowledge, during the drilling process, the rock drill (especially the drill rod) is located in front of the hole position (reflector). Therefore, a three-dimensional space adapted to the rock drill (especially the drill rod) can be constructed in front of the target plane based on the first target point, so that the three-dimensional space can include the rock drill (especially the drill rod) as much as possible and exclude interference from other equipment. At this time, the joint point mark of the first point cluster located in the three-dimensional space and forming a connected area (because the rock drill is a whole) can represent the candidate point of the rock drill.

[0107] In a specific implementation, on the target plane, with the first target point as the reference, the length and width of the minimum enclosing rectangle are enlarged (i.e., the length and width of the minimum enclosing rectangle are multiplied by corresponding enlargement factors, which can be calculated based on a standard ratio) to obtain the expanded area.

[0108] With the expansion area as the bottom surface, the specified length (the length can be calculated according to the standard ratio) is extended in front of the target plane to obtain a three-dimensional space adapted to the rock drill.

[0109] Step 104: Perform semantic segmentation on the candidate points according to the color components and the coordinates to obtain a second target point representing the drill rod in the rock drill.

[0110] In practical applications, the color components and coordinates can be used to perform semantic segmentation on the candidate points, and the candidate point with the semantics of a drill rod in a rock drill can be marked as the second target point.

[0111] In one embodiment of the present application, step 104 may include the following steps:

[0112] Step 1041: Load the semantic segmentation network.

[0113] In this embodiment, a semantic segmentation network can be constructed and trained based on deep learning, so that the semantic segmentation network can be used to segment drill rods in a rock drill.

[0114] Among them, the structure of the semantic segmentation network is not limited to artificially designed neural networks, such as PointNet (point network), PointCNN (point convolutional neural network), DGCNN (dynamic graph convolutional neural network), etc., and can also be a neural network optimized by a model quantization method, a neural network searched for the characteristics of a rock drill by a NAS (Neural Architecture Search) method, etc. This embodiment does not impose any restrictions on this.

[0115] like Figure 3As shown in the figure, the semantic segmentation network applies the Seq2Seq (Sequence-to-sequence) architecture, which includes an encoder and a decoder. The encoder is responsible for converting the input sequence into a context vector of a fixed degree, which is usually implemented using CNN (convolutional neural network), RNN (recurrent neural network) and their variants. The decoder is responsible for generating an output sequence from the context vector, which is usually implemented using CNN (convolutional neural network), RNN (recurrent neural network) and their variants.

[0116] Step 1042: Convert the color components and coordinates into original features of the candidate points.

[0117] In this embodiment, each candidate point is traversed, and vectorization is performed on its color components and coordinates to convert them into original features of the candidate point.

[0118] Step 1043: Input the original features of the candidate points into the encoder and perform encoding operations at multiple scales to obtain candidate features.

[0119] In this embodiment, the original features of the candidate points are input into the encoder to perform encoding operations at multiple scales to obtain candidate features. In this process, a larger receptive field can be obtained, thereby extracting more contextual information, thereby improving the performance of the semantic segmentation network.

[0120] For example, Figure 3 As shown, the encoder Encoder includes a first convolutional layer Conv1, a second convolutional layer Conv2, a third convolutional layer Conv3, a fourth convolutional layer Conv4, a fifth convolutional layer Conv5, a sixth convolutional layer Conv6, a seventh convolutional layer Conv7, a first multi-head attention mechanism layer MHA1, a second multi-head attention mechanism layer MHA2 and a third multi-head attention mechanism layer MHA3.

[0121] Among them, the first convolution layer Conv1, the second convolution layer Conv2, the third convolution layer Conv3, the fourth convolution layer Conv4, the fifth convolution layer Conv5, the sixth convolution layer Conv6 and the seventh convolution layer Conv7 are all convolution layers, which can provide convolution operations, BN (BatchNormlization), ReLU (Rectified Linear Unit) and other processing.

[0122] The parameters (such as convolution kernel, step size, etc.) of the third convolution layer Conv3, the fourth convolution layer Conv4 and the fifth convolution layer Conv5 are different, forming multiple scales.

[0123] The first multi-head attention mechanism layer MHA1, the second multi-head attention mechanism layer MHA2 and the third multi-head attention mechanism layer MHA3 all apply the multi-head attention mechanism, which can independently learn h groups (such as h = 8) of different linear projections to transform the query, key and value. These h groups of transformed queries, keys and values ​​will be sent to the attention pool in parallel, and the outputs of these h attention pools will be spliced ​​together and transformed through another learnable linear projection to produce the final output. In this way, it can increase the potential capture of multiple semantic associations in the sequence more comprehensively, pay attention to the position, shape, color and other aspects of each component of the rock drill, and improve the learning ability and expressiveness of the semantic segmentation model.

[0124] Accordingly, the candidate features include a first coding feature, a second coding feature, a third coding feature, a fourth coding feature, a fifth coding feature and a sixth coding feature.

[0125] Then, in this example, the original features of the candidate points are sequentially input into the first convolutional layer Conv1 and the second convolutional layer Conv2 for processing to obtain the first encoded features.

[0126] Split the first encoded feature into three outputs:

[0127] First path: The first encoded feature is processed by the third convolutional layer Conv3 and the first multi-head attention layer MHA1 in sequence to obtain the second encoded feature.

[0128] The second path: the first encoded feature is processed by the fourth convolutional layer Conv4 and the second multi-head attention layer MHA2 in sequence to obtain the third encoded feature.

[0129] The third path: The first encoded feature is processed by the fifth convolutional layer Conv5 and the third multi-head attention layer MHA3 in sequence to obtain the fourth encoded feature.

[0130] The second coding feature, the third coding feature and the fourth coding feature are concatenated into a fifth coding feature.

[0131] The fifth coding feature is sequentially input into the sixth convolutional layer Conv6 and the seventh convolutional layer Conv7 for processing to obtain the sixth coding feature.

[0132] Step 1044: Input the candidate features into a decoder to perform a decoding operation, and obtain a second target point whose semantics is a drill rod in a rock drill.

[0133] In this embodiment, the candidate features are input into a decoder to perform a decoding operation, and a candidate point having the semantics of a drill rod in a rock drill is obtained as the second target point.

[0134] For example, Figure 3 As shown, the decoder Decoder includes a first deconvolution layer Deconv1, a second deconvolution layer Deconv2, a third deconvolution layer Deconv3, a fourth deconvolution layer Deconv4 and a fully connected layer (Fully Connected Layer, FC).

[0135] Among them, the first deconvolution layer Deconv1, the second deconvolution layer Deconv2, the third deconvolution layer Deconv3 and the fourth deconvolution layer Deconv4 are all deconvolution layers. The deconvolution layer is also called a transposed convolution layer, which can expand the size of the feature and rotate the convolution kernel 180 degrees for convolution calculation.

[0136] In this example, the first encoded feature is sequentially input into the first deconvolution layer Deconv1 and the second deconvolution layer Deconv2 for processing to obtain the first decoded feature. At this time, a jump connection is formed to increase the depth of the lightweight semantic segmentation network and improve the performance of the semantic segmentation network.

[0137] Concatenate the first decoded feature and the sixth encoded feature into a second decoded feature.

[0138] The second decoding feature is sequentially input into the third deconvolution layer Deconv3 and the fourth deconvolution layer Deconv4 for processing to obtain the third decoding feature.

[0139] The third decoded feature is input into the fully connected layer FC and mapped into the semantics of the candidate point, which includes the drill rod and others in the rock drill.

[0140] The candidate point with the semantics of the drill rod in the rock drill is selected from all the candidate points as the second target point.

[0141] In the case where the candidate points representing the rock drill have been separated, a lightweight semantic segmentation network with a simpler structure can be selected in the example to reduce the amount of computation, improve the response speed, and meet the needs of real-time monitoring of the rock drilling operation of the rock drill.

[0142] Step 105: Mark the current time point as a first monitoring point, and mark the time point after the first monitoring point as a second monitoring point.

[0143] In this embodiment, the rock drilling operation of the rock drill can be monitored in real time in a cycle. At the initialization of each cycle, the current time point is marked as the first monitoring point, and multiple time points after the first monitoring point are marked as second monitoring points.

[0144] Step 106: At the first monitoring point, plan a guided drilling path from the second target point to the first target point.

[0145] At the first monitoring point, according to the degrees of freedom of the rock drill's robotic arm, a path pointing from the second target point (especially the end) to the first target point (especially the center point) that meets the movement specifications of the rock drill's robotic arm can be planned and recorded as the guided drilling path.

[0146] Step 107: Generate a real-time drilling path at multiple second monitoring points using multiple second target points.

[0147] At the multiple second monitoring points, a polynomial function, a linear function or the like is used to sequentially connect the corresponding multiple second target points (especially the ends) to obtain a real-time drilling path.

[0148] Through the antenna installed on the rock drill, real-time reception of positioning information can be achieved, and the positioning information can be solved, the data can be verified and filtered in conjunction with the second target point to form a real-time drilling path and output three-dimensional coordinates in real time. By importing design data, drilling navigation can be achieved to guide engineers in drilling construction operations.

[0149] At this time, the information of each hole position (position deviation, drilling depth, drilling speed, drilling time, inclination) can be recorded to facilitate the retrospective statistical analysis of the historical working information of the rock drill.

[0150] Step 108 : If the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, an alarm operation is performed, and an attenuation operation is performed on the parameters controlling the drill rod in the rock drill, and the process returns to step 105 .

[0151] In this embodiment, the deviation between the real-time drilling path and the guided drilling path in terms of angle, distance and other factors may be calculated, and the deviation between the real-time drilling path and the guided drilling path may be compared with a preset drilling threshold.

[0152] If the deviation between the real-time drilling path and the guided drilling path is less than or equal to the drilling threshold, it means that the deviation between the real-time drilling path and the guided drilling path is small, and the real-time drilling path basically conforms to the guided drilling path, and the rock drilling operation of the rock drill continues to be monitored in real time.

[0153] If the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, it means that the deviation between the real-time drilling path and the guided drilling path is large. At this time, an alarm operation is performed, and the corresponding alarm information is displayed on the industrial-grade flat panel and display system to prompt the engineer to pay attention to controlling the drill rod in the rock drill. When N (N is a positive integer, such as 3) alarm operations are accumulated, an attenuation operation can be performed on the parameters of the control drill rod in the rock drill to reduce the movement amplitude of the drill rod in the rock drill, provide a certain margin for adjusting the control drill rod in the rock drill, reduce the probability of the drilling position deviating from the hole position, and return to execute step 105 to enter the next cycle of real-time monitoring of the rock drilling operation of the rock drill.

[0154] In the specific implementation, the rock drill mainly uses the PID (Proportional Integral Derivative) mechanism to control the drill rod. Then, the parameters for controlling the drill rod in the rock drill are mainly proportional parameters, integral parameters and differential parameters. Then, the attenuation operation can be expressed as follows:

[0155] P t+1 =P t -△P

[0156] I t+1 =I t -△P

[0157] D t+1 =D t -△P

[0158] Among them, P t+1 is the proportional parameter at time point (t+1), P t is the proportional parameter at time point t, △P is the step size for adjusting the proportional parameter, I t+1 is the integral parameter at time point (t+1), I t is the integral parameter at time point t, △I is the step size for adjusting the integral parameter, D t+1 is the differential parameter at time point (t+1), D t is the differential parameter at time point t, and △D is the step size for adjusting the differential parameter.

[0159] Generally speaking, △P∈∈[-2, 2], △I∈∈[-0.05, 0.05], △D∈∈[-0.2, 0.2].

[0160] Step 109: If the first target point overlaps with the second target point, it is determined that the position of the hole drilled by the drill rod on the rock is aligned with the hole position.

[0161] If the first target point (especially the center point) is initially observed to overlap with the second target point (especially the end), it can be determined that during the initial drilling, the position of the drill rod in the rock is aligned with the hole position set on the rock without deviation.

[0162] At this time, the reflective sheet will be penetrated by the drill rod of the rock drill, but its first target point has been recorded. Subsequently, the rock drilling operation of the rock drill can be monitored in real time based on the first target point in the historical record. When the first target point in the historical record overlaps with the second target point, it is determined that during the drilling process, the position of the hole drilled by the drill rod on the rock is aligned with the hole position. When the first target point in the historical record does not overlap with the second target point, it is determined that during the drilling process, the position of the hole drilled by the drill rod on the rock deviates from the hole position. At this time, an alarm operation is executed, and the corresponding alarm information is displayed on the industrial-grade flat panel and display system, prompting engineers to pay attention to controlling the drill rod in the rock drill.

[0163] In this embodiment, image data and point cloud data are synchronously collected from the rock and the rock drill at multiple time points; reflective sheets are pasted on the set hole positions on the rock; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity; the laser points of the point cloud data above the ground are projected onto the pixel points of the image data to obtain joint points; the first target point representing the reflective sheet and the candidate point representing the rock drill are separated from the joint point based on the coordinates and the reflection intensity; semantic segmentation is performed on the candidate points based on the color components and the coordinates to obtain the second target point representing the drill rod in the rock drill. point; mark the current time point as the first monitoring point, and mark the time point after the first monitoring point as the second monitoring point; at the first monitoring point, plan a guided drilling path from the second target point to the first target point; at multiple second monitoring points, use multiple second target points to generate a real-time drilling path; if the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, perform an alarm operation, and perform an attenuation operation on the parameters of the control drill rod in the rock drill; if the first target point overlaps with the second target point, determine that the position of the drill rod drilling on the rock is aligned with the hole position. This embodiment uses external image data and point cloud data to observe the drilling operation of the rock drill, visually and tactilely locate the hole position and the drill rod of the rock drill, and the positioning accuracy is high, thereby realizing a method that is independent of the rock drill and supervises the drilling operation of the rock drill, which can effectively track and calibrate the drilling operation of the rock drill, reduce the probability of deviation of the rock drill when drilling, improve the accuracy of drilling, and ensure the progress of the project.

[0164] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] Reference Figure 4, shows a schematic diagram of a hole position verification device for a rock drill provided in an embodiment of the present application, which may specifically include the following modules:

[0166] The sensing module 41 is used to synchronously collect image data and point cloud data from the rock and the rock drill at multiple time points; the hole positions set on the rock are pasted with reflective sheets; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity;

[0167] A point cloud projection module 42, used to project the laser points of the point cloud data above the ground onto the pixel points of the image data to obtain joint points;

[0168] A joint point separation module 43, used to separate a first target point representing the reflective sheet and a candidate point representing the rock drill from the joint points according to the coordinates and the reflection intensity;

[0169] A semantic segmentation module 44 is used to perform semantic segmentation on the candidate point according to the color component and the coordinates to obtain a second target point representing the drill rod in the rock drill;

[0170] A time point marking module 45, configured to mark the current time point as a first monitoring point, and mark the time point after the first monitoring point as a second monitoring point;

[0171] A guided drilling path planning module 46 is used to plan a guided drilling path from the second target point to the first target point at the first monitoring point;

[0172] A real-time drilling path generating module 47, configured to generate a real-time drilling path at a plurality of the second monitoring points using a plurality of the second target points;

[0173] a deviation processing module 48, configured to, if the deviation between the real-time drilling path and the guided drilling path is greater than a drilling threshold, perform an alarm operation, and perform an attenuation operation on a parameter of the rock drill for controlling the drill rod, and return to execute the time point marking module 45;

[0174] The alignment determination module 49 is used to determine that the position of the hole drilled by the drill rod on the rock is aligned with the hole position if the first target point overlaps with the second target point.

[0175] In one embodiment of the present application, the joint point separation module 43 includes:

[0176] A first clustering module, used for clustering the joint points using the coordinates to obtain two first point clusters;

[0177] a second clustering module, configured to cluster the joint points in each of the first point clusters using the reflection intensity and the coordinates to obtain a plurality of second point clusters;

[0178] An identification condition matching module, used for matching the geometric features of the second point cluster with the identification conditions set for the reflective sheet;

[0179] a target point determination module, configured to determine the second point cluster as a first target point representing the reflective sheet if the match is successful;

[0180] A target plane construction module, used to construct a target plane representing the rock according to the first point cluster, taking the first target point as a reference;

[0181] A three-dimensional space construction module, used to construct a three-dimensional space adapted to the rock drill in front of the target plane based on the first target point;

[0182] The candidate point marking module is used to mark the first point clusters located in the three-dimensional space and forming a connected area to represent candidate points of the rock drill.

[0183] In one embodiment of the present application, the second clustering module is further used for:

[0184] Initializing a plurality of second point clusters in the first point cluster; each of the second point clusters has a center point;

[0185] Encoding the coordinates and the reflection intensity of each of the joint points into a vector;

[0186] Calculating the Euclidean distance between the vector of the joint point and each of the central points;

[0187] Dividing the joint point into the second point cluster with the smallest Euclidean distance;

[0188] In each of the second point clusters, assigning the average value between the vectors of the joint points as the central point;

[0189] In all the second point clusters, summing the squares of the errors between the vector of the joint point and the central point to obtain a loss value;

[0190] Determine whether the change between the current loss value and the previous loss value is less than the training threshold;

[0191] If not, returning to execute the step of calculating the Euclidean distance between the vector of the joint point and each of the center points;

[0192] If yes, then in each of the second point clusters, an erosion operation is performed on the joint point according to the coordinates, and when the erosion operation is completed, an average value of the reflection intensities of the joint points is calculated to obtain an average intensity;

[0193] If the difference between the average intensities of two adjacent second point clusters is less than a deviation threshold, the two adjacent second point clusters are merged.

[0194] In one embodiment of the present application, the identification condition matching module is further used to:

[0195] Constructing a coordinate system; the coordinate system includes an X-axis, a Y-axis and a Z-axis;

[0196] sorting the second point clusters in descending order according to the average intensity;

[0197] Projecting the joint points in the plurality of the second point clusters with the highest ranking onto a base plane formed by an X-axis and a Z-axis;

[0198] In the base plane, adding a minimum circumscribed rectangle to the joint points in the same second point cluster;

[0199] If the ratio between the length and the width of the minimum circumscribed rectangle is within the first identification range and the area of ​​the minimum circumscribed rectangle is within the second identification range, it is determined that the geometric features of the second point cluster successfully match the identification conditions set for the reflective sheeting.

[0200] In one embodiment of the present application, the target plane construction module is further used to:

[0201] constructing a plurality of original planes passing through the first target point;

[0202] In the first point cluster where the first target point is located, calculating an average value of distances between each of the joint points and the original plane to obtain a cluster distance;

[0203] The original plane with the smallest cluster distance is determined as the target plane representing the rock.

[0204] In one embodiment of the present application, the three-dimensional space construction module is also used for:

[0205] On the target plane, taking the first target point as a reference, enlarging the length and width of the minimum circumscribed rectangle to obtain an extended area;

[0206] Taking the expansion area as the bottom surface, a specified length is extended forward of the target plane to obtain a three-dimensional space adapted to the rock drill.

[0207] In one embodiment of the present application, the semantic segmentation module 44 includes:

[0208] A semantic segmentation network loading module, used to load a semantic segmentation network; the semantic segmentation network includes an encoder and a decoder;

[0209] An original feature conversion module, used for converting the color component and the coordinate into the original feature of the candidate point;

[0210] An encoding module, used for inputting the original features of the candidate points into the encoder to perform encoding operations at multiple scales to obtain candidate features;

[0211] A decoding module is used to input the candidate feature into the decoder to perform a decoding operation to obtain a second target point whose semantics is the drill rod in the rock drill.

[0212] In one embodiment of the present application, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, a first multi-head attention mechanism layer, a second multi-head attention mechanism layer and a third multi-head attention mechanism layer;

[0213] The decoder comprises a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, a fourth deconvolution layer and a fully connected layer;

[0214] The candidate features include a first coding feature, a second coding feature, a third coding feature, a fourth coding feature, a fifth coding feature and a sixth coding feature;

[0215] The encoding module is also used for:

[0216] Inputting the original features of the candidate points into the first convolution layer and the second convolution layer in sequence for processing to obtain a first encoding feature;

[0217] Processing the first coding feature in turn through the third convolutional layer and the first multi-head attention layer to obtain a second coding feature;

[0218] Processing the first coding feature in turn through the fourth convolutional layer and the second multi-head attention layer to obtain a third coding feature;

[0219] Processing the first coding feature in turn through the fifth convolutional layer and the third multi-head attention layer to obtain a fourth coding feature;

[0220] splicing the second coding feature, the third coding feature and the fourth coding feature into a fifth coding feature;

[0221] Inputting the fifth coding feature into the sixth convolutional layer and the seventh convolutional layer in sequence for processing to obtain a sixth coding feature;

[0222] The decoding module is also used for:

[0223] Inputting the first encoding feature into the first deconvolution layer and the second deconvolution layer in sequence for processing to obtain a first decoding feature;

[0224] concatenating the first decoding feature and the sixth encoding feature into a second decoding feature;

[0225] Inputting the second decoding feature into the third deconvolution layer and the fourth deconvolution layer in sequence for processing to obtain a third decoding feature;

[0226] Inputting the third decoded feature into the fully connected layer to map it into the semantics of the candidate point;

[0227] The candidate point with the semantics of the drill rod in the rock drill is screened out as the second target point.

[0228] An embodiment of the present application provides a hole position calibration device for a rock drill. By using the device, each step in the aforementioned method embodiments can be implemented.

[0229] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0230] Reference Figure 5 , shows a schematic diagram of a terminal device provided in an embodiment of the present application. Figure 5 As shown, the terminal device 400 in the embodiment of the present application includes: a processor 410, a memory 420, and a computer program 421 stored in the memory 420 and executable on the processor 410. When the processor 410 executes the computer program 421, the steps in each embodiment of the hole position verification method of the rock drill are implemented. Alternatively, when the processor 410 executes the computer program 421, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0231] Exemplarily, the computer program 421 may be divided into one or more modules / units, which are stored in the memory 420 and executed by the processor 410 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which may be used to describe the execution process of the computer program 421 in the terminal device 400.

[0232] The terminal device 400 may include, but is not limited to, a processor 410 and a memory 420. Those skilled in the art will appreciate that Figure 5 It is only an example of the terminal device 400 and does not constitute a limitation on the terminal device 400. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 400 may also include input and output devices, network access devices, buses, etc.

[0233] The processor 410 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0234] The memory 420 may be an internal storage unit of the terminal device 400, such as a hard disk or memory of the terminal device 400. The memory 420 may also be an external storage device of the terminal device 400, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 400. Further, the memory 420 may also include both an internal storage unit of the terminal device 400 and an external storage device. The memory 420 is used to store the computer program 421 and other programs and data required by the terminal device 400. The memory 420 may also be used to temporarily store data that has been output or is to be output.

[0235] An embodiment of the present application also discloses a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hole position verification method for the rock drill as described in the aforementioned embodiments is implemented.

[0236] The embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the hole position verification method of the rock drill as described in the above-mentioned embodiments is implemented.

[0237] The embodiment of the present application further discloses a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the hole position verification method of the rock drill described in the above-mentioned embodiments.

[0238] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for checking the hole position of a rock drill, characterized in that: include: Synchronously collect image data and point cloud data from the rock and the rock drill at multiple time points; The hole positions set on the rock are pasted with reflective sheets; the pixel points of the image data have color components, and the laser points of the point cloud data have coordinates and reflection intensity; Projecting the laser points of the point cloud data above the ground onto the pixel points of the image data to obtain joint points; Separating a first target point representing the reflective sheet and a candidate point representing the rock drill from the joint points according to the coordinates and the reflection intensity; Performing semantic segmentation on the candidate point according to the color component and the coordinates to obtain a second target point representing a drill rod in the rock drill; Mark the current time point as a first monitoring point, and mark the time point after the first monitoring point as a second monitoring point; At the first monitoring point, planning a guided drilling path from the second target point to the first target point; At the plurality of the second monitoring points, generating a real-time drilling path using the plurality of the second target points; If the deviation between the real-time drilling path and the guided drilling path is greater than the drilling threshold, an alarm operation is performed, and a decay operation is performed on the parameter of the rock drill that controls the drill rod, and the process of marking the current time point as a first monitoring point and marking the time point after the first monitoring point as a second monitoring point is returned to execution; If the first target point overlaps with the second target point, it is determined that the position of the hole drilled by the drill rod on the rock is aligned with the hole position.

2. The method according to claim 1, characterized in that The step of separating the first target point representing the reflective sheet and the candidate point representing the rock drill from the joint point according to the coordinates and the reflection intensity comprises: Clustering the joint points using the coordinates to obtain two first point clusters; In each of the first point clusters, clustering the joint points using the reflection intensity and the coordinates to obtain a plurality of second point clusters; Matching the geometric features of the second point cluster with the identification conditions set for the reflective sheet; If the match is successful, determining the second point cluster as the first target point representing the reflective sheet; Taking the first target point as a reference, constructing a target plane representing the rock according to the first point cluster; Taking the first target point as a reference, constructing a three-dimensional space adapted to the rock drill in front of the target plane; The first point clusters located in the three-dimensional space and constituting a connected area are marked as candidate points representing the rock drill.

3. The method according to claim 2, characterized in that The method of clustering the joint points in the first point cluster using the reflection intensity and the coordinates to obtain a plurality of second point clusters includes: Initializing a plurality of second point clusters in the first point cluster; each of the second point clusters has a center point; Encoding the coordinates and the reflection intensity of each of the joint points into a vector; Calculating the Euclidean distance between the vector of the joint point and each of the central points; Dividing the joint point into the second point cluster with the smallest Euclidean distance; In each of the second point clusters, assigning the average value between the vectors of the joint points as the central point; In all the second point clusters, summing the squares of the errors between the vector of the joint point and the central point to obtain a loss value; Determine whether the change between the current loss value and the previous loss value is less than the training threshold; If not, returning to execute the step of calculating the Euclidean distance between the vector of the joint point and each of the center points; If yes, then in each of the second point clusters, an erosion operation is performed on the joint point according to the coordinates, and when the erosion operation is completed, an average value of the reflection intensities of the joint points is calculated to obtain an average intensity; If the difference between the average intensities of two adjacent second point clusters is less than a deviation threshold, the two adjacent second point clusters are merged.

4. The method according to claim 3, characterized in that The step of matching the geometric features of the second point cluster with the identification conditions set for the reflective sheet includes: Constructing a coordinate system; the coordinate system includes an X-axis, a Y-axis and a Z-axis; sorting the second point clusters in descending order according to the average intensity; Projecting the joint points of the plurality of the second point clusters with the highest ranking onto a base plane formed by an X-axis and a Z-axis; In the base plane, adding a minimum circumscribed rectangle to the joint points in the same second point cluster; If the ratio between the length and the width of the minimum circumscribed rectangle is within the first identification range and the area of ​​the minimum circumscribed rectangle is within the second identification range, it is determined that the geometric features of the second point cluster successfully match the identification conditions set for the reflective sheeting.

5. The method according to claim 4, characterized in that The step of constructing a target plane representing the rock based on the first point cluster and taking the first target point as a reference includes: constructing a plurality of original planes passing through the first target point; In the first point cluster where the first target point is located, calculating an average value of distances between each of the joint points and the original plane to obtain a cluster distance; The original plane with the smallest cluster distance is determined as the target plane representing the rock.

6. The method according to claim 4, characterized in that The step of constructing a three-dimensional space adapted to the rock drill in front of the target plane based on the first target point includes: On the target plane, taking the first target point as a reference, enlarging the length and width of the minimum circumscribed rectangle to obtain an extended area; Taking the expansion area as the bottom surface, a specified length is extended forward of the target plane to obtain a three-dimensional space adapted to the rock drill.

7. The method according to any one of claims 1 to 6, characterized in that The performing semantic segmentation on the candidate point according to the color component and the coordinate to obtain a second target point representing the drill rod in the rock drill includes: Loading a semantic segmentation network; the semantic segmentation network includes an encoder and a decoder; Convert the color components and the coordinates into original features of the candidate points; Inputting the original features of the candidate points into the encoder to perform encoding operations at multiple scales to obtain candidate features; The candidate features are input into the decoder to perform a decoding operation, and a second target point having the semantics of a drill rod in the rock drill is obtained.

8. The method according to claim 7, characterized in that The encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, a first multi-head attention mechanism layer, a second multi-head attention mechanism layer and a third multi-head attention mechanism layer; The decoder comprises a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, a fourth deconvolution layer and a fully connected layer; The candidate features include a first coding feature, a second coding feature, a third coding feature, a fourth coding feature, a fifth coding feature and a sixth coding feature; The step of inputting the original features of the candidate points into the encoder and performing encoding operations at multiple scales to obtain candidate features includes: Inputting the original features of the candidate points into the first convolution layer and the second convolution layer in sequence for processing to obtain a first encoding feature; Processing the first coding feature in turn through the third convolutional layer and the first multi-head attention layer to obtain a second coding feature; Processing the first coding feature in turn through the fourth convolutional layer and the second multi-head attention layer to obtain a third coding feature; Processing the first coding feature in turn through the fifth convolutional layer and the third multi-head attention layer to obtain a fourth coding feature; splicing the second coding feature, the third coding feature and the fourth coding feature into a fifth coding feature; Inputting the fifth coding feature into the sixth convolutional layer and the seventh convolutional layer in sequence for processing to obtain a sixth coding feature; The step of inputting the candidate feature into the decoder to perform a decoding operation to obtain a second target point having a semantic meaning of a drill rod in the rock drill includes: Inputting the first encoding feature into the first deconvolution layer and the second deconvolution layer in sequence for processing to obtain a first decoding feature; concatenating the first decoding feature and the sixth encoding feature into a second decoding feature; Inputting the second decoding feature into the third deconvolution layer and the fourth deconvolution layer in sequence for processing to obtain a third decoding feature; Inputting the third decoded feature into the fully connected layer to map it into the semantics of the candidate point; The candidate point with the semantics of the drill rod in the rock drill is screened out as the second target point.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the hole position verification method for the rock drill according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the hole position verification method for a rock drill according to any one of claims 1 to 8 is implemented.

Images