Anchor hole identifying and positioning system and method, anchor protection equipment and storage medium

Abstract

The invention relates to an anchor hole recognition and positioning system and method, anchor protection equipment and a storage medium. The anchor hole recognition and positioning system comprises a visual sensor for collecting original image information within preset time, a laser radar for obtaining original point cloud information within the preset time and an image processor. The image processor comprises a first processing module and a second processing module; the first processing module processes the original image information based on an adaptive gamma correction algorithm and a defogging algorithm to obtain updated image information, and determines a two-dimensional target pixel area according to the updated image information and an anchor hole recognition model obtained by preset training; the second processing module processes the original point cloud information based on a pre-constructed first joint calibration relation between the visual sensor and the laser radar to obtain a three-dimensional target area; and then processing the three-dimensional target area to obtain an anchor hole position and an anchor hole center. The anchor hole identification precision can be improved.

Description

Technical Field

[0001] This invention belongs to the field of automatic borehole positioning technology, specifically relating to an anchor hole identification and positioning system, method, anchoring equipment, and storage medium. Background Technology

[0002] Currently, the intelligent development of fully mechanized mining faces is rapid, while the intelligent development of tunneling faces is slow. This is mainly because, with the increase in coal mining depth, geological conditions are becoming increasingly complex, and roadway support is becoming increasingly important. Coal mine roadway support has undergone a long process, evolving from timber support, masonry arch support, and steel support to bolt support. Among these, bolt support involves drilling holes in the surrounding rock after roadway excavation and then inserting bolts into the holes. Essentially, it alters the mechanical state of the surrounding rock through the bolts inside, utilizing the combined action of the bolts and the surrounding rock to maintain roadway stability.

[0003] The main implementation schemes of anchor bolt support:

[0004] 1) Manual method: In coal mines, the anchor bolt support operation is carried out by workers who control the anchor bolt drilling machine to continuously adjust the angle and posture for hole positioning; the operation procedure of anchor support is complex, and the manual method is not only inefficient and labor-intensive, but also has a harsh working environment and the personal safety of workers is difficult to guarantee.

[0005] 2) Visual Recognition: Most existing visual recognition solutions use monocular vision sensors. Based on the data collected by the vision sensor, dehazing and enhancement processing are performed to obtain an enhanced image. Then, based on an anchor hole recognition model, the anchor hole bounding boxes in the enhanced image are identified to determine the drilled anchor holes and confirm their center position information. Currently, anchor hole recognition based solely on vision sensors is not very accurate.

[0006] Therefore, it is urgent to solve the problems of low manual efficiency and low anchor hole recognition accuracy caused by using only visual sensors.

[0007] The above content is only for understanding the technical solution of the present invention and does not mean that the above technical content is prior art. Summary of the Invention

[0008] The purpose of this invention is to provide an anchor hole identification and positioning system, method, anchoring equipment, and storage medium to solve the problem of low anchor hole identification accuracy in the prior art.

[0009] To solve the above problems, the anchor hole identification and positioning system involved in this invention adopts the following technical solution:

[0010] Anchor hole identification and positioning system includes a vision sensor, LiDAR, and image processor;

[0011] The visual sensor is used to acquire raw image information at a preset time, and the image information covers the anchor hole area and the first outer contour area of ​​the anchor hole.

[0012] The lidar is used to acquire raw point cloud information at the preset time, and the raw point cloud information covers the anchor hole area and the second outer contour area of ​​the anchor hole; the second outer contour area includes the first outer contour area.

[0013] The image processor includes a first processing module and a second processing module;

[0014] The first processing module is used to perform dual enhancement processing on the original image information based on the adaptive gamma correction algorithm and the dehazing algorithm to obtain updated image information, and to determine the two-dimensional target pixel region based on the updated image information and the anchor hole recognition model obtained by preset training.

[0015] The second processing module is used to process the original point cloud information based on a pre-built first joint calibration relationship between the visual sensor and the lidar to obtain updated point cloud information, and to process the updated point cloud information based on the two-dimensional target pixel information to obtain a three-dimensional target region; then, the three-dimensional target region is processed to obtain the anchor hole position and its center.

[0016] In some embodiments, the step of performing dual enhancement processing on the original image information based on an adaptive gamma correction algorithm and a dehazing algorithm to obtain updated image information includes:

[0017] Obtain the image brightness data and illumination component data of each pixel in the original image information;

[0018] The image brightness data and illumination component data of each pixel are processed using an adaptive gamma correction algorithm to obtain corrected image information;

[0019] The corrected image information is processed using a preset dehazing algorithm to obtain the updated image information.

[0020] In some embodiments, the step of processing the image brightness data and illumination component data of each pixel using an adaptive gamma correction algorithm to obtain corrected image information includes:

[0021] The average image brightness was calculated by using regression analysis to analyze the illumination component data of all pixels.

[0022] The average brightness of the image and the illumination component data of each pixel are processed to obtain the correction parameters for each pixel;

[0023] Corrected image information is obtained by correcting the image brightness data of the corresponding pixel based on the correction parameters of each pixel.

[0024] In some embodiments, determining the two-dimensional target pixel region based on the updated image information and the anchor hole recognition model trained in a preset manner includes:

[0025] The updated image information is input into the pre-trained anchor hole recognition model to obtain the confidence score and location information of multiple two-dimensional pixel regions.

[0026] The confidence scores corresponding to each two-dimensional pixel region are compared and processed according to a preset confidence threshold to determine the two-dimensional target pixel region.

[0027] In some embodiments, processing the original point cloud information based on a pre-built first joint calibration relationship between the visual sensor and the lidar to obtain updated point cloud information includes:

[0028] Obtain the rotation matrix and translation vector between the position information of the visual sensor and the position information of the lidar;

[0029] A first joint calibration relationship between the visual sensor and the lidar is constructed based on the rotation matrix and the translation vector.

[0030] Based on the first joint calibration relationship, the original point cloud information is transformed to obtain updated point cloud information, and the updated point cloud information is in the same coordinate system as the image sensor.

[0031] In some embodiments, processing the three-dimensional target region to obtain the anchor hole location and its center includes:

[0032] The random sample consensus algorithm is used to perform an optimal planar fitting model on the three-dimensional target region.

[0033] The edge contour extraction method based on normal estimation is used to process the optimal plane fitting model to obtain the anchor hole contour;

[0034] The anchor hole position is determined based on the coordinate information of the anchor hole profile; and the anchor hole center is obtained by processing the anchor hole profile using the perpendicular bisector theorem.

[0035] To address the above problems, the present invention provides an anchor hole identification and positioning method, comprising:

[0036] At a preset time, acquire the raw image information collected by the visual sensor and the raw point cloud information collected by the lidar;

[0037] The original image information is enhanced by a dual enhancement process based on an adaptive gamma correction algorithm and a dehazing algorithm to obtain updated image information. The two-dimensional target pixel region is then determined based on the updated image information and the anchor hole recognition model obtained by a preset training.

[0038] The original point cloud information and the two-dimensional target pixel region are processed based on the first joint calibration relationship between the pre-built visual sensor and the lidar to obtain the three-dimensional target region;

[0039] The three-dimensional target region is processed to obtain the anchor hole position and the anchor hole center position.

[0040] To solve the above problems, the present invention provides an anchor arm control method, comprising:

[0041] The anchor hole identification and positioning method described in claim 7 is used to obtain the anchor hole position and the anchor hole center position;

[0042] Based on a pre-determined second joint calibration relationship between the visual sensor and the anchor arm, the anchor hole center position is transformed to obtain the anchor hole center coordinates; the anchor hole center coordinates and the anchor arm coordinates are in the same coordinate system;

[0043] The anchor arm is controlled to align with the center of the anchor hole based on the coordinates of the anchor hole center.

[0044] To solve the above problems, the present invention relates to an anchoring device, which includes an anchor arm, a processor, and a controller;

[0045] The processor uses the above-described anchor hole identification and positioning method to obtain the anchor hole position and its center position; and based on the pre-determined second joint calibration relationship between the visual sensor and the anchor arm, it performs a conversion process on the anchor hole center position to obtain the anchor hole center coordinates;

[0046] The controller controls the anchor arm to align with the center of the anchor hole based on the center coordinates of the anchor hole.

[0047] To address the aforementioned problems, the present invention relates to a computer-readable storage medium storing a computer program, which, when executed on a processor, implements the aforementioned anchor hole identification and positioning method or the aforementioned anchor arm control method.

[0048] The beneficial effects of this invention are as follows:

[0049] This application achieves stable identification and localization of small anchor holes by fusing feature information from image information acquired by a visual sensor and point cloud information acquired by a lidar at the same time, thereby improving the efficiency and accuracy of anchor hole identification. Specifically, the original image information is dually enhanced using an adaptive gamma correction algorithm and a dehazing algorithm, achieving dual enhancement of brightness and contrast. This improves the overall brightness of the original image information while highlighting local details, thus improving imaging quality under low-light conditions in mines. Simultaneously, the process of processing updated point cloud information based on two-dimensional target pixel information to obtain the three-dimensional target region involves mapping the two-dimensional target pixel region onto the updated point cloud information. The anchor hole region and the first outer contour region covered by the updated point cloud information are then extracted to obtain the three-dimensional target region. This processing effectively reduces the massive amount of original point cloud data by an order of magnitude, significantly reducing the computational burden of subsequent point cloud processing. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below:

[0051] Figure 1 This is a schematic diagram of the principle structure of the anchor hole identification and positioning system in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating the anchor hole identification and positioning method in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the process for obtaining updated image information in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the process for obtaining the two-dimensional target pixel region in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the process for obtaining updated point cloud information in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram illustrating the process of obtaining updated point cloud information for a three-dimensional target region in an embodiment of the present invention;

[0057] Figure 7 This is a flowchart illustrating the anchor arm control method in an embodiment of the present invention;

[0058] Figure 8 This is a display effect diagram of the original image information in an embodiment of the present invention;

[0059] Figure 9 This is a display effect diagram of updated image information in an embodiment of the present invention. Detailed Implementation

[0060] To make the technical objectives, technical solutions, and beneficial effects of the present invention clearer, the technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention; that is, the described embodiments are merely some embodiments of the present invention, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0061] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0062] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0063] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0064] This application addresses the urgent need to improve the intelligence and automation level of anchor bolt support, considering factors such as increasing work efficiency, ensuring worker safety, and reducing workload. Automatic anchor hole positioning technology is one of the key technologies in this regard. The realization of automatic anchor hole positioning can improve the safety of operators and reduce the workload of anchor bolt support. Therefore, this application proposes an anchor hole identification and positioning system, method, anchoring equipment, and storage medium for environments with large differences in lighting and complex backgrounds underground. By fusing feature information from image information collected by a visual sensor and point cloud information collected by lidar at the same time, stable identification and positioning of small anchor holes can be achieved, thereby improving the efficiency and accuracy of anchor hole identification.

[0065] like Figure 1 The diagram shown is a schematic diagram of the principle structure of the anchor hole identification and positioning system in this application embodiment; the anchor hole identification and positioning system will be described below with reference to some specific embodiments.

[0066] The anchor hole identification and positioning system of this application includes a vision sensor, a lidar, an image processor, and a display.

[0067] The visual sensor is installed at a preset position on the anchor arm to acquire raw image information at a preset time. The image information covers the anchor hole area and the first outer contour area of ​​the anchor hole. The lidar is installed at a preset position on the anchor excavator to acquire raw point cloud information at the preset time. The raw point cloud information covers the anchor hole area and the second outer contour area of ​​the anchor hole. The second outer contour area includes the first outer contour area.

[0068] The image processor includes a first processing module and a second processing module. The first processing module performs dual enhancement processing on the original image information based on an adaptive gamma correction algorithm and a dehazing algorithm to obtain updated image information, and determines the two-dimensional target pixel region based on the updated image information and a pre-trained anchor hole recognition model. The second processing module processes the original point cloud information based on a pre-built first joint calibration relationship between the visual sensor and the lidar to obtain updated point cloud information, and processes the updated point cloud information based on the two-dimensional target pixel information to obtain a three-dimensional target region. Then, the three-dimensional target region is processed to obtain the anchor hole position and its center position.

[0069] Since the dark environment of coal mine roadways affects the detection performance of the model, AGC (Adaptive Gamma Correction) is used to improve the overall brightness of images in low-light environments, reduce the impact of uneven illumination, suppress the brightness of high-illumination areas, and enhance the brightness of low-illumination areas. Therefore, in the embodiments of this application, such as Figure 3 As shown, the process of obtaining updated image information by performing dual enhancement processing on the original image information based on the adaptive gamma correction algorithm and the dehazing algorithm includes:

[0070] S110, acquire the image brightness data and illumination component data of each pixel in the original image information.

[0071] Specifically, the image brightness data F(x,y) and illumination component data I(x,y) of each pixel in the original image information are obtained, where x and y are the horizontal and vertical coordinates of the pixel in the original image information, respectively.

[0072] S120 uses an adaptive gamma correction algorithm to process the image brightness data and illumination component data of each pixel to obtain corrected image information.

[0073] For example, the process of using the Adaptive Gamma Correction (AGC) algorithm to process the image brightness data and illumination component data of each pixel to obtain corrected image information includes:

[0074] (1) The average image brightness was calculated by using regression analysis to analyze the illumination component data of all pixels;

[0075] (2) The average brightness of the image and the illumination component data of each pixel are processed to obtain the correction parameters of each pixel;

[0076] Specifically, the correction parameters for each pixel are calculated as follows:

[0077]

[0078] In the formula, γ(x,y) is the correction parameter for each pixel; m is the average image brightness; and I(x,y) is the illumination component data for each pixel.

[0079] (3) Correct the image brightness data of the corresponding pixel based on the correction parameters of each pixel to obtain the corrected image information.

[0080] Specifically, the image brightness correction data for each pixel of the corrected image information is as follows:

[0081]

[0082] The corrected image information is obtained based on the image brightness correction data of all pixels.

[0083] S130, The corrected image information is processed using a preset dehazing algorithm to obtain updated image information.

[0084] In this embodiment, the image enhancement-based dehazing algorithm (CLAHE) is used. As another implementation method, CNN-based dehazing algorithms or image restoration-based dehazing algorithms can also be used.

[0085] This application employs an adaptive gamma correction algorithm and a dehazing algorithm to perform dual enhancement processing on the original image information, achieving a dual enhancement of brightness and contrast. This improves the overall brightness of the original image information while highlighting local details, thus improving imaging quality under low-light conditions in mines. For example, the display effect of the original image information is as follows: Figure 8 As shown, the display effect of updating image information is as follows: Figure 9 As shown, the details of the anchor holes are clearly visible after the double reinforcement treatment.

[0086] In the embodiments of this application, such as Figure 4 As shown, the process of determining the two-dimensional target pixel region based on updated image information and a pre-trained anchor hole recognition model includes:

[0087] S140, the updated image information is input into the pre-trained anchor hole recognition model to obtain the confidence score and location information of multiple two-dimensional pixel regions.

[0088] In this embodiment, the pre-trained anchor hole recognition model is obtained using existing training methods. Since the focus of this application is not on how to train, the training process and the trained anchor hole recognition model will not be described in detail.

[0089] Specifically, the updated image information is input into the anchor hole recognition model. The updated image information is divided into multiple two-dimensional pixel regions, and the confidence score and location information of each two-dimensional pixel region are obtained respectively.

[0090] S150, compare the confidence scores corresponding to each two-dimensional pixel region according to the preset confidence threshold to determine the two-dimensional target pixel region.

[0091] In this embodiment of the application, if the confidence score of a certain two-dimensional pixel region is higher than a preset confidence threshold, the location information corresponding to the two-dimensional pixel region is taken as the two-dimensional target pixel region.

[0092] In the embodiments of this application, such as Figure 5 As shown, the process of processing the original point cloud information to obtain updated point cloud information based on the pre-built first joint calibration relationship between the visual sensor and the lidar includes:

[0093] S210, obtain the rotation matrix and translation vector between the position information of the visual sensor and the position information of the lidar.

[0094] In some embodiments, the extrinsic parameter matrix between the position information of the visual sensor and the position information of the lidar is determined based on a large amount of experimental data. A calibration object that can be observed simultaneously by both sensors is prepared. Multiple frames of synchronized data are acquired using both the lidar and visual sensors. The coordinates of corresponding feature points in the two coordinate systems are extracted. The rigid body transformation matrix between them is solved, and the results are further optimized using a nonlinear optimization algorithm. The extrinsic parameter matrix in this application includes a rotation matrix R and a translation vector t, which enables the laser point cloud to be correctly projected into the image.

[0095] S220, construct the first joint calibration relationship between the visual sensor and the lidar based on the rotation matrix and translation vector.

[0096] In some embodiments, after obtaining the rotation matrix and translation vector between the visual sensor and the lidar, the coordinate information of each pixel in the original point cloud information is transformed to the coordinate system of the image information. The first joint calibration relationship is: P cam =R·

[0097] P lidar +t, where P lidar =[x,y,z] T P represents the coordinates of each pixel in the point cloud, i.e., the radar point coordinates. cam This refers to the coordinates of the radar point in the visual sensor coordinate system after transformation.

[0098] S230, based on the first joint calibration relationship, the original point cloud information is transformed to obtain updated point cloud information, and the updated point cloud information is in the same coordinate system as the image sensor.

[0099] In some embodiments, the coordinates of the original point cloud information are transformed by means of the constructed first joint calibration relationship, that is, the pixel coordinates of the original point cloud information are transformed into a coordinate system with the same origin as the visual sensor, which facilitates subsequent mapping.

[0100] In applications combining LiDAR and visual sensors, calibration is crucial for ensuring data synchronization and fusion accuracy. LiDAR provides high-precision distance information, while visual sensors offer rich color and texture information. Accurate calibration aligns the raw point cloud information with the image information captured by the visual sensor, enabling more accurate environmental understanding and scene interpretation.

[0101] In this embodiment, the process of processing updated point cloud information based on two-dimensional target pixel information to obtain a three-dimensional target region actually involves mapping the two-dimensional target pixel region onto the updated point cloud information, and extracting the anchor hole region and the first outer contour region covered by the updated point cloud information, thereby obtaining the three-dimensional target region. This processing can effectively reduce the massive amount of original point cloud data by an order of magnitude, thereby significantly reducing the computational burden of subsequent point cloud processing.

[0102] In this embodiment, the RANSAC (Random Sample Consensus) method is used for initial plane fitting, and the optimal fitted plane model with the most interior points is determined during the iteration. Then, the outer contour of the anchor hole is extracted from the optimal fitted plane model based on normal estimation. Then, the anchor hole contour of the anchor hole boundary point cloud is refined and preserved by reducing the rectangular box using the redundancy characteristics of the view frustum data. Furthermore, the anchor hole contour is processed to obtain the anchor hole position.

[0103] For example, such as Figure 6 As shown, the process of processing the three-dimensional target area to obtain the anchor hole positions and their center positions includes:

[0104] S240 uses a random sample consensus algorithm to perform optimal planar fitting modeling on the three-dimensional target region.

[0105] Specifically, three non-collinear points are randomly selected in the three-dimensional target region, and an initial planar model is constructed using two directional vectors. Then, it is determined whether each point in the three-dimensional target region is within the initial planar model in order to find the optimal planar fitting model with the most interior points.

[0106] For example, randomly select three non-collinear points in a three-dimensional target region. Construct two direction vectors Then perform the cross product to calculate the normal vector of the plane. Then, any point is substituted into the plane equation to solve for the constant term. Thus, a complete initial plane fitting model is constructed. That is, ax + by + cz + d = 0, where Let (x, y, z) be any point in space.

[0107] For example, for any point in a three-dimensional target region Based on the distance from any point to the initial plane Determine if a point is an interior point of the initial planar model; if dist iIf the value <∈(preset threshold), it is a point in the plane; then, in multiple iterations, the optimal plane fitting model with the most interior points is found, and finally the optimal plane parameters, normal vector and relatively stable point set on the optimal plane can be obtained, which provides the basis for subsequent boundary extraction and attitude calculation.

[0108] S250, the edge contour extraction method based on normal estimation is used to process the optimal plane fitting model to obtain the anchor hole contour.

[0109] In some embodiments, on the optimal plane, the anchor hole contour and the outer contour of the three-dimensional target region are obtained by an edge contour extraction method based on normal estimation; the outer contour of the three-dimensional target region is removed by utilizing the redundancy characteristics of the view frustum to obtain the anchor hole contour.

[0110] S260, determine the anchor hole position based on the coordinate information of the anchor hole profile; and use the perpendicular bisector theorem to process the anchor hole profile to obtain the center position of the anchor hole.

[0111] In some embodiments, the anchor hole position can be determined based on the coordinate information of the anchor hole profile.

[0112] In some embodiments, based on the anchor hole profile obtained by the above fitting, the perpendicular bisector theorem is used to obtain the equation of the perpendicular bisector by arbitrarily selecting two chord segments AB and CD, and calculating their midpoints and slopes respectively. Solve the equations of the two perpendicular bisectors simultaneously to find their intersection point: y0=k m1 (x0-x m1 )+y m1 Then, using the point-to-plane distance formula, the distance from the origin to the fitting plane ax + by + cz + d = 0 is the z0 value of the center of the fitted circle in the coordinate system. Therefore, point P(x0, y0, z0) is the center of the fitted circle. It is the normal vector of the plane.

[0113] This application also proposes an anchor hole identification and positioning method, such as... Figure 2 As shown, it includes:

[0114] S100 acquires raw image information from the visual sensor and raw point cloud information from the lidar within the same preset time.

[0115] Specifically, in order to ensure the consistency of data processing, the raw image information collected by the visual sensor and the raw point cloud information collected by the lidar are extracted at the same preset time.

[0116] S200 performs dual enhancement processing on the original image information based on the adaptive gamma correction algorithm and the dehazing algorithm to obtain updated image information, and determines the two-dimensional target pixel region based on the updated image information and the anchor hole recognition model obtained by preset training.

[0117] Specifically, firstly, an adaptive gamma correction algorithm is used to process the image brightness data and illumination component data of each pixel in the original image information to obtain corrected image information; then, an image enhancement-based dehazing algorithm (CLAHE) is used to process the corrected image information to obtain updated image information. Alternatively, a CNN-based dehazing algorithm or an image restoration-based dehazing algorithm can also be used; finally, the updated image information is input into a pre-trained anchor hole recognition model to obtain the confidence scores and location information corresponding to multiple two-dimensional pixel regions; and the confidence scores corresponding to each two-dimensional pixel region are compared and processed according to a preset confidence threshold to determine the two-dimensional target pixel region.

[0118] The S300 processes the raw point cloud information and the two-dimensional target pixel region based on the pre-built first joint calibration relationship between the visual sensor and the lidar to obtain the three-dimensional target region.

[0119] For example, a first joint calibration relationship between the visual sensor and the LiDAR is constructed based on the rotation matrix and translation vector between the position information of the visual sensor and the position information of the LiDAR. Then, the original point cloud information is processed based on the first joint calibration relationship to obtain updated point cloud information, so that the updated point cloud information and the image sensor are in the same coordinate system. Finally, the updated point cloud information is processed based on the two-dimensional target pixel information to obtain a three-dimensional target region. The process of obtaining a three-dimensional target region by processing the updated point cloud information based on the two-dimensional target pixel information in this application is actually to map the two-dimensional target pixel region to the updated point cloud information, and extract the anchor hole area and the first outer contour area covered by the updated point cloud information, thereby obtaining the three-dimensional target region. This processing can effectively reduce the massive amount of original point cloud data by an order of magnitude, thereby greatly reducing the computational burden of subsequent point cloud processing.

[0120] S400 processes the three-dimensional target area to obtain the anchor hole position and the anchor hole center position.

[0121] In some embodiments, the optimal plane fitting model of the three-dimensional target region is first obtained by using the random sample consensus algorithm, and then the optimal plane fitting model is processed by the edge contour extraction method based on normal estimation to obtain the anchor hole contour. Finally, the anchor hole position is determined according to the coordinate information of the anchor hole contour, and the anchor hole center position is obtained by processing the anchor hole contour using the perpendicular bisector theorem.

[0122] This application also proposes a method such as... Figure 7 The anchor boom control method shown includes:

[0123] S500, obtain the anchor hole position and the anchor hole center position.

[0124] In some embodiments, the anchor hole location and its center location are obtained using the above-described anchor hole identification and positioning method.

[0125] It is understood that the anchor arm control method of this embodiment includes the anchor hole identification and positioning method of the above embodiment. The options in the above embodiment are also applicable to this embodiment, so they will not be described again here.

[0126] S600, based on the predetermined second joint calibration relationship between the visual sensor and the anchor arm, performs transformation processing on the anchor hole center position to obtain the anchor hole center coordinates, wherein the anchor hole center coordinates and the anchor arm coordinates are in the same coordinate system.

[0127] In some embodiments, the anchor hole center coordinates in the visual sensor coordinate system are calculated as described above, and the anchor hole center position is then converted into the anchor arm coordinate system (i.e., anchor hole center coordinates) to enable the anchor arm to perform the correct actions in the linkage control. Through prior linkage calibration between the visual sensor and the anchor arm, the rotation matrix R from the visual sensor coordinate system to the anchor arm coordinate system is obtained. cam→arm Translation vector t cam→arm The coordinates of the anchor hole center after coordinate transformation are P. arm =R cam→arm ·P+t cam→arm The anchor hole normal vector is

[0128] S700 controls the anchor arm to align with the center of the anchor hole based on the center coordinates of the anchor hole.

[0129] After obtaining the three-dimensional coordinates of the anchor hole in the anchor arm coordinate system and the anchor hole normal vector, the corresponding postures of each joint of the anchor arm are calculated through inverse kinematics. This posture information is used as the input for positioning and posture constraints, and the anchor arm is linked to perform anchoring or drilling operations. Specifically, the anchor hole center coordinates are used to guide the position of the anchor arm end, and the normal vector is used to determine the alignment direction of the anchor arm posture, achieving precise alignment of the anchor hole center and adjustment of the incident angle, thereby improving drilling accuracy and automation.

[0130] The anchor hole identification and positioning system of the present invention has the following advantages:

[0131] (1) The system realizes a fully automated process from image acquisition to anchor hole identification and positioning and anchor arm linkage, which reduces the need for manual operation, lowers labor costs and improves production efficiency.

[0132] (2) Data acquisition is carried out using lidar and vision sensors, and the resulting point cloud information is more comprehensive, enabling the system to have depth perception capabilities, obtain high-precision three-dimensional information in real time, be insensitive to changes in illumination, provide redundant perception, and improve the robustness of the system.

[0133] (3) Advanced image processing technology is adopted. AGC (Adaptive Gamma Correction) and CLAHE (Image Enhancement-based Dehazing Algorithm) can achieve dual enhancement of brightness and contrast, improve the overall brightness of the image while highlighting local details, improve the imaging quality under low light conditions in the mine, enhance the image structure and edge information, and improve the accuracy and robustness of subsequent recognition and detection tasks.

[0134] (4) The identification and positioning methods used by the system have the advantages of high detection accuracy, strong real-time performance and high control accuracy, which can effectively improve the efficiency of anchor protection work and thus improve production efficiency.

[0135] This application also proposes an anchoring device, including an anchor arm, a processor, and a controller; the processor uses an anchor hole identification and positioning method to obtain the anchor hole position and its center position; and based on a pre-determined second joint calibration relationship between the visual sensor and the anchor arm, it performs conversion processing on the anchor hole center position to obtain the anchor hole center coordinates; the controller controls the anchor arm to align with the anchor hole center according to the anchor hole center coordinates.

[0136] It is understood that the anchor hole identification and positioning method used in the anchor protection device of this embodiment is the same as the anchor hole identification and positioning method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0137] This application also proposes a computer-readable storage medium storing a computer program that, when executed on a processor, implements an anchor hole identification and positioning method or an anchor arm control method.

[0138] It is understood that the anchor hole identification and positioning method or anchor arm control method involved in this embodiment is the same as that in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0139] For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0141] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0142] Finally, it should be noted that the above embodiments are only for illustration and not for limiting the technical solutions of the present invention. Any equivalent substitutions, modifications or partial substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An anchor hole identification and positioning system, characterized by, The visual sensor, the laser radar and the image processor are comprised; The visual sensor is used for collecting original image information at a preset time, the image information covering an anchor hole region and a first outer contour region of the anchor hole; The laser radar is used for acquiring original point cloud information at the preset time, the original point cloud information covering the anchor hole region and a second outer contour region of the anchor hole; the second outer contour region comprises the first outer contour region; The image processor comprises a first processing module and a second processing module; The first processing module is used for performing double enhancement processing on the original image information based on an adaptive gamma correction algorithm and a defogging algorithm to obtain updated image information, and determining a two-dimensional target pixel region according to the updated image information and an anchor hole recognition model obtained through preset training; The second processing module is used for processing the original point cloud information based on a first joint calibration relationship between the visual sensor and the laser radar to obtain updated point cloud information, and processing the updated point cloud information according to the two-dimensional target pixel information to obtain a three-dimensional target region; then, the three-dimensional target region is processed to obtain an anchor hole position and an anchor hole center thereof.

2. The anchor hole identification and positioning system of claim 1, wherein, The double enhancement processing on the original image information based on the adaptive gamma correction algorithm and the defogging algorithm to obtain the updated image information comprises: obtaining image brightness data and illumination component data of each pixel point in the original image information; processing the image brightness data and the illumination component data of each pixel point using the adaptive gamma correction algorithm to obtain corrected image information; processing the corrected image information using a preset defogging algorithm to obtain the updated image information.

3. The anchor hole identification and location system of claim 2, wherein, The processing of the image brightness data and the illumination component data of each pixel point using the adaptive gamma correction algorithm to obtain the corrected image information comprises: calculating the illumination component data of all pixel points using regression calculation analysis to obtain an image brightness mean value; processing the image brightness mean value and the illumination component data of each pixel point to obtain a correction parameter of each pixel point; correcting the image brightness data of the corresponding pixel point based on the correction parameter of each pixel point to obtain the corrected image information.

4. The anchor hole identification and location system of claim 1, wherein, The determination of the two-dimensional target pixel region according to the updated image information and the anchor hole recognition model obtained through preset training comprises: inputting the updated image information into the pre-trained anchor hole recognition model to obtain confidence scores and position information of a plurality of two-dimensional pixel regions; comparing and processing the confidence scores of each two-dimensional pixel region according to a preset confidence threshold to determine the two-dimensional target pixel region.

5. The anchor hole identification and location system of claim 1, wherein, The processing of the original point cloud information based on the first joint calibration relationship between the visual sensor and the laser radar to obtain the updated point cloud information comprises: obtaining a rotation matrix and a translation vector between position information of the visual sensor and position information of the laser radar; constructing the first joint calibration relationship between the visual sensor and the laser radar according to the rotation matrix and the translation vector; The original point cloud information is coordinate-converted based on the first joint calibration relationship to obtain updated point cloud information, which is in the same coordinate system as the image sensor.

6. The anchor hole identification and location system of claim 1, wherein, The three-dimensional target region is processed to obtain an anchor hole position and an anchor hole center. An optimal plane fitting model is obtained for the three-dimensional target region by using a random sample consensus algorithm. The optimal plane fitting model is processed by an edge contour extraction method based on normal estimation to obtain an anchor hole contour. The anchor hole center is determined according to the coordinate information of the anchor hole contour, and the anchor hole contour is processed by using the bisection theorem to obtain the anchor hole center.

7. A method for identifying and positioning an anchor hole, characterized by, The method comprises the following steps: At a preset time, original image information collected by a visual sensor and original point cloud information collected by a laser radar are obtained. The original image information is double-enhanced by using an adaptive gamma correction algorithm and a dehazing algorithm to obtain updated image information, and a two-dimensional target pixel region is determined according to an anchor hole recognition model obtained by preset training and the updated image information. The original point cloud information and the two-dimensional target pixel region are processed based on a first joint calibration relationship between the visual sensor and the laser radar to obtain a three-dimensional target region. The three-dimensional target region is processed to obtain an anchor hole position and an anchor hole center.

8. A method of controlling an anchor arm, characterized by, The method comprises the following steps: An anchor hole position and an anchor hole center are obtained by using the anchor hole recognition and positioning method of claim 7. The anchor hole center coordinate is obtained by converting the anchor hole center position based on a second joint calibration relationship between the visual sensor and an anchor arm. The anchor arm is controlled to align with the anchor hole center according to the anchor hole center coordinate.

9. An anchoring device, characterized in that The anchor arm, the processor and the controller are included. The anchor hole position and the anchor hole center are obtained by using the anchor hole recognition and positioning method of claim 7, and the anchor hole center coordinate is obtained by converting the anchor hole center position based on a second joint calibration relationship between the visual sensor and an anchor arm. The anchor arm is controlled to align with the anchor hole center according to the anchor hole center coordinate.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the storage medium, and when the computer program is executed on the processor, the anchor hole recognition and positioning method of claim 7 or the anchor arm control method of claim 8 is implemented.

Images