Mining mobile robot and drilling control method thereof

Abstract

The embodiment of the invention provides a mining mobile robot and a drilling control method of the mining mobile robot, and the mining mobile robot is provided with a visual camera, a mechanical arm, a moving device and a control device; drilling equipment is arranged at the tail end of the mechanical arm, the control device can control the moving device to move firstly so that the mining mobile robot can move to the preset position of a mine, and then the visual camera is controlled to shoot original cross-section images of the mine. The method comprises the following steps of: acquiring a first distance between a visual camera and a section and a first section image marked with punching point position information to be punched, and finally determining an actual punching position on the section by a control device based on the punching point position information, the first distance and a preset conversion matrix for converting a tail end coordinate system of a mechanical arm and a camera coordinate system. And the mechanical arm is controlled to punch in the actual punching position. In this way, the mining mobile robot can conduct automatic punching, and the punching efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mining, and in particular to a mobile robot for mining and a drilling control method of the mobile robot for mining. Background Art

[0002] When mining in non-coal mines with relatively hard texture, drilling and blasting is a commonly used method, and rock drilling is the key link in the drilling and blasting method. In rock drilling operations, one type of operation task is horizontal drilling, that is, horizontal drilling operations are carried out on the vertical section of the mine, and then blasting devices are placed in the horizontal holes to blast the mine, and finally mining is carried out.

[0003] In traditional rock drilling operations, holes are usually marked manually on the cross section, and then drilled manually using drilling tools. However, this drilling method is prone to large deviations in drilling position markings due to manual work, and the manual work efficiency is not high. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a mining mobile robot and a drilling control method of the mining mobile robot to solve the problems of large drilling position mark deviation and low manual operation efficiency caused by manual operation in the prior art. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a mobile robot for mining, which is provided with a visual camera, a mechanical arm, a mobile device and a control device; a drilling device is provided at the end of the mechanical arm,

[0006] The control device is used to control the movement of the mobile device so as to move the mining mobile robot;

[0007] The control device is also used to control the visual camera to capture the original cross-sectional image of the mine when the mining mobile robot moves to a preset position in the mine;

[0008] The control device is also used to obtain a first distance between the visual camera and the cross section and a processed first cross-sectional image, wherein the first cross-sectional image is marked with information on punching points to be punched;

[0009] The control device is also used to determine the actual drilling position on the cross section based on the drilling point information, the first distance, and the preset transformation matrix, and control the robotic arm to drive the drilling equipment to drill at the actual drilling position, wherein the preset transformation matrix is ​​used to transform the end coordinate system of the robotic arm and the camera coordinate system.

[0010] Optionally, the mining mobile robot communicates wirelessly with the back-end device through the control device; the control device is specifically used to receive motion control information sent by the back-end device, and control the mobile device to move based on the motion control information, so as to move the mining mobile robot from an initial position to a preset position;

[0011] The control device is also used to send the original cross-sectional image to the back-end device after controlling the visual camera to capture the original cross-sectional image of the mine; and obtain the processed first cross-sectional image sent by the back-end device.

[0012] Optionally, the mining mobile robot is also provided with an IMU (Inertial Measurement Unit);

[0013] The control device is specifically used to obtain the IMU data collected by the IMU and the visual data collected by the visual camera, and based on the IMU data and the visual data collected by the visual camera, the SLAM (Simultaneous Localization and Mapping) algorithm is used for positioning and navigation to locate and navigate the mining mobile robot from an initial position to a preset position;

[0014] The control device is also used to: after controlling the visual camera to capture the original cross-sectional image of the mine, divide the original cross-sectional image into multiple grid images according to a preset grid size, and mark the center point of each grid image as a drilling point to be drilled, so as to obtain a processed first cross-sectional image.

[0015] Optionally, the mining mobile robot communicates wirelessly with the back-end equipment through the control device; the mining mobile robot is also provided with a three-color light;

[0016] Correspondingly, the control device is also used to obtain the status information of the mining mobile robot in real time; the status information includes: moving status, drilling status and abnormal status;

[0017] When the status information is a moving status, controlling the three-color light to flash yellow;

[0018] When the status information is a punching status, controlling the three-color light to flash green;

[0019] When the status information is in an abnormal state, the three-color light is controlled to flash red, and a control request message is sent to a back-end device to request the back-end device to control the mining mobile robot through the control device.

[0020] Optionally, the control device is specifically used to determine the actual drilling position on the section based on the drilling point information, the first distance, and the preset conversion matrix, plan the drilling path according to the actual drilling position, and control the robotic arm to drive the drilling equipment to drill according to the drilling path.

[0021] Optionally, the control device is further used to: obtain obstacle point information on the first cross-sectional image; and determine the actual position of the obstacle on the cross-section based on the obstacle point information, the first distance, and a preset conversion matrix;

[0022] The control device is also used to: plan a drilling path according to the actual drilling position and the actual position of the obstacle, and control the mechanical arm to drive the drilling equipment to drill holes according to the drilling path.

[0023] In a second aspect, an embodiment of the present invention further provides a drilling control method for a mining mobile robot, which is applied to a control device on the mining mobile robot. The mining mobile robot is also provided with a visual camera, a mechanical arm and a mobile device; a drilling device is provided at the end of the mechanical arm. The method comprises:

[0024] When the mining mobile robot moves to a preset position in the mine, an original cross-sectional image of the mine taken by a visual camera is obtained;

[0025] Acquire a first distance between the visual camera and the cross section and a processed first cross-sectional image, wherein the first cross-sectional image is marked with information on punching points to be punched;

[0026] Based on the drilling point information, the first distance, and the preset transformation matrix, the actual drilling position on the cross section is determined, and the robotic arm is controlled to drive the drilling equipment to drill at the actual drilling position, wherein the preset transformation matrix is ​​used to transform the end coordinate system of the robotic arm and the camera coordinate system.

[0027] Optionally, the mining mobile robot communicates wirelessly with the back-end device through the control device; before the mining mobile robot moves to a preset position in the mine and obtains the original cross-sectional image of the mine taken by the visual camera, the method further includes:

[0028] Receive motion control information sent by the back-end device, and control the mobile device to move based on the motion control information to move the mining mobile robot from an initial position to a preset position;

[0029] After acquiring the original cross-sectional image of the mine taken by the visual camera, the method further comprises:

[0030] The original cross-sectional image is sent to the back-end device; and the processed first cross-sectional image sent by the back-end device is obtained.

[0031] Optionally, the mining mobile robot is further provided with an IMU; accordingly, before the mining mobile robot moves to a preset position of the mine, obtaining the original cross-sectional image of the mine taken by the visual camera, the method further includes:

[0032] Obtain the IMU data collected by the IMU and the visual data collected by the visual camera, and based on the IMU data and the visual data collected by the visual camera, use the SLAM algorithm for positioning and navigation to locate and navigate the mining mobile robot from the initial position to the preset position;

[0033] After acquiring the original cross-sectional image of the mine taken by the visual camera, the method further comprises:

[0034] According to a preset grid size, the original cross-sectional image is divided into a plurality of grid images, and the center point of each grid image is marked as a punching point to be punched, so as to obtain a processed first cross-sectional image.

[0035] Optionally, the mining mobile robot communicates wirelessly with the back-end device through the control device; the mining mobile robot is also provided with a three-color light; accordingly, the method further includes:

[0036] Acquire the status information of the mining mobile robot in real time; the status information includes: moving status, drilling status and abnormal status;

[0037] When the status information is a moving status, controlling the three-color light to flash yellow;

[0038] When the status information is a punching status, controlling the three-color light to flash green;

[0039] When the status information is in an abnormal state, the three-color light is controlled to flash red, and a control request message is sent to a back-end device to request the back-end device to control the mining mobile robot through the control device.

[0040] Optionally, based on the drilling point information, the first distance, and the preset conversion matrix, determining the actual drilling position on the cross section, and controlling the mechanical arm to drive the drilling device to drill holes at the actual drilling position includes:

[0041] Determine the actual drilling position on the cross section based on the drilling point information, the first distance, and the preset conversion matrix;

[0042] Plan the drilling path according to the actual drilling position, and control the robotic arm to drive the drilling equipment to drill holes along the drilling path.

[0043] Optionally, before planning a drilling path according to the actual drilling position and controlling the mechanical arm to drive the drilling device to perform drilling according to the drilling path, the method further includes:

[0044] Obtaining obstacle point information on the first cross-sectional image; and determining the actual position of the obstacle on the cross-sectional image based on the obstacle point information, the first distance, and a preset conversion matrix;

[0045] Plan the drilling path according to the actual drilling position, and control the robot arm to drive the drilling equipment to drill holes according to the drilling path, including:

[0046] The drilling path is planned according to the actual drilling position and the actual position of the obstacle, and the robotic arm is controlled to drive the drilling equipment to drill holes according to the drilling path.

[0047] Beneficial effects of the embodiments of the present invention:

[0048] The embodiment of the present invention provides a mining mobile robot and a drilling control method of the mining mobile robot. The mining mobile robot is provided with a visual camera, a mechanical arm, a mobile device and a control device; a drilling device is provided at the end of the mechanical arm, and the mining mobile robot communicates wirelessly with a back-end device through the control device;

[0049] The control device can first control the movement of the mobile device to move the mining mobile robot; when the mining mobile robot moves to the preset position of the mine, the visual camera is controlled to shoot the original cross-sectional image of the mine; and the original cross-sectional image is sent to the back-end device; then the control device obtains the first distance between the visual camera and the cross-section and the first cross-sectional image sent by the back-end device and marked with the punching point information to be punched, and finally the control device determines the actual punching position on the cross-section based on the punching point information, the first distance, and the preset conversion matrix for converting the end coordinate system of the manipulator and the camera coordinate system, and controls the manipulator to punch at the actual punching position. In this way, by shooting the cross-sectional image of the mine cross-section to be punched and marking the punching point information to be punched on the cross-sectional image, the mining mobile robot can be automatically controlled to automatically punch holes. There is no need to manually mark the cross-section, which solves the problem of large errors when manually marking the hole position on the cross-section, and because the mining mobile robot is used for automatic drilling, the drilling efficiency is improved. Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0051] Figure 1This is a schematic diagram of the structure of a mining mobile robot according to an embodiment of the present invention;

[0052] Figure 2 A schematic flow chart of a first implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention;

[0053] Figure 3 A schematic flow chart of a second implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention;

[0054] Figure 4 A schematic flow chart of a third implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention;

[0055] Figure 5 A schematic flow chart of a fourth implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention;

[0056] Figure 6 It is a flowchart of a fifth implementation manner of a drilling control method for a mining mobile robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0058] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a mobile mining robot and a drilling control method for the mobile mining robot, so as to solve the problems of large drilling position marking deviation and low manual operation efficiency caused by manual operation in the prior art.

[0059] Next, a mining mobile robot and a drilling control method of the mining mobile robot according to an embodiment of the present invention are introduced.

[0060] like Figure 1 FIG. 1 is a schematic diagram of the structure of a mobile robot for mining according to an embodiment of the present invention. Figure 1 In the embodiment, the mining mobile robot comprises: a visual camera 110, a mechanical arm 120, a mobile device 130 and a control device 140; a drilling device 121 is provided at the end of the mechanical arm 120, and the mining mobile robot communicates wirelessly with the back-end device through the control device 140;

[0061] The mobile device 130 drives the mining mobile robot to move under the control of the control device 140. The mobile device 130 may be a chassis with tracks installed, on which components such as an engine, a transmission system, and a gearbox may be installed, and the power generated by the engine may be transmitted to the tracks to drive the mining mobile robot to move.

[0062] In some examples, the control device 140 may be provided with a wireless communication module and a control module. The mining mobile robot may communicate wirelessly with the back-end device through the wireless communication module. The back-end device may be provided with a screen, and the visual camera may take a video of the environment in real time during the movement, and then send it to the back-end device through the control device. The user of the back-end device may observe through the screen, and the user may send motion control information to the control device through the back-end device. After receiving the motion control information, the control device may control the mobile device to move based on the motion control information, so as to move the mining mobile robot from the initial position to the preset position.

[0063] The motion control information may include control information such as forward, left, right, backward, stop, etc. Among them, the forward direction of the mining mobile robot is the control information of "forward", the left side of the forward direction of the mining mobile robot is the control information of "left", the right side of the forward direction of the mining mobile robot is the control information of "right", and the backward direction opposite to the forward direction of the mining mobile robot is the control information of "backward".

[0064] These motion control information can be adjusted according to the actual scene. For example, when moving from the initial position to the preset position, the direction of movement to the preset position can be used as the forward direction, and when moving from the preset position to the initial position, the direction of movement to the initial position can be used as the forward direction.

[0065] When the user observes through the screen that the mining mobile robot moves to a preset position, a "stop" control message may be sent to the control device. In some examples, the preset position is a pre-set position.

[0066] When the mining mobile robot moves to a preset position in the mine, the control device 140 can control the visual camera to capture the original cross-sectional image of the mine; and send the original cross-sectional image to the back-end device.

[0067] After the control device sends the captured original cross-sectional image to the back-end device through the wireless communication module, the user can operate the back-end device to mark the position to be punched on the back-end device, so that the back-end device can obtain the punching point information marked on the original cross-sectional image. Then, the back-end device can send the original cross-sectional image marked with the punching point information to be punched as the processed first cross-sectional image to the control device. Therefore, the control device can obtain the processed first cross-sectional image sent by the back-end device.

[0068] In some examples, in addition to being able to move to a preset position under the motion control information sent by the user through the back-end device, the embodiment of the present invention also provides a possible implementation method.

[0069] Specifically, the mining mobile robot is also provided with an IMU; when the mining mobile robot starts from an initial position, the IMU begins to collect IMU data and image frames collected by the visual camera, and then based on the collected IMU data and image frames collected by the visual camera, a SLAM algorithm is used to perform positioning and navigation, so as to locate and navigate the mining mobile robot from the initial position to a preset position.

[0070] In some examples, the SLAM algorithm can be an OKVIS (Open Keyframe-based Visual-Inertial SLAM) algorithm or a VINS-Mono (Monocular Visual-Inertial System) algorithm, wherein the visual data used by the OKVIS algorithm is visual data collected by a binocular visual camera, and the visual data used by the VINS-Mono algorithm is visual data collected by a barrage visual camera. The visual camera is used to collect images of the mining environment, and then extract feature information of the mining environment based on the images of the mining environment.

[0071] The mining mobile robot is interconnected with the back-end equipment through the control device, and obtains the existing map information by connecting to the existing geographic information system of the mine. This information is used as the initial map for the robot's movement and forms the final global route planning. During operation, the visual camera is used to perceive the surrounding environment and plan the robot's route during operation, so that the mining mobile robot conforms to the global optimal route as much as possible and matches it with the map, thereby determining its own position and direction of action.

[0072] At the same time, IMU is used as an auxiliary. The inertial navigation system calculates the position change of the robot by measuring its acceleration and angular velocity, and can provide accurate positioning information in a short time. In the process of operation, it implements obstacle avoidance, searches for avoidance and travel paths, and adds heuristic functions during the travel process to reduce the search time.

[0073] After controlling the visual camera to capture the original cross-sectional image of the mine, the control device can divide the original cross-sectional image into multiple grid images according to a preset grid size, and mark the center point of each grid image as a drilling point to be drilled, thereby obtaining a processed first cross-sectional image.

[0074] Specifically, after the visual camera captures the original cross-sectional image, the image forms a pixel coordinate system in the camera. Since the image is slightly distorted from the actual image due to the manufacturing process of the lens after the lens reaches the sensor photoreceptor during the shooting process, the Taylor series around the principal point is used to expand the image to form a pixel coordinate system that matches the actual object. Based on the pixel coordinate system, the entire image is divided into a grid map of a fixed size, and divided into grid maps of the same size. Each grid represents a spatial unit, and each small grid of the grid map is used as a unit to form a node of the entire map. Then, the center point of each small grid is used as the position to be punched to mark it, so that the processed first cross-sectional image can be obtained.

[0075] Then, the control device can also obtain a first distance between the visual camera and the cross section, wherein the visual camera can measure the depth between the visual camera and the cross section, thereby obtaining the first distance between the visual camera and the cross section.

[0076] Finally, the control device can determine the actual drilling position on the cross section based on the drilling point information, the first distance, and the preset conversion matrix, and control the robot arm to drive the drilling equipment to drill at the actual drilling position.

[0077] In some examples, the preset transformation matrix may be a transformation matrix obtained in advance through calibration, which can be used to transform the end coordinate system of the robot arm and the camera coordinate system. Then, combined with the punching point information and the first distance, the actual position of the robot arm to punch a hole on the cross section can be determined. Then, the robot arm can be controlled to rotate so that the drilling device of the robot arm can drill a hole at the actual drilling position.

[0078] Specifically, the control device verifies the distance between the mining mobile robot and the drilling surface measured by the laser radar and its own position actually calculated by the inertial navigation system in combination with the position of the section actually required to be drilled in the existing mining geographic information system to determine the accuracy of the drilling position information.

[0079] The control device verifies and confirms the distance of the actual controllable position of its own robotic arm and the distance between the mining mobile robot and the drilling surface. If the above distance information cannot be met, the mining mobile robot autonomously adjusts the optimal position according to the actual distance.

[0080] After marking the center point of each small grid as the position to be drilled and obtaining the processed first cross-sectional image, the control device can pre-set the drilling density and drilling positions of the mining design in the mining mobile robot control device / back-end device based on the drilling point information, the first distance, the preset conversion matrix and the processed first cross-sectional image. The mining mobile robot accurately marks the drilling positions on the full-map nodes of the grid map by processing in the control device or remotely sending them to the mining mobile robot, determines the actual drilling positions on the cross-section, converts the full-map nodes into machine-recognizable instructions, and controls the robotic arm to drive the drilling equipment to drill holes at the actual drilling positions, wherein the preset conversion matrix is ​​used to convert the end coordinate system of the robotic arm and the camera coordinate system.

[0081] In some examples, the robotic arm can establish a coordinate system with the center point of the grid map as the coordinate origin, and then divide the grid map into four quadrants. Then, according to the set punching points, starting from the center position of the grid map, first punch holes one by one in the first quadrant (x, y), and then extend to the second quadrant (-x, +y), the third quadrant (-x, -y), and the fourth quadrant (+x, -y) to punch holes one by one.

[0082] The embodiment of the present invention provides a mining mobile robot, which can automatically control the mining mobile robot to automatically perform drilling by shooting a cross-sectional image of a mine section to be drilled and marking the drilling point information to be drilled on the cross-sectional image. There is no need to manually mark the cross section, which solves the problem of large errors when manually marking the hole positions on the cross section, and because the mining mobile robot is used for automatic drilling, the drilling efficiency is improved.

[0083] In some examples, the mining mobile robot can communicate with the back-end equipment wirelessly through the control device; the mining mobile robot is also provided with a three-color light; accordingly,

[0084] The control device can also obtain the status information of the mining mobile robot in real time during the process of moving from the initial position to drilling and returning to the initial position after drilling; the status information includes: moving status, drilling status and abnormal status;

[0085] In some examples, when the mining mobile robot is moving, the control device can obtain the speed information of the mining mobile robot, so as to confirm that the mining mobile robot is moving, and further determine that the state information of the mining mobile robot is a moving state. At this time, the control device can control the three-color light to flash yellow.

[0086] In some other examples, when the speed information of the mobile mining robot is zero and the robotic arm of the mobile mining robot is moving, it can be confirmed that the mobile mining robot is actually preparing to drill or is already drilling. At this time, it can be determined that the state of the mobile mining robot is the drilling state, and therefore, the three-color light can be controlled to flash green.

[0087] In some other examples, the mobile mining robot may be equipped with various other sensors. When the mobile mining robot fails, the sensor may obtain abnormal data information, thereby determining the abnormal information. Then, the control device may obtain the abnormal information on the mobile mining robot and determine that the mobile mining robot fails. At this time, the three-color light may be controlled to flash red to provide a light prompt. At this time, a control request message may be sent to the back-end device to request the back-end device to control the mobile mining robot through the control device. In this way, after the mobile mining robot fails, the back-end device may take over the control of the mobile mining robot.

[0088] In some examples, after determining the actual drilling position on the cross section based on the drilling point information, the first distance, and the preset conversion matrix, in order to complete the drilling operation as quickly as possible and further improve the drilling efficiency, in an embodiment of the present invention, the drilling path can also be planned according to the actual drilling position, and then the robotic arm can be controlled to drive the drilling equipment to perform drilling according to the drilling path.

[0089] In some instances, when planning the perforation path according to the actual perforation positions, an RRT (Rapid Random Tree) algorithm may be used to plan the perforation path so as to minimize the time required from the start of perforation to the end of perforation and improve the perforation efficiency.

[0090] In some other examples, there may be obstacles on the section to be drilled in the mine. In order to avoid obstacles and further reduce the time required from the start of drilling to the end of drilling, when the control device sends the original cross-sectional image taken to the back-end device, the user can mark the obstacle position on the original cross-sectional image. Therefore, the back-end device can obtain the obstacle point information on the first cross-sectional image, and send the first cross-sectional image marked with the obstacle point information to the control device. After receiving the first cross-sectional image, the control device can obtain the obstacle point information on the first cross-sectional image; then determine the actual position of the obstacle on the section based on the obstacle point information, the first distance, and the preset conversion matrix;

[0091] Finally, the control device can plan a drilling path according to the actual drilling position and the actual position of the obstacle, and control the mechanical arm to drive the drilling device to drill holes according to the drilling path. In this way, the planned drilling path can avoid obstacles as much as possible, so that the mechanical arm can move from one drilling position to another as quickly as possible, reducing the time required from the start of drilling to the end of drilling.

[0092] Corresponding to the above-mentioned mining mobile robot, an embodiment of the present invention further provides a drilling control method of the mining mobile robot, which can be applied to a control device on the mining mobile robot, and the mining mobile robot is also provided with a visual camera, a mechanical arm and a mobile device; a drilling device is provided at the end of the mechanical arm, and the mining mobile robot communicates wirelessly with the back-end device through the control device; Figure 2 As shown, the method includes:

[0093] S210, when the mining mobile robot moves to a preset position of the mine, obtaining an original cross-sectional image of the mine taken by a visual camera;

[0094] S220, acquiring a first distance between the visual camera and the cross section and a processed first cross-sectional image; wherein the first cross-sectional image is marked with information on punching points to be punched;

[0095] S230, based on the drilling point information, the first distance, and the preset conversion matrix, determine the actual drilling position on the cross section, and control the mechanical arm to drive the drilling device to drill at the actual drilling position. The preset conversion matrix is ​​used to convert the end coordinate system of the mechanical arm and the camera coordinate system.

[0096] exist Figure 2 Based on the drilling control method of a mining mobile robot shown in FIG. , the embodiment of the present invention further provides a possible implementation method, such as Figure 3 FIG. 1 is a flow chart of a second implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention. The method may include:

[0097] S310, receiving motion control information sent by the back-end device, and controlling the mobile device to move based on the motion control information, so as to move the mining mobile robot from an initial position to a preset position;

[0098] S320, when the mining mobile robot moves to a preset position of the mine, an original cross-sectional image of the mine taken by the visual camera is obtained; and the cross-sectional image is sent to a back-end device;

[0099] S330, obtaining a first distance between the visual camera and the cross section and a processed first cross-sectional image sent by the back-end device; wherein the first cross-sectional image is marked with information on punching points to be punched;

[0100] S340, based on the drilling point information, the first distance, and the preset conversion matrix, determine the actual drilling position on the cross section, and control the mechanical arm to drive the drilling device to drill at the actual drilling position. The preset conversion matrix is ​​used to convert the end coordinate system of the mechanical arm and the camera coordinate system.

[0101] It can be understood that steps S320 to S340 in the embodiment of the present invention are the same as or similar to steps S210 to S230 in the first implementation manner.

[0102] In some examples, the mining mobile robot is also provided with an IMU; Figure 2 Based on the drilling control method of a mining mobile robot shown in FIG. , the embodiment of the present invention further provides a possible implementation method, such as Figure 4 FIG. 1 is a flow chart of a third implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention. The method may include:

[0103] S410, obtaining IMU data collected by the IMU and visual data collected by the visual camera, and based on the IMU data and the visual data collected by the visual camera, using the SLAM algorithm for positioning and navigation, positioning and navigating the mining mobile robot from an initial position to a preset position;

[0104] S420, when the mining mobile robot moves to a preset position of the mine, obtaining an original cross-sectional image of the mine taken by a visual camera;

[0105] S430, dividing the original cross-sectional image into a plurality of grid images according to a preset grid size, and marking the center point of each grid image as a punching point to be punched, to obtain a processed first cross-sectional image; wherein the first cross-sectional image is marked with information on the punching points to be punched;

[0106] S440, obtaining a first distance between the visual camera and the cross section;

[0107] S450, based on the drilling point information, the first distance, and the preset transformation matrix, determine the actual drilling position on the cross section, and control the robot arm to drive the drilling equipment to drill at the actual drilling position, wherein the preset transformation matrix is ​​used to transform the end coordinate system of the robot arm and the camera coordinate system.

[0108] It can be understood that steps S420, S440 and S450 in the embodiment of the present invention are the same as or similar to S210 to S230 in the first implementation manner.

[0109] exist Figure 2 Based on the drilling control method of a mining mobile robot shown in FIG. , the embodiment of the present invention further provides a possible implementation method, such as Figure 5 FIG. 1 is a flow chart of a fourth implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention. The method may include:

[0110] S510, when the mining mobile robot moves to a preset position of the mine, obtaining an original cross-sectional image of the mine taken by a visual camera;

[0111] S520, acquiring a first distance between the visual camera and the cross section and a processed first cross-sectional image; wherein the first cross-sectional image is marked with information on punching points to be punched;

[0112] S530, determining an actual drilling position on the cross section based on the drilling point information, the first distance, and a preset conversion matrix;

[0113] S540, planning a drilling path according to the actual drilling position, and controlling the mechanical arm to drive the drilling device to drill holes according to the drilling path. The preset conversion matrix is ​​used to convert the end coordinate system of the mechanical arm and the camera coordinate system.

[0114] It can be understood that steps S510 to S520 in the embodiment of the present invention are the same as or similar to steps S210 to S220 in the first implementation manner.

[0115] exist Figure 5 Based on the drilling control method of a mining mobile robot shown in FIG. , the embodiment of the present invention further provides a possible implementation method, such as Figure 6 FIG. 1 is a flow chart of a fifth implementation of a drilling control method for a mining mobile robot according to an embodiment of the present invention. The method may include:

[0116] S610, when the mining mobile robot moves to a preset position of the mine, obtaining an original cross-sectional image of the mine taken by a visual camera;

[0117] S620, acquiring a first distance between the visual camera and the cross section and a processed first cross-sectional image; wherein the first cross-sectional image is marked with information on punching points to be punched;

[0118] S630, determining an actual drilling position on the cross section based on the drilling point information, the first distance, and a preset conversion matrix;

[0119] S640, obtaining obstacle point information on the first cross-sectional image; and determining the actual position of the obstacle on the cross-sectional image based on the obstacle point information, the first distance, and a preset conversion matrix;

[0120] S650, planning a drilling path according to the actual drilling position and the actual position of the obstacle, and controlling the robot arm to drive the drilling device to drill holes according to the drilling path. The preset transformation matrix is ​​used to transform the end coordinate system of the robot arm and the camera coordinate system.

[0121] It can be understood that steps S610 to S630 in the embodiment of the present invention are the same as or similar to steps S510 to S530 in the first implementation manner.

[0122] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0123] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0124] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method embodiment, since it is basically similar to the mining mobile robot embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A mobile robot for mining, characterized in that: The mining mobile robot is provided with a visual camera, a mechanical arm, a moving device and a control device; a drilling device is provided at the end of the mechanical arm. The control device is used to control the movement of the mobile device so as to move the mining mobile robot; The control device is also used to control the visual camera to capture the original cross-sectional image of the mine when the mining mobile robot moves to a preset position of the mine; The control device is also used to obtain a first distance between the visual camera and the cross section and a processed first cross-sectional image, wherein the first cross-sectional image is marked with information on punching points to be punched; The control device is also used to determine the actual drilling position on the section based on the drilling point information, the first distance, and a preset transformation matrix, and control the robotic arm to drive the drilling equipment to drill a hole at the actual drilling position, wherein the preset transformation matrix is ​​used to transform the end coordinate system of the robotic arm and the camera coordinate system.

2. The mining mobile robot according to claim 1, characterized in that: The mining mobile robot communicates wirelessly with the back-end equipment via the control device; The control device is specifically used to receive the motion control information sent by the back-end device, and control the movement of the mobile device based on the motion control information to move the mining mobile robot from an initial position to the preset position; The control device is also used to send the original cross-sectional image to the back-end device after controlling the visual camera to capture the original cross-sectional image of the mine; and obtain the processed first cross-sectional image sent by the back-end device.

3. The mining mobile robot according to claim 1, characterized in that: The mining mobile robot is also provided with an inertial measurement unit IMU; The control device is specifically used to obtain the IMU data collected by the IMU and the visual data collected by the visual camera, and based on the IMU data and the visual data collected by the visual camera, and using a simultaneous positioning and mapping SLAM algorithm for positioning and navigation, the mining mobile robot is positioned and navigated from an initial position to the preset position; The control device is also used to: after controlling the visual camera to capture the original cross-sectional image of the mine, divide the original cross-sectional image into multiple grid images according to a preset grid size, and mark the center point of each grid image as a drilling point to be drilled, so as to obtain a processed first cross-sectional image.

4. The mining mobile robot according to claim 3, characterized in that: The mining mobile robot communicates wirelessly with the back-end equipment through the control device; the mining mobile robot is also provided with a three-color light; The control device is also used to obtain the status information of the mining mobile robot in real time; the status information includes: moving status, drilling status and abnormal status; When the status information is a moving status, controlling the three-color light to flash yellow; When the status information is a punching status, controlling the three-color light to flash green; When the status information is in an abnormal state, the three-color light is controlled to flash red, and a control request message is sent to a back-end device to request the back-end device to control the mining mobile robot through the control device.

5. The mining mobile robot according to claim 1, characterized in that: The control device is specifically used to plan a drilling path according to the actual drilling position after determining the actual drilling position on the section based on the drilling point information, the first distance, and the preset conversion matrix, and to control the robotic arm to drive the drilling equipment to perform drilling according to the drilling path.

6. The mining mobile robot according to claim 5, characterized in that: The control device is further used to: obtain obstacle point information on the first cross-sectional image; and determine the actual position of the obstacle on the cross-section based on the obstacle point information, the first distance, and a preset conversion matrix; The control device is also used to: plan a drilling path according to the actual drilling position and the actual position of the obstacle, and control the mechanical arm to drive the drilling equipment to drill holes according to the drilling path.

7. A drilling control method for a mining mobile robot, characterized in that: A control device applied to a mobile mining robot, wherein the mobile mining robot is also provided with a visual camera, a mechanical arm and a mobile device; a drilling device is provided at the end of the mechanical arm, and the mobile mining robot communicates wirelessly with a back-end device through the control device; the method comprises: When the mining mobile robot moves to a preset position in the mine, an original cross-sectional image of the mine taken by the visual camera is acquired; Acquire a first distance between the visual camera and the cross section and a processed first cross-sectional image, wherein the first cross-sectional image is marked with information of punching points to be punched; Based on the drilling point information, the first distance, and the preset transformation matrix, the actual drilling position on the section is determined, and the robotic arm is controlled to drive the drilling equipment to drill a hole at the actual drilling position, wherein the preset transformation matrix is ​​used to transform the end coordinate system of the robotic arm and the camera coordinate system.

8. The method according to claim 7, characterized in that The mining mobile robot is also provided with an IMU; before the mining mobile robot moves to a preset position of the mine and obtains the original cross-sectional image of the mine taken by the visual camera, the method further includes: Acquire the IMU data collected by the IMU and the visual data collected by the visual camera, and based on the IMU data and the visual data collected by the visual camera, use the SLAM algorithm to perform positioning and navigation, and locate and navigate the mining mobile robot from an initial position to the preset position; After acquiring the original cross-sectional image of the mine taken by the visual camera, the method further includes: According to a preset grid size, the original cross-sectional image is divided into a plurality of grid images, and the center point of each grid image is marked as a punching point to be punched, so as to obtain a processed first cross-sectional image.

9. The method according to claim 7, characterized in that: The mining mobile robot communicates wirelessly with the back-end equipment through the control device, and the mining mobile robot is also provided with a three-color light; the method further includes: Acquire the status information of the mining mobile robot in real time; the status information includes: moving status, drilling status and abnormal status; When the status information is a moving status, controlling the three-color light to flash yellow; When the status information is a punching status, controlling the three-color light to flash green; When the status information is in an abnormal state, the three-color light is controlled to flash red, and a control request message is sent to a back-end device to request the back-end device to control the mining mobile robot through the control device.

10. The method according to claim 7, characterized in that The method of determining the actual drilling position on the cross section based on the drilling point information, the first distance, and a preset conversion matrix, and controlling the mechanical arm to drive the drilling device to drill a hole at the actual drilling position includes: Determine an actual drilling position on the cross section based on the drilling point information, the first distance, and a preset conversion matrix; A drilling path is planned according to the actual drilling position, and the robot arm is controlled to drive the drilling device to perform drilling according to the drilling path.

Images