Mechanical arm punching method and device, electronic equipment and storage medium

Abstract

The invention discloses a mechanical arm punching method and device, electronic equipment and a storage medium, and relates to the technical field of tunnel punching. The method comprises the steps of projecting a two-dimensional coordinate of a current punching point in a punching task to a three-dimensional point cloud collected by a three-dimensional camera based on a pre-calibrated conversion relation between a two-dimensional wall coordinate system and a three-dimensional camera coordinate system, and obtaining a current space point coordinate corresponding to the current punching point in a three-dimensional space; according to the three-dimensional point cloud data collected by the three-dimensional camera and the coordinates of the current space point, the punching direction of the current space point is determined; according to the current space point coordinates and preset hole depth parameters, a mechanical arm is controlled to conduct punching operation in the punching direction; according to the real-time collected actual current values of the key part of the mechanical arm and the percussion drill, whether an obstacle is encountered or not is determined; and if the robot encounters the obstacle, the mechanical arm is controlled to execute gun withdrawing operation in the direction opposite to the punching direction. According to the technical scheme, the stability and reliability of punching operation are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of mechanical arm drilling technology, in particular to the technical field of tunnel drilling, and specifically to a mechanical arm drilling method, device, electronic equipment and storage medium. BACKGROUND

[0002] As one of the important application directions of engineering automation, the mechanical arm drilling method has been gradually introduced into the tunnel construction scene to replace the traditional manual drilling method to improve the efficiency and quality of the equipment installation and other work links in the tunnel.

[0003] The existing tunnel drilling still mainly relies on manual positioning of the drilling point in a narrow or dusty environment, which is affected by factors such as light, smoke, uneven wall, etc., and is prone to positioning deviation, resulting in low drilling efficiency and poor accuracy. In order to improve this situation, existing automated equipment tries to be equipped with laser ranging, depth camera or total station measuring devices, which scans the tunnel wall surface and manually inputs or selects the drilling position to realize semi-automatic drilling operation. However, such devices usually need to arrange calibration targets on the wall, and through multiple manual calibrations, the correspondence between the device coordinate system, the camera coordinate system and the wall surface coordinate system is established, which is overall cumbersome and sensitive to construction conditions; when the construction environment changes or the device moves, it often needs to be recalibrated, resulting in poor device adaptability.

[0004] Therefore, the existing technology still has the problem that the drilling direction cannot be automatically determined, which makes it difficult to ensure the stability and reliability of the drilling operation. SUMMARY

[0005] The present application provides a mechanical arm drilling method, device, electronic equipment and storage medium to improve the stability and reliability of the drilling operation.

[0006] According to an aspect of the present application, a mechanical arm drilling method is provided, comprising:

[0007] Based on the conversion relationship between the two-dimensional wall coordinate system and the three-dimensional camera coordinate system pre-calibrated, the two-dimensional coordinates of the current drilling point in the drilling task are projected into the three-dimensional point cloud collected by the three-dimensional camera to obtain the corresponding current spatial point coordinates of the current drilling point in the three-dimensional space;

[0008] According to the three-dimensional point cloud data collected by the three-dimensional camera and the current spatial point coordinates, the drilling direction of the current spatial point is determined;

[0009] According to the current spatial point coordinates and the preset hole depth parameter, the mechanical arm is controlled to perform drilling operation along the drilling direction;

[0010] According to the real-time collected key positions of the mechanical arm and the actual current value of the percussion drill, it is determined whether an obstacle is encountered; if an obstacle is encountered, the mechanical arm is controlled to perform a gun withdrawing operation in the opposite direction of the drilling direction.

[0011] According to another aspect of the present application, a mechanical arm drilling device is provided, comprising:

[0012] A three-dimensional space point coordinate determination module is configured to project a two-dimensional coordinate of a current drilling point in a drilling task to a three-dimensional point cloud collected by a three-dimensional camera based on a conversion relationship between a two-dimensional wall coordinate system and a three-dimensional camera coordinate system, to obtain a current space point coordinate corresponding to the current drilling point in a three-dimensional space.

[0013] A drilling direction determination module is configured to determine a drilling direction of the current space point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current space point coordinate.

[0014] A mechanical arm drilling control module is configured to control the mechanical arm to perform a drilling operation along the drilling direction according to the current space point coordinate and a preset hole depth parameter.

[0015] An obstacle determination module is configured to determine whether an obstacle is encountered according to real-time collected key positions of the mechanical arm and the actual current value of the percussion drill; if an obstacle is encountered, the mechanical arm is controlled to perform a gun withdrawing operation in the opposite direction of the drilling direction.

[0016] According to another aspect of the present application, an electronic device is provided, comprising:

[0017] At least one processor; and

[0018] A memory in communication connection with the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the mechanical arm drilling method according to any one of the embodiments of the present application.

[0020] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to execute the mechanical arm drilling method according to any one of the embodiments of the present application.

[0021] According to another aspect of the present application, a computer program product is provided, which comprises a computer program, and the computer program, when executed by a processor, implements any one of the mechanical arm drilling methods provided by the embodiments of the present application.

[0022] The technical scheme of the embodiment of the application realizes accurate projection of the two-dimensional punching position into the three-dimensional point cloud by establishing an accurate mapping relationship between the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system, so as to obtain a real and reliable punching starting point coordinate in the three-dimensional space, and automatically determine a punching direction consistent with the normal direction of the wall surface in combination with the point cloud data, so that the mechanical arm can stably punch in the correct posture, and avoid punching position deviation and hole direction inclination caused by improper manual operation or uneven wall surface; the punching process is quantitatively controlled by the preset hole depth parameter, so as to ensure the consistency of the punching depth and the punching quality; the real-time monitoring of the key parts of the mechanical arm and the working current of the impact drill can timely identify the change of the working state and automatically trigger the gun withdrawal operation when encountering abnormal obstacles such as steel bars or hard interlayers, so as to effectively prevent the equipment from being stuck, the drill bit from being damaged or the wall from being excessively damaged, improve the operation safety, the equipment reliability and the overall construction efficiency, and improve the stability and reliability of the punching operation.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of a mechanical arm punching method according to an embodiment of the application;

[0025] Figure 2 is a flowchart of another mechanical arm punching method according to an embodiment of the application;

[0026] Figure 3 is a structural schematic diagram of a mechanical arm punching device according to an embodiment of the application;

[0027] Figure 4 is a structural schematic diagram of an electronic device for implementing the mechanical arm punching method of the application. DETAILED DESCRIPTION

[0028] In order to enable personnel in the technical field to better understand the application scheme, the technical solutions in the embodiments of the application will be clearly and completely described below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment one

[0031] Figure 1 is a flowchart of a mechanical arm punching method according to an embodiment of the present application. The present embodiment can be applied to the application scenario of automatically performing punching work according to the punching task in the tunnel. The method can be executed by a mechanical arm punching device, which can be realized in the form of hardware and / or software. The mechanical arm punching device can be configured in a computer device. As shown in Figure 1 the method comprises:

[0032] S101, based on the conversion relationship between the two-dimensional wall coordinate system and the three-dimensional camera coordinate system calibrated in advance, projecting the two-dimensional coordinates of the current punching point in the punching task into the three-dimensional point cloud collected by the three-dimensional camera to obtain the corresponding current space point coordinates of the current punching point in the three-dimensional space.

[0033] In the embodiment, the two-dimensional wall coordinate system refers to a reference coordinate system formed after two-dimensional parameterization of the tunnel wall, which is used to describe the planar positions of the drilling points in the drilling wall in the drilling task. The three-dimensional camera coordinate system refers to a spatial reference system established by the three-dimensional camera when collecting point cloud data, which is used to represent the three-dimensional spatial positions of the points in the point cloud. The three-dimensional camera is different from the ordinary two-dimensional camera which can only obtain a planar image. In addition to shooting images, the three-dimensional camera can also measure the depth information or spatial coordinates of an object. Under different camera conditions, the imaging quality and point cloud accuracy often differ. In actual applications, if affected by strong light reflection of the material or environmental light scattering, the three-dimensional camera may not be able to stably obtain complete point cloud, so that the normal information of the corresponding spatial point cannot be extracted. Even if the point cloud is successfully collected, due to noise interference or insufficiently significant local surface features, the calculated normal may deviate from the true direction, resulting in normal confusion, and the facing relationship of drilling needs to be re-determined by subsequent drilling direction correction. The current drilling point refers to the target position in the wall where the drilling operation is to be performed in the drilling task, which is given in the form of two-dimensional wall coordinates. The current spatial point coordinate refers to the specific position of the current drilling point in the actual three-dimensional space obtained by projecting the two-dimensional coordinate of the current drilling point into the point cloud data collected by the three-dimensional camera, which is used to guide the subsequent drilling direction determination and mechanical arm drilling control.

[0034] Specifically, by using the conversion relationship between the two-dimensional wall coordinate system and the three-dimensional camera coordinate system calibrated in advance, the two-dimensional coordinate of the current drilling point in the drilling task is mapped into the three-dimensional point cloud collected by the three-dimensional camera to obtain the corresponding current spatial point coordinate of the current drilling point in the three-dimensional space. The planar position is corresponded to the spatial data in the actual construction environment, and the position of the two-dimensional point in the three-dimensional point cloud is located by mathematical transformation, so as to obtain the specific coordinate of the current drilling point in the real tunnel space. The above-mentioned method aligns the two-dimensional coordinate information with the three-dimensional space, so that each drilling point has a unique and explicit three-dimensional coordinate representation in the construction environment, which provides a basis for subsequent drilling direction determination and accurate mechanical arm control.

[0035] S102, determining the drilling direction of the current spatial point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current spatial point coordinate.

[0036] In the embodiment, the drilling direction refers to the spatial direction required by the mechanical arm when performing the drilling operation for the current spatial point, which is used to ensure that the drilling is performed along the correct angle and trajectory. Specifically, based on the three-dimensional point cloud data collected by the three-dimensional camera and the obtained current spatial point coordinates, the drilling direction of the point can be determined by analyzing the positional relationship of the current spatial point in the overall structure of the wall surface or the local spatial features, the drilling task target is accurately matched with the actual three-dimensional environment, and the mechanical arm is controlled to perform drilling along the direction perpendicular to the wall surface, so that the angle and depth of the drilling meet the construction requirements. The above-mentioned method can effectively avoid the drilling deviation caused by uneven wall surface or complex spatial structure, improve the drilling accuracy and construction reliability, and provide clear direction reference for subsequent automatic operation of the mechanical arm, thereby significantly improving the efficiency and accuracy of tunnel drilling operation.

[0037] S103, controlling the mechanical arm to perform the drilling operation along the drilling direction according to the current spatial point coordinates and the preset hole depth parameter.

[0038] In the embodiment, the preset hole depth parameter refers to the drilling depth set for each drilling point in the drilling task design stage, which is used to control the feed length of the mechanical arm drilling to ensure that the drilling meets the drilling task requirements. The feed length refers to the total distance of the component performing the drilling action advancing forward along the drilling direction. The drilling operation refers to the actual action of the mechanical arm performing drilling along the determined drilling direction, including drilling, maintaining hole depth, and stopping process, which is used to complete the actual construction operation of the tunnel drilling task.

[0039] Specifically, according to the obtained current spatial point coordinates and the preset hole depth parameter, the mechanical arm is controlled to perform the drilling operation at the current spatial point coordinate position along the determined drilling direction, i.e., the mechanical arm is accurately moved to the current spatial point coordinate position in the three-dimensional space and drilled according to the designed hole depth, so that the drilling task target is accurately matched with the actual construction environment, and the drilling action is accurately performed. The above-mentioned method can ensure that the angle and depth of each hole meet the construction task requirements, avoid the deviation caused by manual operation or complex wall structure, and improve the efficiency and reliability of the drilling operation, thereby providing a stable and repeatable execution basis for automatic drilling in the tunnel.

[0040] S104, determining whether an obstacle is encountered according to the real-time collected key parts of the mechanical arm and the actual current value of the percussion drill; if an obstacle is encountered, controlling the mechanical arm to perform the gun withdrawal operation along the opposite direction of the drilling direction.

[0041] In the embodiment, the key parts of the mechanical arm refer to joints or execution units that have representative and key roles in position, posture and motion state of the mechanical arm during the execution of the drilling operation, which are used to monitor the running state of the mechanical arm and assist in judging the operation abnormality. The actual current value refers to the real-time current measurement value generated by the key parts of the mechanical arm and the impact drill due to the load change during the drilling process of the mechanical arm, which is used to reflect the resistance or abnormal situation encountered by the mechanical arm in the work. The obstacle refers to a physical object or a local abnormal area of the wall surface that may block the drilling action on the drilling path, which may cause the drilling to deviate from the designed hole position or damage the drill. The gun withdrawal operation refers to moving the mechanical arm in the opposite direction of the drilling direction when the obstacle is detected, so that the drill bit of the impact drill exits the drilled or contacted area, to avoid damaging the drill or the wall surface, and to provide conditions for subsequent adjustment of the drilling path.

[0042] Specifically, by collecting the actual current values of the key parts of the mechanical arm and the impact drill in real time, the load change borne by the mechanical arm and the drill bit during the drilling process can be monitored, so as to judge whether there is an obstacle blocking the drilling at the current position; when the obstacle is detected, the gun withdrawal operation is performed in the opposite direction of the drilling direction to make the drill bit exit the contact area to avoid damaging the wall surface or the drill, and to provide conditions for subsequent adjustment of the drilling path. Through the above-mentioned manner, unexpected obstacles occurring in the construction process can be responded in time, the safety and continuity of the drilling operation are ensured, the operation errors caused by collision or deviation are reduced, and the reliability and overall operation efficiency of the tunnel drilling are improved.

[0043] The technical scheme of the embodiment of the application establishes the accurate mapping relationship between the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system, realizes accurate projection of the two-dimensional drilling position into the three-dimensional point cloud, thereby obtaining the real and reliable drilling starting point coordinates in the three-dimensional space, and automatically determines the drilling direction consistent with the normal direction of the wall surface in combination with the point cloud data, so that the mechanical arm can stably drill in the correct posture, avoiding the hole position deviation and hole direction inclination caused by improper manual operation or uneven wall surface; the drilling process is quantitatively controlled by the pre-set hole depth parameter, to ensure the drilling depth consistency and drilling quality; through real-time monitoring of the working current of the key parts of the mechanical arm and the impact drill, the working state change can be identified in time and the gun withdrawal operation can be automatically triggered when encountering abnormal obstacles such as steel bars or hard interlayers, thereby effectively preventing equipment jamming, drill bit damage or excessive damage to the wall, improving the operation safety, equipment reliability and overall construction efficiency, and improving the stability and reliability of the drilling operation.

[0044] Embodiment Two

[0045] Figure 2 is a flowchart of another mechanical arm drilling method provided by Embodiment Two of the application. The technical scheme of the embodiment further refines the determination method of the drilling direction on the basis of the technical scheme of the above-mentioned embodiment. As shown in the figure,Figure 2 The method comprises:

[0046] S201, based on the conversion relationship between the two-dimensional wall coordinate system and the three-dimensional camera coordinate system calibrated in advance, projecting the two-dimensional coordinates of the current drilling point in the drilling task into the three-dimensional point cloud collected by the three-dimensional camera to obtain the corresponding current spatial point coordinates of the current drilling point in the three-dimensional space.

[0047] S202, acquiring the three-dimensional point cloud data collected by the three-dimensional camera and the drilling operation surface form of the drilling task.

[0048] S203, taking the drilling operation surface form as a constraint condition, determining the drilling direction of the current spatial point according to the three-dimensional point cloud data and the coordinates of the current spatial point.

[0049] In this embodiment, the drilling operation surface form refers to the geometric appearance and structural features of the target surface or local area of the processed object used for drilling operation in space, which is used to describe the shape, curved surface trend and its relationship with the surrounding structure in three-dimensional space, thereby providing form constraint basis for subsequent drilling direction determination. For example, if the laser calibration point itself has an inclination, for example, the laser device is not perpendicular to the wall, the coordinate system constructed based on this point will not be parallel to the wall. In this case, the normal direction of the three-dimensional point after projection will directly bring in the inclination error, so that the normal direction deviates from the normal direction of the target plane. If drilling is performed accordingly, the hole axis direction will be inconsistent with the preset direction, and finally an unacceptable inclined hole will be formed, so it is necessary to correct the normal relationship and the calibrated coordinate system to restore the correct drilling direction reference.

[0050] Specifically, in the process of automatic drilling of the tunnel, it is necessary to accurately determine the drilling direction of each drilling point in the complex actual wall structure. By acquiring the point cloud data collected by the three-dimensional camera and the drilling operation surface form as input, the geometric form of the wall to be drilled in the real space is represented; according to the coordinates of the current spatial point, and taking the operation surface form as a direction constraint, the drilling direction consistent with the wall form and meeting the construction requirements is derived by analyzing the spatial relationship between the point and the local wall surface structure in the point cloud, so as to ensure that the drilling direction executed by the mechanical arm is consistent with the actual wall normal or the preset structure direction. The above-mentioned method can effectively avoid the error caused by manual direction judgment, and solve the problem that the drilling direction is difficult to accurately determine when the wall is uneven, curved or has local deformation, so that the drilling direction is more accurate and the overall operation quality is significantly improved.

[0051] S204, according to the current spatial point coordinates and the preset hole depth parameter, controlling the mechanical arm to perform drilling operation along the drilling direction.

[0052] S205, according to the real-time collected key parts of the mechanical arm and the actual current value of the impact drill, it is determined whether an obstacle is encountered; if an obstacle is encountered, the mechanical arm is controlled to perform a retreat operation in the opposite direction of the punching direction.

[0053] The technical scheme of the embodiment further refines the determination method of the punching direction on the basis of the technical scheme of the above embodiment, introduces the punching operation surface form as a direction constraint on the basis of obtaining the three-dimensional point cloud data, so that the punching direction conforming to the real attitude of the wall surface to be punched can be accurately deduced in combination with the actual coordinates of the current space point, thereby effectively avoiding the punching direction deviation caused by inaccurate direction determination, improving the accuracy and execution stability of the punching direction, and making the overall punching operation have higher adaptability and reliability under different wall surface conditions.

[0054] In an optional embodiment, the punching direction of the current space point is determined according to the three-dimensional point cloud data and the coordinates of the current space point, with the punching operation surface form as a constraint condition, including: extracting the domain point cloud data of the current space point from the three-dimensional point cloud data according to the coordinates of the current space point, and calculating an initial normal vector according to the domain point cloud data; constructing a corresponding normal constraint space according to the punching operation surface form, projecting the initial normal vector to the normal constraint space for modification, and taking the modified normal vector as the punching direction of the current space point.

[0055] In the embodiment, the domain point cloud data refers to a set of neighboring points within a certain range around the current space point extracted from the three-dimensional point cloud, which is used to analyze the local wall surface structure characteristics and assist in calculating the initial normal vector. The initial normal vector refers to a vector that preliminarily represents the local direction of the wall surface at the point calculated based on the domain point cloud data of the current space point, which is used to reflect the local space orientation of the wall surface and is a preliminary basis for determining the punching direction. The normal constraint space refers to a spatial range or direction limit constructed by the punching operation surface form, which is used to limit the modified normal vector to conform to the direction requirement of the overall geometric characteristics of the wall surface, ensuring that the drilling direction is consistent with the operation surface form.

[0056] Specifically, in order to ensure that the mechanical arm can punch holes in the correct direction on different wall surfaces, the geometric characteristics of the local wall surface around the point are analyzed by extracting the field point cloud data centered on the current spatial point from the three-dimensional point cloud data, and the initial normal vector is calculated according to the geometric characteristics, which provides a preliminary reference for the punching direction; the initial normal vector is projected and corrected into the normal constraint space constructed according to the punching work surface form, so as to obtain the corrected normal vector conforming to the overall wall surface form, and the corrected normal vector is taken as the final punching direction of the current spatial point. Through the above-mentioned manner, the mechanical arm can accurately match the real attitude of the wall surface for drilling under different local wall surface conditions, effectively avoid the drilling deviation caused by local concave-convex or inclination, improve the accuracy of the punching direction determination and the stability of the work, and ensure the reliability of the drilling direction planning and execution.

[0057] In an optional embodiment, the corresponding normal constraint space is constructed according to the punching work surface form, and the initial normal vector is projected into the normal constraint space for correction, including: if the punching work surface form is a plane, a target plane equation is fitted according to a plurality of three-dimensional point cloud data in the punching region, a plane normal vector is extracted, and the initial normal vector is projected into a vertical plane of the plane normal vector to obtain a corrected normal vector; if the punching work surface form is a cylinder, a cylinder axial vector is determined, a radial vector is calculated along the axial vertical direction, and the initial normal vector is adjusted to be consistent with the direction of the radial vector.

[0058] In the embodiment, the plane normal vector refers to the normal vector of the target plane in space obtained by fitting a plurality of three-dimensional point cloud data in the punching region, which is used to represent the vertical direction of the plane surface and serves as the reference direction for correcting the initial normal vector. The vertical plane refers to a plane with the plane normal vector as the normal, which is used to project the initial normal vector onto the plane, so as to adjust the direction to conform to the geometric constraint of the target plane. The cylinder axial vector refers to a unit vector describing the direction of the central axis of the cylinder in the punching work region, which is used to determine the spatial orientation and rotation direction of the cylinder. The radial vector refers to a vector along the vertical direction of the cylinder axis, pointing from the center of the cylinder to a point on the surface, which is used to limit the punching direction to be consistent with the normal of the cylinder surface.

[0059] Specifically, in order to adapt to different geometrical punching operation surfaces, when the punching operation surface is a plane, a target plane equation is obtained by fitting a plurality of three-dimensional point cloud data in the punching area, a plane normal vector is extracted, and an initial normal vector is projected onto a vertical plane with the plane normal vector as the normal to be corrected, so as to obtain a punching direction consistent with the plane shape; when the punching operation surface is a cylinder, the axial vector of the cylinder is determined, the radial vector is calculated along the axial vertical direction, and the initial normal vector is adjusted to be consistent with the direction of the radial vector, so as to obtain a punching direction consistent with the surface characteristics of the cylinder. Through the above-mentioned manner, whether the wall surface is a plane or a cylinder, the mechanical arm can perform drilling in the correct direction consistent with the geometric characteristics of the operation surface, effectively avoiding the hole position deviation and direction error caused by the surface curvature or inclination, improving the accuracy and stability of the drilling direction determination, and enhancing the adaptability of the mechanical arm punching method in the complex tunnel environment.

[0060] In an optional embodiment, whether an obstacle is encountered is determined according to actual currents of the mechanical arm and the percussion drill collected in real time, including: if the actual current of the percussion drill exceeds a preset percussion drill current threshold and the actual current of the mechanical arm exceeds a preset mechanical arm current threshold, it is determined that an obstacle is encountered; wherein the percussion drill current threshold is determined according to a power curve of the percussion drill.

[0061] In the embodiment, the actual current of the percussion drill refers to the current value actually consumed by the percussion drill device during drilling, which is used to reflect the load state and resistance encountered by the drill bit when drilling. The preset percussion drill current threshold refers to the upper limit of the current preset according to the power curve and rated working parameters of the percussion drill, which is used to determine whether the drilling tool encounters abnormal resistance or obstacles. The actual current of the mechanical arm refers to the current value actually consumed by each key driving part of the mechanical arm during drilling, which is used to reflect the load condition of the mechanical arm when performing drilling actions. The preset mechanical arm current threshold refers to the upper limit of the current preset based on the rated parameters of the mechanical arm driving system, which is used to determine whether the mechanical arm is hindered or abnormally loaded during execution.

[0062] Specifically, in order to ensure the safety and continuity of the drilling operation of the mechanical arm, the actual current values of the percussion drill and each key part of the mechanical arm are collected in real time, which are used to determine whether there is abnormal resistance during drilling. When the actual current of the percussion drill exceeds the preset current threshold and the actual current of the mechanical arm also exceeds the corresponding threshold, it is determined that an obstacle is encountered, wherein the percussion drill current threshold is preset according to its power curve and rated working parameters. Through the above-mentioned manner, the existence of steel bars, hard interlayers or other obstacles can be identified in time during drilling, and corresponding protection measures and gun withdrawal operations are automatically triggered, so as to effectively avoid the risks of drilling tool damage, wall damage or mechanical arm jamming, and significantly improve the operation safety, equipment reliability and construction continuity.

[0063] For example, by monitoring the actual current values of the six key joints of the mechanical arm and the actual current value of the impact drill during drilling, it is determined whether the drill bit encounters abnormal resistance. When both types of current exceed their corresponding current threshold values, it is confirmed that the drill bit encounters an obstacle, thereby triggering the retreat action to avoid damage to the equipment. The current threshold value of the impact drill can be determined according to its power curve, that is, by analyzing the current change law of the motor during load rising, the characteristic point of the torque drop after the current rises to the critical value is found, and the current value corresponding to the characteristic point is set as the threshold value. The current threshold value of the mechanical arm can be calibrated through multiple drilling experiments, and the corresponding relationship between the current peak value and the drill bit hitting hard obstacles such as steel bars is identified through repeated verification, and a reliable determination reference is determined accordingly. Through the above method, the abnormal resistance can be quickly identified and processed during drilling, and the operation stability and equipment safety are improved.

[0064] In an optional embodiment, the conversion relationship is obtained by projecting a cross laser point on the wall surface by a laser device located below the three-dimensional camera on the impact drill head, taking the position of the cross laser on the wall surface as the coordinate origin of the two-dimensional wall surface coordinate system, and calibrating the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system.

[0065] Specifically, in order to establish the conversion relationship between the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system, a cross laser point is projected on the wall surface by a laser device installed on the impact drill head and located below the three-dimensional camera, and the position of the cross laser on the wall surface is taken as the coordinate origin of the two-dimensional wall surface coordinate system, thereby constructing a two-dimensional reference coordinate system for drilling task positioning. By calibrating the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system, the accurate mapping relationship between the two is determined, so that the two-dimensional drilling position can be accurately projected into the three-dimensional point cloud to obtain the real coordinates of the current drilling point in the three-dimensional space. Through the above method, the accurate correspondence between the two-dimensional coordinate information and the three-dimensional space position is realized, which effectively eliminates the drilling deviation caused by coordinate error, and also provides a reliable foundation for the subsequent automatic determination of drilling direction and execution of drilling task by the mechanical arm.

[0066] Embodiment Three

[0067] Figure 3 is a structural schematic diagram of a mechanical arm drilling device provided by Embodiment Three of the present application. The present embodiment can be applied to application scenarios in which drilling tasks need to be automatically executed in tunnels. The mechanical arm drilling device can be realized in the form of hardware and / or software, and can be configured in a computer device. As shown in the figure, the mechanical arm drilling device 300 includes: Figure 3

[0068] ​The three-dimensional space point coordinate determination module 310 is configured to project the two-dimensional coordinates of the current drilling point in the drilling task to the three-dimensional point cloud collected by the three-dimensional camera based on the conversion relationship between the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system, to obtain the corresponding current space point coordinates of the current drilling point in the three-dimensional space.

[0069] The drilling direction determination module 320 is configured to determine the drilling direction of the current space point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current space point coordinates.

[0070] The mechanical arm drilling control module 330 is configured to control the mechanical arm to perform the drilling operation along the drilling direction according to the current space point coordinates and a preset hole depth parameter.

[0071] The obstacle determination module 340 is configured to determine whether an obstacle is encountered according to the real-time collected key parts of the mechanical arm and the actual current value of the impact drill, and control the mechanical arm to perform the retreat operation in the opposite direction of the drilling direction if the obstacle is encountered.

[0072] In an optional implementation, the drilling direction determination module 320 is specifically configured to:

[0073] Obtain the three-dimensional point cloud data collected by the three-dimensional camera and the drilling work surface form of the drilling task.

[0074] Determine the drilling direction of the current space point according to the three-dimensional point cloud data and the coordinates of the current space point, with the drilling work surface form as a constraint condition.

[0075] In an optional implementation, the drilling direction determination module 320 is specifically configured to:

[0076] Extract the field point cloud data of the current space point from the three-dimensional point cloud data according to the current space point coordinates, and calculate an initial normal vector according to the field point cloud data.

[0077] Construct a corresponding normal constraint space according to the drilling work surface form, project the initial normal vector to the normal constraint space for correction, and take the corrected normal vector as the drilling direction of the current space point.

[0078] In an optional implementation, the drilling direction determination module 320 further includes an initial normal vector module, and the initial normal vector is specifically configured to:

[0079] If the drilling work surface form is a plane, fit a target plane equation according to a plurality of three-dimensional point cloud data in the drilling area, extract a plane normal vector, and project the initial normal vector to a vertical plane of the plane normal vector to obtain a corrected normal vector.

[0080] If the drilling operation surface shape is a cylinder, a cylinder axial vector is determined, a radial vector is calculated along the axial perpendicular direction, and the initial normal vector is adjusted to be consistent with the radial vector direction.

[0081] In an optional embodiment, the obstacle determination module 340 is specifically configured to:

[0082] If the actual current of the percussion drill exceeds a preset percussion drill current threshold, and the actual current of the mechanical arm exceeds a preset mechanical arm current threshold, it is determined that an obstacle is encountered; wherein the percussion drill current threshold is determined according to the power curve of the percussion drill.

[0083] In an optional embodiment, the mechanical arm drilling device 300 can also be used to:

[0084] The conversion relationship is obtained by projecting a cross laser point on the wall surface by using a laser below the three-dimensional camera on the percussion drill bit, taking the position of the cross laser on the wall surface as the coordinate origin of the two-dimensional wall surface coordinate system, constructing the two-dimensional wall surface coordinate system, and calibrating the two-dimensional wall surface coordinate system and the three-dimensional camera coordinate system.

[0085] The mechanical arm drilling device provided in the embodiments of the present application can execute the mechanical arm drilling method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0086] The embodiments of the present application also provide an electronic device, a readable storage medium and a computer program product. The computer program is stored on the computer readable storage medium, and the program is executed by the processor to realize any mechanical arm drilling method of the present application.

[0087] Embodiment four

[0088] Figure 4 is a structural schematic diagram of an electronic device for implementing the mechanical arm drilling method of the embodiments of the present application, Figure 4 A structural schematic diagram of an electronic device 410 that can be used to implement the embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (such as headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0089] As Figure 4As shown, the electronic device 410 includes at least one processor 411, and a memory, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., connected to the at least one processor 411 in communication. The memory stores computer programs executable by the at least one processor 411, and the processor 411 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 412 or loaded from the storage unit 418 into the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other through a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0090] Various components in the electronic device 410 are connected to the I / O interface 415, including an input unit 416, such as a keyboard, a mouse, etc., an output unit 417, such as various types of displays, a speaker, etc., a storage unit 418, such as a magnetic disk, an optical disk, etc., and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0091] The processor 411 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 performs various methods and processes described above, such as the robotic arm hole punching method.

[0092] In some embodiments, the robotic arm hole punching method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 418. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the robotic arm hole punching method described above can be performed. Alternatively, in other embodiments, the processor 411 can be configured to perform the robotic arm hole punching method by any other appropriate means, such as by means of firmware.

[0093] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0094] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0095] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0097] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0098] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0099] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0100] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and scope of the disclosure. Any alternatives, modifications, equivalents, and the like of all of the above described devices, systems, compositions, methods, and / or other related embodiments are intended to be encompassed by the present disclosure.

Claims

1. A mechanical arm hole punching method, characterized by, The method comprises the following steps: projecting the two-dimensional coordinates of the current drilling point in the drilling task to the three-dimensional point cloud collected by the three-dimensional camera according to the conversion relationship between the two-dimensional wall coordinate system and the three-dimensional camera coordinate system, to obtain the corresponding current spatial point coordinates of the current drilling point in the three-dimensional space; determining the drilling direction of the current spatial point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current spatial point coordinates; controlling the mechanical arm to perform the drilling operation along the drilling direction according to the current spatial point coordinates and the preset hole depth parameter; determining whether an obstacle is encountered according to the real-time collected key parts of the mechanical arm and the actual current value of the impact drill; if an obstacle is encountered, controlling the mechanical arm to perform the retreat operation in the opposite direction of the drilling direction.

2. The method of claim 1, wherein, The method for determining the drilling direction of the current spatial point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current spatial point coordinates comprises the following steps: acquiring the three-dimensional point cloud data collected by the three-dimensional camera and the drilling surface form of the drilling task; determining the drilling direction of the current spatial point according to the three-dimensional point cloud data and the current spatial point coordinates under the constraint condition of the drilling surface form.

3. The method of claim 2, wherein, The method for determining the drilling direction of the current spatial point according to the three-dimensional point cloud data and the current spatial point coordinates under the constraint condition of the drilling surface form comprises the following steps: extracting the field point cloud data of the current spatial point from the three-dimensional point cloud data according to the current spatial point coordinates, and calculating an initial normal vector according to the field point cloud data; constructing a corresponding normal constraint space according to the drilling surface form, projecting the initial normal vector to the normal constraint space for modification, and taking the modified normal vector as the drilling direction of the current spatial point.

4. The method of claim 3, wherein, The method for constructing the corresponding normal constraint space according to the drilling surface form and projecting the initial normal vector to the normal constraint space for modification comprises the following steps: if the drilling surface form is a plane, fitting a target plane equation according to a plurality of three-dimensional point cloud data in the drilling area, extracting a plane normal vector, and projecting the initial normal vector to the vertical plane of the plane normal vector to obtain the modified normal vector; if the drilling surface form is a cylinder, determining a cylinder axial vector, calculating a radial vector along the axial vertical direction, and adjusting the initial normal vector to be consistent with the direction of the radial vector.

5. The method of claim 1, wherein, The method for determining whether an obstacle is encountered according to the real-time collected actual current of the mechanical arm and the impact drill comprises the following steps: if the actual current of the impact drill exceeds a preset impact drill current threshold value, and the actual current of the mechanical arm exceeds a preset mechanical arm current threshold value, it is determined that an obstacle is encountered; wherein the impact drill current threshold value is determined according to the power curve of the impact drill.

6. The method of claim 1, wherein, The conversion relationship is obtained by projecting a cross laser point on the wall surface with a laser located on the impact drill head below the three-dimensional camera, taking the position of the cross laser on the wall surface as the coordinate origin of the two-dimensional wall coordinate system, constructing the two-dimensional wall coordinate system, and calibrating the two-dimensional wall coordinate system and the three-dimensional camera coordinate system.

7. A mechanical arm punch device characterized by, The method comprises the following steps: The three-dimensional space point coordinate determination module is configured to project the two-dimensional coordinates of the current drilling point in the drilling task to a three-dimensional point cloud collected by the three-dimensional camera based on a conversion relationship between a two-dimensional wall surface coordinate system and a three-dimensional camera coordinate system calibrated in advance, to obtain a current space point coordinate corresponding to the current drilling point in the three-dimensional space; The drilling direction determination module is configured to determine a drilling direction of the current space point according to the three-dimensional point cloud data collected by the three-dimensional camera and the current space point coordinate; The mechanical arm drilling control module is configured to control the mechanical arm to perform a drilling operation along the drilling direction according to the current space point coordinate and a preset hole depth parameter; The obstacle determination module is configured to determine whether an obstacle is encountered according to a real-time collected key part of the mechanical arm and an actual current value of the impact drill. If an obstacle is encountered, the mechanical arm is controlled to perform a retreat operation in the opposite direction of the drilling direction.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the mechanical arm drilling method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the mechanical arm drilling method in any one of claims 1-6 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program implements the mechanical arm drilling method according to any one of claims 1-6 when executed by the processor. The computer program product comprises a computer program, and the computer program implements the mechanical arm drilling method according to any one of claims 1-6 when executed by the processor.

Images