Mechanical arm cleaning control method, device and system for roadway cleaning

Abstract

The invention relates to a mechanical arm cleaning control method, device and system for roadway cleaning. The method comprises the following steps: acquiring real-time environment data of a roadway cleaning area; performing feature extraction on the environmental data, updating a three-dimensional model of the roadway cleaning area based on a feature extraction result, and positioning a cleaning target and obstacle coordinates from the updated three-dimensional model; based on the three-dimensional model and the obstacle coordinates, an improved RRT algorithm is adopted to plan a target cleaning path; and controlling the mechanical arm to perform target cleaning operation according to the target cleaning path, pausing the current cleaning operation in response to the recognized newly-added obstacle, generating a local optimization path by taking the coordinate of the newly-added obstacle as a constraint, and controlling the mechanical arm to perform obstacle avoidance or execute intermediate operation according to the local optimization path. And the mechanical arm is controlled to be connected to the target cleaning path again to complete remaining cleaning operation. According to the scheme, intelligent control over roadway cleaning operation is improved.

Description

Technical Field

[0001] This disclosure relates to the field of tunnel cleaning technology, and in particular to a robotic arm cleaning control method, device and system for tunnel cleaning. Background Technology

[0002] In related technologies, robotic arms are the core execution components of roadway cleaning robots in underground coal mine roadway cleaning operations. The level of intelligence in their cleaning control directly determines cleaning efficiency, operational accuracy, and safety performance. Currently, the control methods for roadway cleaning robotic arms have many limitations: First, cleaning route planning is mostly based on preset fixed paths, lacking adaptability to dynamic environments such as actual roadway deformation and coal and rock accumulation patterns, easily leading to cleaning omissions or repeated operations; second, data processing relies on single sensor inputs, failing to form a precise perception system that integrates multi-source data, resulting in significant positioning deviations in the cleaning area; third, path adjustment lacks real-time capability, and when faced with sudden obstacles such as exposed anchor bolts during the cleaning process, it is impossible to quickly interrupt and reconstruct the route, requiring manual intervention to continue operation; fourth, insufficient rationality in route planning easily leads to problems such as robotic arm jamming and low operational efficiency. These shortcomings make it difficult for existing robotic arm cleaning control methods to meet the needs of efficient, accurate, and autonomous cleaning in complex underground roadway environments, hindering the intelligent upgrading of roadway cleaning robots and the improvement of safety production levels. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a robotic arm cleaning control method, device and system for roadway cleaning.

[0004] According to a first aspect of the present disclosure, a robotic arm cleaning control method for roadway cleaning is provided, comprising: Acquire real-time environmental data of the alleyway cleaning area; the real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters; Feature extraction is performed on the environmental data, the three-dimensional model of the alleyway cleaning area is updated based on the feature extraction results, and the coordinates of the cleaning target and obstacles are located from the updated three-dimensional model; Based on the three-dimensional model and the coordinates of the obstacles, an improved RRT algorithm is used to plan the target cleaning path; the target cleaning path is obtained by combining the joint motion parameters of the robotic arm with the coal and rock hardness data of the roadway cleaning area for multi-constraint optimization; The robotic arm is controlled to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, the current cleaning operation is paused, and a locally optimized path is generated with the coordinates of the new obstacle as a constraint. The robotic arm is then controlled to avoid obstacles or perform intermediate operations according to the locally optimized path. Finally, the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations.

[0005] According to a second aspect of the present disclosure, a robotic arm cleaning control device for roadway cleaning is provided, comprising: The acquisition unit is used to acquire real-time environmental data of the alleyway cleaning area; the real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters. The positioning unit is used to extract features from the environmental data, update the three-dimensional model of the alleyway cleaning area based on the feature extraction results, and locate the coordinates of the cleaning target and obstacles from the updated three-dimensional model. The planning unit is used to plan the target cleaning path based on the three-dimensional model and the coordinates of the obstacles using an improved RRT algorithm; the target cleaning path is obtained by combining the joint motion parameters of the robotic arm with the coal and rock hardness data of the roadway cleaning area for multi-constraint optimization; The work unit is used to control the robotic arm to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, the current cleaning operation is paused, a locally optimized path is generated with the coordinates of the new obstacle as a constraint, the robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path, and the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations.

[0006] According to a third aspect of the present disclosure, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of the first aspects.

[0007] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0009] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: real-time environmental data of the roadway cleaning area is acquired; the real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters; features are extracted from the environmental data, the three-dimensional model of the roadway cleaning area is updated based on the feature extraction results, and the coordinates of the cleaning target and obstacles are located from the updated three-dimensional model; based on the three-dimensional model and obstacle coordinates, an improved RRT algorithm is used to plan the target cleaning path; the target cleaning path is obtained by combining the robotic arm joint motion parameters and the coal and rock hardness data of the roadway cleaning area with multiple constraints; the robotic arm is controlled to perform target cleaning operations according to the target cleaning path, and in response to the identification of new obstacles, the current cleaning operation is paused, a locally optimized path is generated with the coordinates of the new obstacles as constraints, the robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path, and the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations. Thus, through multi-source data fusion and dynamic path planning, the adaptability, efficiency, and safety of roadway cleaning operations are significantly improved, and fully autonomous intelligent control is achieved.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0012] Figure 1 This is a flowchart illustrating a robotic arm cleaning control method for tunnel cleaning according to an exemplary embodiment.

[0013] Figure 2 This is a block diagram illustrating a robotic arm cleaning control device for roadway cleaning according to an exemplary embodiment.

[0014] Figure 3 This is a block diagram illustrating an apparatus for a robotic arm cleaning control method for roadway cleaning, according to an exemplary embodiment. Detailed Implementation

[0015] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0016] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0017] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the words “if” and “suppose” as used herein may be interpreted as “when”, “when”, or “in response to a determination”.

[0018] Furthermore, various forms of processes shown in the embodiments of this disclosure can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0019] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0020] Figure 1 This is a flowchart illustrating a robotic arm cleaning control method for tunnel cleaning according to an exemplary embodiment, such as... Figure 1 As shown, it should be noted that the robotic arm cleaning control method for roadway cleaning in this embodiment of the present disclosure is applied to a robotic arm cleaning control device for roadway cleaning. For example... Figure 1 As shown, the method may include the following steps: Step 101: Obtain real-time environmental data of the alleyway cleaning area.

[0021] The real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters.

[0022] In some embodiments, real-time three-dimensional point cloud data of the roadway cleaning area can be collected by lidar to obtain spatial geometric information of coal and rock accumulation and roadway deformation; high-definition image data can be collected by industrial cameras to identify visual features such as coal and rock boundaries and exposed anchor bolts; and real-time motion parameters (such as joint angles and angular velocities) of each joint can be collected by the joint encoder of the robotic arm as the basic data for the spatial constraints of the robotic arm's motion.

[0023] As one possible implementation method, a Gaussian filtering algorithm can be used to remove noise interference from the point cloud data.

[0024] Step 102: Extract features from the environmental data, update the 3D model of the alleyway cleaning area based on the feature extraction results, and locate the coordinates of the cleaning target and obstacles from the updated 3D model.

[0025] In some embodiments of this application, step 102, which involves extracting features from environmental data and updating the 3D model of the tunnel clearing area based on the feature extraction results, may include the following steps: The Gaussian filtering algorithm was used to denoise the point cloud data, and the SLAM mapping algorithm was used to perform inter-frame feature matching and dynamic stitching to obtain the three-dimensional point cloud model of the alleyway. The image data was converted to grayscale, and the visual features of the coal and rock boundaries and exposed anchor bolts were extracted using the Canny edge detection algorithm. Visual features are fused with the 3D point cloud model of the alleyway, and the 3D model is updated based on the fusion result.

[0026] In this embodiment, the collected point cloud data and image data are processed in parallel: on the one hand, a Gaussian filtering algorithm is used to denoise the laser point cloud to eliminate interference caused by measurement errors, and a SLAM (Simultaneous Localization and Mapping) algorithm is used to perform feature matching and dynamic stitching of point clouds between frames to construct a high-precision, continuous three-dimensional point cloud model of the tunnel; on the other hand, the images collected by the camera are processed to grayscale to reduce computational complexity, and an edge detection algorithm is used to accurately extract the contour boundaries of coal and rock accumulations and the visual features of prominent obstacles such as exposed anchor bolts.

[0027] In this embodiment, visual features are calibrated and overlaid with a 3D point cloud model using data fusion technology, thereby dynamically updating the 3D model of the tunnel cleaning area. Subsequently, the updated model is sliced ​​at preset intervals to obtain the actual geometric dimensions of each section, which are then compared with the original design section of the tunnel to accurately calculate the target area requiring cleaning due to deformation or accumulation, and defined using diagonal coordinates in 3D space. Simultaneously, the fused and calibrated image information further precisely identifies the specific spatial coordinates of obstacles such as exposed anchor bolts, ultimately outputting a quantified environmental model containing the cleaning target area and the precise locations of obstacles, providing a solid foundation for subsequent intelligent path planning.

[0028] In some embodiments of this application, locating the coordinates of the target and obstacles from the updated 3D model in step 102 may include the following steps: The 3D model is sliced ​​at preset intervals to obtain multiple cross sections, and the actual dimensions and geometry of each cross section are obtained. The actual dimensions of each section are compared with the original design section data of the roadway to calculate the three-dimensional coordinate range of the roadway deformation and coal and rock accumulation area, and the target area for cleaning is determined in the form of diagonal point coordinates. Based on features extracted from image data, coordinate calibration is performed on the boundaries of the coal and rock accumulation area and the spatial location of exposed anchor bolts to determine the coordinates of obstacles.

[0029] In one embodiment, the constructed 3D model can be sliced ​​at preset intervals (e.g., 0.5 meters) to obtain a series of continuous roadway cross-sections, and the actual width, height, and geometric contour of each cross-section can be accurately obtained. Then, the actual dimensions of these cross-sections are automatically compared with the standard cross-section CAD data of the original roadway design. The areas where deformation occurs and coal and rock accumulation exists are identified through coordinate difference calculation. Then, the target area to be cleared is accurately defined in the form of diagonal coordinates in 3D space. Finally, the visual features of coal and rock boundaries and anchor bolts extracted from image data are deeply integrated. The calculated boundaries of the accumulation area and the spatial positions of the identified exposed anchor bolts and other obstacles are cross-validated and calibrated at the pixel level. Finally, the precise quantitative coordinates of the clearing target area and the coordinates of the obstacles are output, laying a precise environmental perception foundation for subsequent path planning.

[0030] Step 103: Based on the 3D model and obstacle coordinates, an improved RRT algorithm is used to plan the target clearing path.

[0031] The target cleaning path is obtained by optimizing the robotic arm's joint motion parameters and the coal and rock hardness data of the roadway cleaning area under multiple constraints.

[0032] In one embodiment, a 3D model and obstacle coordinates can be used as environmental constraints. An improved RRT (Rapidly-exploring Random Tree) algorithm is employed to quickly search within the robotic arm's motion space, generating an initial path that covers the entire target area. Furthermore, path planning is not completed in one step but involves crucial multi-constraint optimization: a motion constraint matrix is ​​formed by combining the robotic arm's joint motion parameters; simultaneously, the coal and rock hardness data obtained from the reflection intensity inversion of LiDAR point clouds is quantified and converted into speed adjustment coefficients at path nodes. Finally, the initial path is smoothed using Bézier curves to eliminate abrupt changes, and differentiated operating speeds are adaptively assigned to different sections (such as hard rock areas and soft coal areas) based on hardness data, thereby generating an optimal target cleaning path that balances operational efficiency, motion smoothness, equipment safety, and cleaning effectiveness.

[0033] In one embodiment, the RRT algorithm can plan motion paths for a robotic arm in a high-dimensional, dynamic roadway cleaning environment. Through random sampling and tree-based expansion, it rapidly explores feasible paths within a complex three-dimensional space containing obstacles (such as coal and rock deposits and exposed anchor bolts). It can also optimize these paths by incorporating multiple factors, including robotic arm joint motion constraints and coal and rock hardness, to generate a target cleaning path that balances efficiency and safety. Furthermore, when the system identifies new obstacles in real time, RRT can quickly replan the local path using the obstacle's coordinates as constraints, achieving dynamic obstacle avoidance and seamless workflow integration. This significantly improves the autonomy, adaptability, and intelligence of roadway cleaning operations.

[0034] For example, the specific process of using RRT for path planning is as follows: First, a 3D model of the tunnel is constructed and dynamically updated based on real-time acquired point cloud and image data to accurately locate the coordinates of the cleaning target and obstacles. Then, using this model and obstacle coordinates as spatial constraints, an improved RRT algorithm is used to randomly sample and expand the tree structure in the movement space of the robotic arm, quickly searching and generating an initial path that can cover the entire cleaning area. On this basis, a motion constraint matrix is ​​formed by combining the joint motion parameters of the robotic arm (such as angle and speed limits). At the same time, based on the coal and rock hardness data obtained by inverting the reflection intensity of the lidar point cloud, it is quantified into the speed adjustment coefficient of each path node. Then, the initial path nodes are smoothly fitted by Bézier curves to eliminate sharp turns and abrupt changes. Based on the hardness coefficient, different operating speeds are assigned to path nodes in different areas (such as hard rock areas and soft coal areas) point by point, finally generating an optimized cleaning path that links position and speed, takes into account operating efficiency, smoothness of movement, and equipment safety. Furthermore, when a new obstacle is detected in real time, the obstacle coordinates are used as constraints to generate a local obstacle avoidance or operation path in real time using the fast local RRT algorithm. After completion, the path is smoothly reconnected to the original target path, thereby achieving adaptive planning and continuous operation in dynamic environments.

[0035] In some embodiments of this application, step 103 may include the following steps: The motion limit angles and angular velocity thresholds of each joint of the robotic arm, as well as the working range of the end effector, are extracted from the joint motion parameters of the robotic arm to form a joint motion constraint matrix; Based on the reflection intensity value of lidar point cloud, the coal and rock hardness of the roadway clearing area is inverted through grayscale mapping algorithm and quantified into the speed adjustment coefficient of path nodes. The Bezier curve fitting algorithm is used to smooth the initial path nodes planned by the improved RRT algorithm to eliminate sharp turns and abrupt changes. Based on the speed adjustment coefficient, the operation speed is assigned to each smoothed path node to generate a target cleaning path that is linked to speed and position; the target cleaning path includes multiple path nodes.

[0036] In this embodiment, the reflection intensity information contained in the lidar point cloud data can be utilized. Because it has a physical correlation with the physical properties of the medium surface (such as hardness and density), the average reflection intensity value of each point or region can be converted into the corresponding coal and rock hardness level by using a grayscale mapping algorithm based on the pre-calibrated correspondence between reflection intensity and coal and rock hardness. Subsequently, this hardness level is quantified into a specific speed adjustment coefficient according to preset rules. For example, high-hardness rock strata areas are mapped to a deceleration coefficient of 0.6-0.8 to ensure crushing effect and equipment safety, while low-hardness coal and rock areas are mapped to a speed-increasing coefficient of 1.2-1.5 to optimize operation efficiency. Finally, this coefficient is associated with the corresponding spatial coordinate node in the path planning module, thereby realizing refined and adaptive control of the operating speed of the robotic arm end effector.

[0037] In this embodiment, key nodes in the RRT path can be used as control points of the Bézier curve. By adjusting the curve order and the weight of the control points, a smooth trajectory with continuous curvature that passes through all key areas is generated. This effectively eliminates sharp turns and abrupt changes in the original path, avoiding joint impacts and jamming caused by sudden changes in direction of the robotic arm. It also ensures that the path meets the kinematic constraints of the robotic arm, ultimately forming an optimized operation path that conforms to the joint motion characteristics and can be stably tracked by the robotic arm.

[0038] In some embodiments of this application, step 103, which assigns a working speed to each smoothed path node based on a speed adjustment coefficient to generate a target cleaning path that links speed and position, may include the following steps: Multiply the speed adjustment coefficient by a preset base operating speed to obtain the actual operating speed at each path node; Specifically, for path nodes identified as hard rock areas, a speed adjustment coefficient of less than 1 is assigned to reduce the operating speed; for path nodes identified as soft coal areas, a speed adjustment coefficient of greater than 1 is assigned to increase the operating speed.

[0039] In one embodiment of this application, the aforementioned speed adjustment coefficient can be multiplied by a preset base operating speed to calculate the accurate actual operating speed for each smoothed path node. This process establishes a complete "position-speed" linkage mapping relationship. For path nodes identified as hard rock areas, the system assigns a speed adjustment coefficient less than 1 (e.g., 0.6) to reduce the operating speed and ensure crushing effect and equipment safety. For path nodes identified as soft coal areas, a coefficient greater than 1 (e.g., 1.2) is assigned to increase the operating speed and optimize overall efficiency. Finally, an optimal target cleaning path with a customized operating speed for each spatial location point is generated.

[0040] Step 104: Control the robotic arm to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, pause the current cleaning operation, generate a locally optimized path with the coordinates of the new obstacle as a constraint, control the robotic arm to avoid obstacles or perform intermediate operations according to the locally optimized path, and control the robotic arm to reconnect to the target cleaning path to complete the remaining cleaning operations.

[0041] In one embodiment of this application, while controlling the robotic arm to strictly follow the target cleaning path, environmental changes can be continuously monitored through the radar-visual fusion sensing module. When a new obstacle (such as a newly exposed anchor bolt) is identified in the real-time data stream, a safety response mechanism is immediately triggered—the current cleaning operation is paused, and a local optimized path containing obstacle avoidance trajectory or special operation (such as anchor bolt shearing) is generated in real time using the three-dimensional coordinates of the new obstacle as a spatial constraint and the fast local RRT algorithm. After the robotic arm completes the execution of the local path, its motion trajectory is smoothly transitioned and accurately reconnected to the breakpoint of the original target cleaning path through the path connection algorithm, thereby achieving a complete cleaning task closed loop without human intervention while ensuring equipment safety and operation continuity.

[0042] In some embodiments of this application, step 104, in response to the detection of a new obstacle, pausing the current clearing operation and generating a locally optimized path with the coordinates of the new obstacle as a constraint, may include the following steps: It receives feedback data from the joint encoder and monitoring data from the Rave vision sensing module in real time. Clustering analysis or target detection algorithms are used on newly added point cloud data and image data to quickly identify the boundary coordinates or shear point coordinates of newly added obstacles and pause the current cleanup operation. Using the identified coordinates as new constraints, a fast local RRT algorithm is employed to generate locally optimized paths for obstacle avoidance or operation.

[0043] In one embodiment, real-time feedback data (including motion state parameters such as joint angle, angular velocity, and acceleration) from encoders at each joint of the robotic arm can be continuously and synchronously collected via industrial fieldbus and Ethernet protocol, as well as monitoring data from the laser-vision fusion perception module (including real-time point cloud streams from the lidar and continuous image frames from the camera). These two types of data are transmitted in parallel to the central processing unit, forming a synchronous perception of the robotic arm's own motion state and the external working environment, providing a continuously updated data foundation for subsequent real-time feature extraction, dynamic obstacle recognition, and path replanning.

[0044] In this embodiment, clustering analysis algorithms can be used to spatially segment the newly acquired point cloud data in real time to identify point cloud clusters independent of the modeled environment. At the same time, target detection algorithms such as YOLO are used to analyze the synchronously acquired image frames in real time. Through cross-validation of the two perception modalities, the precise three-dimensional boundary coordinates of newly added obstacles (such as newly exposed anchor rods) or their key operation points (such as shearing point coordinates) can be quickly identified. When the identification confidence reaches a preset threshold, the system immediately sends an interrupt command to the motion control module to suspend the current cleaning operation of the robotic arm, providing a safe decision-making and execution window for subsequent local path replanning.

[0045] In this embodiment, the coordinates of newly identified obstacles identified in real time can be used as key spatial constraints. In the configuration space near the current working breakpoint of the robotic arm, a fast local RRT algorithm is used for efficient path search. In the local state space formed by the breakpoint and target point of the original global path, by introducing obstacle coordinates as a sampling exclusion domain, a feasible path that can avoid collisions and complete specific tasks (such as anchor bolt shearing) can be quickly generated. By dynamically adjusting the sampling strategy and step size parameters, while ensuring real-time performance, the generated local path can be ensured to meet the kinematic constraints of the robotic arm. Finally, a locally optimized path that is consistent with the global path coordinate system and can be executed immediately is output.

[0046] According to the robotic arm cleaning control method for roadway cleaning proposed in this disclosure, real-time environmental data of the roadway cleaning area is acquired. The real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters. Feature extraction is performed on the environmental data, and the three-dimensional model of the roadway cleaning area is updated based on the feature extraction results. The coordinates of the cleaning target and obstacles are located from the updated three-dimensional model. Based on the three-dimensional model and obstacle coordinates, an improved RRT algorithm is used to plan the target cleaning path. The target cleaning path is obtained by combining the robotic arm joint motion parameters and the coal and rock hardness data of the roadway cleaning area with multiple constraints. The robotic arm is controlled to perform target cleaning operations according to the target cleaning path. In response to the identification of a new obstacle, the current cleaning operation is paused, and a locally optimized path is generated with the coordinates of the new obstacle as a constraint. The robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path. The robotic arm is then controlled to reconnect to the target cleaning path to complete the remaining cleaning operations. Thus, through multi-source data fusion and dynamic path planning, the adaptability, efficiency, and safety of roadway cleaning operations are significantly improved, and fully autonomous intelligent control is achieved.

[0047] Figure 2 This is a block diagram illustrating a robotic arm cleaning control device for tunnel cleaning according to an exemplary embodiment. (Refer to...) Figure 2 The device includes an acquisition unit 201, a positioning unit 202, a planning unit 203, and a working unit 204.

[0048] The acquisition unit 201 is used to acquire real-time environmental data of the tunnel cleaning area; the real-time environmental data includes point cloud data, image data and robotic arm joint motion parameters. The positioning unit 202 is used to extract features from environmental data, update the three-dimensional model of the alleyway cleaning area based on the feature extraction results, and locate the coordinates of the cleaning target and obstacles from the updated three-dimensional model. Planning unit 203 is used to plan the target clearing path based on the 3D model and obstacle coordinates, using an improved RRT algorithm. The target clearing path is obtained by combining the robotic arm joint motion parameters and the coal and rock hardness data of the roadway clearing area for multi-constraint optimization. The work unit 204 is used to control the robotic arm to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, the current cleaning operation is paused, a locally optimized path is generated with the coordinates of the new obstacle as a constraint, the robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path, and the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations.

[0049] In some embodiments of this application, the positioning unit 202 may specifically be used for: The Gaussian filtering algorithm was used to denoise the point cloud data, and the SLAM mapping algorithm was used to perform inter-frame feature matching and dynamic stitching to obtain the three-dimensional point cloud model of the alleyway. The image data was converted to grayscale, and the visual features of the coal and rock boundaries and exposed anchor bolts were extracted using the Canny edge detection algorithm. Visual features are fused with the 3D point cloud model of the alleyway, and the 3D model is updated based on the fusion result.

[0050] In some embodiments of this application, the positioning unit 202 may specifically be used for: The 3D model is sliced ​​at preset intervals to obtain multiple cross sections, and the actual dimensions and geometry of each cross section are obtained. The actual dimensions of each section are compared with the original design section data of the roadway to calculate the three-dimensional coordinate range of the roadway deformation and coal and rock accumulation area, and the target area for cleaning is determined in the form of diagonal point coordinates. Based on features extracted from image data, coordinate calibration is performed on the boundaries of the coal and rock accumulation area and the spatial location of exposed anchor bolts to determine the coordinates of obstacles.

[0051] In some embodiments of this application, the planning unit 203 may specifically be used for: The motion limit angles and angular velocity thresholds of each joint of the robotic arm, as well as the working range of the end effector, are extracted from the joint motion parameters of the robotic arm to form a joint motion constraint matrix; Based on the reflection intensity value of lidar point cloud, the coal and rock hardness of the roadway clearing area is inverted through grayscale mapping algorithm and quantified into the speed adjustment coefficient of path nodes. The Bezier curve fitting algorithm is used to smooth the initial path nodes planned by the improved RRT algorithm to eliminate sharp turns and abrupt changes. Based on the speed adjustment coefficient, the operation speed is assigned to each smoothed path node to generate a target cleaning path that is linked to speed and position; the target cleaning path includes multiple path nodes.

[0052] In some embodiments of this application, the planning unit 203 may specifically be used for: Multiply the speed adjustment coefficient by a preset base operating speed to obtain the actual operating speed at each path node; Specifically, for path nodes identified as hard rock areas, a speed adjustment coefficient of less than 1 is assigned to reduce the operating speed; for path nodes identified as soft coal areas, a speed adjustment coefficient of greater than 1 is assigned to increase the operating speed.

[0053] In some embodiments of this application, the work unit 204 may specifically be used for: It receives feedback data from the joint encoder and monitoring data from the Rave vision sensing module in real time. Clustering analysis or target detection algorithms are used on newly added point cloud data and image data to quickly identify the boundary coordinates or shear point coordinates of newly added obstacles and pause the current cleanup operation. Using the identified coordinates as new constraints, a fast local RRT algorithm is employed to generate locally optimized paths for obstacle avoidance or operation.

[0054] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0055] According to the embodiments of this disclosure, a robotic arm cleaning control device for roadway cleaning acquires real-time environmental data of the roadway cleaning area. This real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters. Feature extraction is performed on the environmental data, and the 3D model of the roadway cleaning area is updated based on the feature extraction results. The coordinates of the cleaning target and obstacles are located from the updated 3D model. Based on the 3D model and obstacle coordinates, an improved RRT algorithm is used to plan the target cleaning path. The target cleaning path is obtained by optimizing the robotic arm joint motion parameters and the coal and rock hardness data of the roadway cleaning area using multiple constraints. The robotic arm is controlled to perform target cleaning operations according to the target cleaning path. In response to the identification of a new obstacle, the current cleaning operation is paused, and a locally optimized path is generated using the coordinates of the new obstacle as a constraint. The robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path. Finally, the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations. Thus, through multi-source data fusion and dynamic path planning, the adaptability, efficiency, and safety of roadway cleaning operations are significantly improved, achieving fully autonomous and intelligent control.

[0056] Figure 3 This is a block diagram illustrating an apparatus for a robotic arm cleaning control method for tunnel cleaning, according to an exemplary embodiment. For example, apparatus 300 may be an electronic device, such as a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0057] Reference Figure 3 The device 300 may include one or more of the following components: a processing component 302, a memory 304, a power component 306, a multimedia component 308, an audio component 310, an input / output (I / O) interface 312, a sensor component 314, and a communication component 316.

[0058] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0059] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0060] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0061] Multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0062] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0063] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0064] Sensor assembly 314 includes one or more sensors for providing status assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0065] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0066] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0067] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0068] In an exemplary embodiment, a computer program product is also provided, including a computer program that implements the above-described method when executed by the processor 320 of the device 300.

[0069] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0070] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A robotic arm cleaning control method for roadway cleaning, characterized in that, include: Acquire real-time environmental data of the alleyway cleaning area; the real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters; Feature extraction is performed on the environmental data, the three-dimensional model of the alleyway cleaning area is updated based on the feature extraction results, and the coordinates of the cleaning target and obstacles are located from the updated three-dimensional model; Based on the 3D model and the obstacle coordinates, an improved fast exploration random tree (RRT) algorithm is used to plan the target clearing path. The target clearing path is obtained by combining the robotic arm joint motion parameters with the coal and rock hardness data of the roadway clearing area for multi-constraint optimization. The robotic arm is controlled to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, the current cleaning operation is paused, and a locally optimized path is generated with the coordinates of the new obstacle as a constraint. The robotic arm is then controlled to avoid obstacles or perform intermediate operations according to the locally optimized path. Finally, the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations.

2. The robotic arm cleaning control method for roadway cleaning according to claim 1, characterized in that, Feature extraction is performed on the environmental data, and the 3D model of the tunnel clearing area is updated based on the feature extraction results, specifically including: The point cloud data is denoised using a Gaussian filtering algorithm, and inter-frame feature matching and dynamic stitching are performed using a SLAM mapping algorithm to obtain a three-dimensional point cloud model of the alleyway. The image data was converted to grayscale, and the visual features of the coal and rock boundary and exposed anchor bolts were extracted using the Canny edge detection algorithm. The visual features are fused with the three-dimensional point cloud model of the alleyway, and the three-dimensional model is updated based on the fusion result.

3. The robotic arm cleaning control method for roadway cleaning according to claim 1, characterized in that, Locate the coordinates of the target and obstacles to be cleared from the updated 3D model, including: The three-dimensional model is sliced ​​at a preset interval to obtain multiple cross-sections, and the actual size and geometry of each cross-section are obtained. The actual dimensions of each section are compared with the original design section data of the roadway to calculate the three-dimensional coordinate range of the roadway deformation and coal and rock accumulation area, and the target area for cleaning is determined in the form of diagonal point coordinates. Based on the features extracted from the image data, the coordinates of the boundary of the coal and rock accumulation area and the spatial position of the exposed anchor bolts are calibrated to determine the coordinates of the obstacle.

4. The robotic arm cleaning control method for roadway cleaning according to claim 1, characterized in that, The step of planning the target clearing path using the improved RRT algorithm based on the 3D model and the obstacle coordinates includes: The motion limit angles and angular velocity thresholds of each joint of the robotic arm, as well as the working range of the end effector, are extracted from the joint motion parameters of the robotic arm to form a joint motion constraint matrix; Based on the reflection intensity value of lidar point cloud, the coal and rock hardness of the roadway clearing area is inverted through grayscale mapping algorithm and quantified into the speed adjustment coefficient of path nodes. The Bezier curve fitting algorithm is used to smooth the initial path nodes planned by the improved RRT algorithm to eliminate sharp turns and abrupt changes. Based on the speed adjustment coefficient, the operation speed is assigned to each smoothed path node to generate the target cleaning path that is linked to speed and position; the target cleaning path includes multiple path nodes.

5. The robotic arm cleaning control method for roadway cleaning according to claim 4, characterized in that, The step of assigning a work speed to each smoothed path node according to the speed adjustment coefficient to generate the target cleaning path that is linked to speed and position includes: The speed adjustment coefficient is multiplied by a preset base operating speed to obtain the actual operating speed at each path node; Specifically, for path nodes identified as hard rock areas, a speed adjustment coefficient of less than 1 is assigned to reduce the operating speed; for path nodes identified as soft coal areas, a speed adjustment coefficient of greater than 1 is assigned to increase the operating speed.

6. The robotic arm cleaning control method for roadway cleaning according to claim 1, characterized in that, The step of pausing the current clearing operation in response to the detection of a new obstacle and generating a locally optimized path with the coordinates of the new obstacle as a constraint includes: It receives feedback data from the joint encoder and monitoring data from the Rave vision sensing module in real time. Clustering analysis or target detection algorithms are used on newly added point cloud data and image data to quickly identify the boundary coordinates or shear point coordinates of newly added obstacles and pause the current cleanup operation. Using the identified coordinates as new constraints, the Fast Local RRT algorithm is employed to generate the locally optimized path for obstacle avoidance or operation.

7. A robotic arm cleaning control device for roadway cleaning, characterized in that, include: The acquisition unit is used to acquire real-time environmental data of the tunnel cleaning area; The real-time environmental data includes point cloud data, image data, and robotic arm joint motion parameters; The positioning unit is used to extract features from the environmental data, update the three-dimensional model of the alleyway cleaning area based on the feature extraction results, and locate the coordinates of the cleaning target and obstacles from the updated three-dimensional model. The planning unit is used to plan the target clearing path based on the three-dimensional model and the obstacle coordinates, using an improved RRT algorithm. The target cleaning path is obtained by combining the joint motion parameters of the robotic arm with the coal and rock hardness data of the roadway cleaning area through multi-constraint optimization. The work unit is used to control the robotic arm to perform target cleaning operations according to the target cleaning path. In response to the detection of a new obstacle, the current cleaning operation is paused, a locally optimized path is generated with the coordinates of the new obstacle as a constraint, the robotic arm is controlled to avoid obstacles or perform intermediate operations according to the locally optimized path, and the robotic arm is controlled to reconnect to the target cleaning path to complete the remaining cleaning operations.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images