Intelligent water cutting explosive-handling robot system and method

Abstract

The invention provides an intelligent water cutting explosive ordnance disposal robot system and method, and the system comprises a walking platform and a water cutting system which comprises a water tank, a water jet generator and a nozzle which are communicated in sequence; the tail end operation mechanism comprises a mechanical arm, and the mechanical arm is connected with the nozzle; the multi-dimensional sensing device comprises an all-round camera shooting sensor, a monitoring sensor, a depth camera and a laser scanner; the control module is connected with the walking platform, the water cutting system, the tail end operation mechanism and the multi-dimensional sensing device. And the remote control station is in remote communication connection with the control module. Through the system provided by the invention, the problem that the prior art and method cannot meet the high-precision explosive-handling operation requirements for unknown environments and target objects is solved.

Description

Technical Field

[0001] The present application relates to the technical field of explosive ordnance disposal robots, and in particular to an intelligent water jet explosive ordnance disposal robot system and method. Background Art

[0002] At present, mobile bomb disposal robots have received widespread attention due to their remote, safe, and flexible features. Bomb disposal robots can replace bomb disposal personnel to enter dangerous areas to conduct reconnaissance, transfer, dismantling, and destruction of explosive devices or weapons. Usually, bomb disposal robots include wheeled or tracked mobile chassis, camera monitoring equipment, manipulators such as grippers and tools, and remote communication and remote control equipment. Due to the complexity of the operating scene, the diversity of the operating objects, and the uncertainty of the bomb disposal targets, it is difficult to use one robot to complete all tasks. In recent years, a variety of bomb disposal robots or their ancillary mechanisms or monitoring and operating devices have been continuously invented to solve various difficult problems in bomb disposal operations.

[0003] Due to the particularity of the objects and environment of bomb disposal operations, water jet cutting (also known as water knife), which is a cold cutting method, does not produce thermal deformation or thermal effects. It has good cutting effects on high-hardness materials such as glass, ceramics, metals, etc., as well as soft materials such as leather, rubber, cloth, etc. Therefore, it has a good application prospect for operations on target objects such as shelters, containers, and explosives. At present, there are inventions that apply it to bomb disposal robots. However, water jet cutting robots based on fixed platforms or bases can only perform predefined operations on specific targets or objects, and cannot adapt to the operation requirements under uncertain scenes or targets. Although water jet cutting robot devices based on mobile platforms or vehicles have a certain degree of freedom, they are still based on proprietary scenes and operation objects, and do not effectively use high-precision detection methods to assist in automated operations, which seriously limits the operation capabilities of bomb disposal robots in complex scenes.

[0004] Therefore, existing bomb disposal robots are mostly based on proprietary scenarios and operating objects, lacking adaptability to unknown environments and intelligent high-precision operating capabilities. Summary of the invention

[0005] In view of the above problems, one of the purposes of the present invention is to provide an intelligent water jet bomb disposal robot system and method to solve the problem that the existing technology cannot meet the needs of high-precision bomb disposal operations in unknown environments and target objects.

[0006] In the first aspect, the present invention provides an intelligent water jetting bomb disposal robot system, which adopts the following technical solution: An intelligent water jetting bomb disposal robot system, the system comprising: a walking platform and: The water cutting system comprises a water tank, a water jet generator and a nozzle which are connected in sequence; a terminal operation mechanism comprises a mechanical arm which is connected to the nozzle; a multi-dimensional sensing device comprises a surround camera sensor, a monitoring sensor, a depth camera and a laser scanner; a control module which is respectively connected to the walking platform, the water cutting system, the terminal operation mechanism and the multi-dimensional sensing device; and a remote control station which is remotely connected to the control module in communication; Wherein, the remote control station is used for: Acquire the surround view image collected by the surround view camera sensor to determine the walking direction of the walking platform; Remotely send a walking instruction to control the walking platform to move to the working area according to the walking direction; Acquire the front image collected by the monitoring sensor to determine the target object located in the operation area; Acquire a three-dimensional image of the target object acquired by the depth camera; In response to a scanning instruction for a target area of ​​the three-dimensional image, acquiring a three-dimensional point cloud image obtained by scanning the target area with the laser scanner; In response to a cutting instruction for a cutting center of the three-dimensional point cloud image, a preset operation trajectory with the cutting center as a starting point is obtained; The robot arm is controlled to transfer the nozzle to the cutting center, and the nozzle is controlled to cut the target object along the preset operation trajectory, while obtaining the front image collected by the monitoring sensor during the cutting process.

[0007] As one of the preferred solutions, at least one of the surround camera sensors is arranged on at least one side of the walking platform, and the monitoring sensor is arranged in front of the walking platform; The laser scanner and the depth camera are installed at a first direction of the end of the robotic arm, and the nozzle is installed at a second direction of the end of the robotic arm, and the first direction and the second direction intersect.

[0008] As one of the preferred solutions, the remote control station includes a robot arm simulation module, which is configured to simulate whether the robot arm collides with the target object when moving along different preset trajectories, and send a motion instruction to the control module to control the robot arm to move along the preset trajectory when there is no collision; wherein the motion instruction includes a first motion instruction, a second motion instruction and a third motion instruction; The robotic arm simulation module is used for: Sending the first motion instruction to control the robotic arm to flip along a first preset trajectory to obtain the three-dimensional image; Sending the second motion instruction to control the robotic arm to move along a second preset trajectory to obtain the three-dimensional point cloud image; The third motion instruction is sent to control the robot arm to cut the target object along the preset operation trajectory.

[0009] As one of the preferred solutions, the multi-dimensional sensing device further includes a hose telescopic endoscope arranged in the first direction of the mechanical arm, and the remote control station is further used for: After the cutting of the target object is completed, the flexible tube telescopic endoscope is inserted into the cut hole, the inspection information collected by the flexible tube telescopic endoscope is obtained, and the cutting result is output; and / or, After the cutting of the target object is completed, the nozzle at the end of the robot arm is controlled to flip, driving the depth camera to face the cut hole, obtaining a three-dimensional image captured by the depth camera, and outputting the cutting result.

[0010] As one of the preferred solutions, the walking platform includes a mobile chassis, and the mobile chassis is also provided with telescopic legs on all four sides; Wherein, the control module is configured with a horizontal sensor, and the control module is connected to each of the telescopic legs to control the telescopic height of each of the telescopic legs.

[0011] In the second aspect, the present invention provides an intelligent water jetting bomb disposal robot operation method, the technical solution adopted is: An intelligent water jetting bomb disposal robot operation method, the method comprising: Obtain the surround images collected by the surround camera sensor to determine the walking direction of the walking platform; Remotely send a walking instruction to control the walking platform to move to the working area according to the walking direction; Acquire the front image collected by the monitoring sensor to determine the target object located in the operation area; Acquire a three-dimensional image of the target object acquired by a depth camera; In response to a scanning instruction for a target area of ​​the three-dimensional image, acquiring a three-dimensional point cloud image obtained by scanning the target area with a laser scanner; In response to a cutting instruction for a cutting center of the three-dimensional point cloud image, a preset operation trajectory with the cutting center as a starting point is obtained; The robot arm is controlled to transfer the nozzle to the cutting center, and the nozzle is controlled to cut the target object according to the preset operation trajectory, and at the same time, the front image collected by the monitoring sensor during the cutting process is obtained.

[0012] As one of the preferred solutions, the step of acquiring the front image collected by the monitoring sensor and determining the target object located in the operation area includes: Acquire the front image captured by the monitoring sensor and the two-dimensional image captured by the depth camera to determine the position information of the target object located in the working area; Based on the position information, the walking platform is controlled to move to a target position.

[0013] As one of the preferred solutions, obtaining a three-dimensional image of the target object acquired by the depth camera includes: Sending a first motion instruction to control the robotic arm to flip along a first preset trajectory; the first motion instruction is characterized by simulating that the robotic arm does not collide with the target object when flipping along the first preset trajectory; Acquire a three-dimensional image of the target object acquired by the depth camera; The step of acquiring a three-dimensional point cloud image obtained by scanning the target area with the laser scanner includes: Sending a second motion instruction to control the robotic arm to move along a second preset trajectory within the target area; the second motion instruction is characterized by simulating that the robotic arm does not collide with the target object when moving along the second preset trajectory; Acquire a three-dimensional point cloud image obtained by scanning the target object in the target area by the laser scanner moving with the mechanical arm; The controlling the nozzle to cut the target object along the preset operation trajectory includes: A third motion instruction is sent to control the robot arm to move along the preset operation trajectory in the target area; the third motion instruction is characterized by simulating that the robot arm does not collide with the target object when moving along the preset operation trajectory.

[0014] As one of the preferred solutions, the step of controlling the walking platform to move to the target position based on the position information further includes: Obtaining level information of the walking platform to determine whether the walking platform is level; When the walking platform is not horizontal, controlling the plurality of telescopic legs to extend at their respective corresponding telescopic heights; When the walking platform is horizontal, the plurality of telescopic legs are controlled to extend at the same preset height.

[0015] As one of the preferred solutions, the controlling the robot arm to transfer the nozzle to the cutting center and controlling the nozzle to cut the target object along the preset operation trajectory, then includes: After the cutting of the target object is completed, the robot arm is controlled to drive the flexible tube telescopic endoscope to extend into the cut hole, obtain the inspection information collected by the flexible tube telescopic endoscope, and output the cutting result; and / or, After the cutting of the target object is completed, the nozzle at the end of the robot arm is controlled to flip, driving the depth camera to face the cut hole, obtaining a three-dimensional image captured by the depth camera, and outputting the cutting result.

[0016] Compared with the prior art, this application has the following advantages: The system provided by the embodiment of the present application provides a wide field of view through the surround camera sensor and the monitoring sensor to determine the position and direction of travel of the target object. The depth camera performs a rough scan to quickly generate a large range of three-dimensional images, determine the general outline of the target object, lock the position, distance and size of the target object, plan the working space and anti-collision boundary of the subsequent laser scanner, and provide the target area scanning range for the laser scanner. The laser scanner can capture details with high precision in a small range, perform fine scanning in the target area calibrated by the depth camera, and obtain an accurate three-dimensional model of the target object through local fine scanning, thereby ensuring accurate analysis and cutting motion planning of the target object. In this way, the robot is provided with good visual sensing for travel and operation through panoramic camera and omnidirectional vision, so that the robot can obtain multi-dimensional visual information in a complex and unknown environment, and realize dynamic, all-round target recognition and fine operation positioning. In addition, the functions of multiple modules in different dimensions are coordinated to realize the gradual refinement of the perception process, so that the collection of subsequent modules can be more centralized and efficient, and the overall operation efficiency and safety are improved. The multi-dimensional perception device enables the robot to have environmental perception capabilities from wide area to microscopic when performing tasks, effectively improving the accuracy and flexibility of recognition, positioning and operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a schematic diagram of the overall structure of the intelligent water jetting bomb disposal robot system according to an embodiment of the present application; Figure 2 It is a step flow chart of the operation method of the intelligent water jet bomb disposal robot described in one embodiment of the present application.

[0019] Description of reference numerals: 1. Mobile chassis; 2. Support frame; 3. Telescopic legs; 4. Water tank; 5. Control module; 6. Surround view camera sensor; 7. Surveillance sensor; 8. Water jet generator; 9. Robotic arm; 10. Nozzle; 11. Laser scanner; 12. Depth camera; 13. Hose telescopic endoscope; 14. Remote control station. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] Explosive ordnance disposal operations are usually carried out in dangerous and complex environments, usually with various obstacles and irregular terrains. The types of explosive ordnance disposal targets are diverse, with complex shapes and structures, and hidden behind different containers or obstacles. However, existing explosive ordnance disposal robots are mostly based on proprietary scenarios and operation objects, lacking adaptability to unknown environments and intelligent high-precision operation capabilities. In particular, existing technologies have failed to provide effective solutions in terms of long-range beyond-visual-range operations, high-precision processing of complex dynamic targets, and multi-functional operation integration.

[0022] In order to overcome the defects in the prior art, the present invention aims to propose an intelligent water jetting bomb disposal robot system for high-precision water jetting bomb disposal operations of uncertain objects in a remote beyond-visual-range unknown environment.

[0023] Reference Figure 1 As shown, Figure 1 FIG. 1 is a schematic diagram of the overall structure of the intelligent water jetting bomb disposal robot system of the present invention. Figure 1 As shown, the main purpose of the embodiment of the present invention is to provide good visual sensing for the robot to move and operate in an unknown environment through panoramic camera and omnidirectional vision, so as to make accurate operation decisions. An intelligent water cutting bomb disposal robot system, the system includes: a walking platform and: a water cutting system, including a water tank 4, a water jet generator 8 and a nozzle 10 connected in sequence, which are all arranged on the walking platform; a terminal operation mechanism, including a mechanical arm 9, which is connected to the nozzle 10; a multi-dimensional sensing device, including a surround camera sensor 6, a monitoring sensor 7, a depth camera 12 and a laser scanner 11; a control module 5, which is respectively connected to the walking platform, the water cutting system, the terminal operation mechanism and the multi-dimensional sensing device; a remote control station 14, which is remotely connected to the control module 5; Among them, the remote control station 14 is used to: obtain the surround image captured by the surround camera sensor 6 to determine the walking direction of the walking platform; remotely send walking instructions to control the walking platform to move to the working area according to the walking direction; obtain the front image captured by the monitoring sensor 7 to determine the target object located in the working area; obtain the three-dimensional image of the target object captured by the depth camera 12; in response to the scanning instruction for the target area of ​​the three-dimensional image, obtain the three-dimensional point cloud image scanned by the laser scanner 11 for the target area; in response to the cutting instruction for the cutting center of the three-dimensional point cloud image, obtain a preset working trajectory with the cutting center as the starting point; control the robot arm 9 to transfer the nozzle 10 to the cutting center, and control the nozzle 10 to cut the target object according to the preset working trajectory, and at the same time obtain the front image captured by the monitoring sensor 7 during the cutting process.

[0024] In this embodiment, the walking platform is the basic structure of the entire system, which is mainly used to carry various modules (water jet cutting system, terminal operation mechanism, multi-dimensional sensing device, control module 5) and drive them to different operation areas. The walking platform can adopt crawler-type, wheel-type and other moving modes, realize 360° rotation and multi-directional movement according to environmental requirements, and is convenient for moving on different terrains.

[0025] In this embodiment, the remote operator can control the walking platform to rotate 360 ​​degrees in place, and then identify the direction of the suspicious object from the surround image collected by the surround camera sensor 6 in front, so as to continue to move in the direction of the suspicious object, and finally move to the working area where the suspicious object is located, and then find the target object through the monitoring sensor 7. In some cases, the monitoring sensor 7 can further determine the target object with the assistance of the depth camera 12.

[0026] Specifically, the water cutting system includes a water tank 4, a water jet generator 8 and a nozzle 10, which are connected in sequence and used to generate a high-pressure water jet to cut materials. The water tank 4 is used to store water required for cutting, and abrasives can be added to the water flow according to the operation requirements. The water jet generator 8 includes a high-pressure plunger pump, a gasoline engine that provides power for the high-pressure plunger pump, and a pressure tank that stores a mixture of high-pressure water and abrasives used for cutting. The high-pressure plunger pump is used to pressurize the water flow to form a high-pressure mortar in the above-mentioned pressure tank, and the high-pressure mortar passes through the nozzle 10 to form a high-speed abrasive water jet. During the cutting operation, the water in the water tank 4 flows into the high-pressure plunger pump through the water pipe, and the water flow is pressurized by the high-pressure plunger pump. The high-pressure water is input into the above-mentioned pressure tank to form a high-pressure mortar. The high-pressure mortar is transported to the nozzle 10, and the nozzle accelerates the high-pressure mortar to an extremely high speed, using its extremely high kinetic energy and the cutting action of the abrasive to separate or destroy the target object.

[0027] Specifically, the end operation mechanism includes a robot arm 9, on which a nozzle 10 is mounted to drive the nozzle 10 to move. The robot arm 9 is mounted at the front end of the system, has six or more degrees of freedom, and receives a path command sent by a remote control station 14 to move. Optionally, the robot arm 9 can be a flexible multi-joint operating arm.

[0028] In some embodiments, the walking platform includes a support frame 2 and a mobile chassis 1. The mechanical arm 9 is fixed on the support frame 2, and the control module 5 and the water jet cutting system are installed on the mobile chassis 1. Among them, the mobile chassis 1 uses components with obstacle-crossing capabilities, and has the ability to cross steps over 100 mm and ditches over 300 mm, and has the ability to move forward, backward, left, right, and 360° in situ, as well as the ability to adjust speed.

[0029] The improvement of this embodiment lies in the design of a multi-dimensional sensing device and the optimized application of the multi-dimensional sensing device, including a surround camera sensor 6, a monitoring sensor 7, a depth camera 12 and a laser scanner 11. Among them, the surround camera sensor 6 collects surround images and provides a wide-angle field of view, so that the robot can quickly obtain an overview of the surrounding environment. Then, with the help of the mobile chassis 1 rotating 360° on the spot, it can easily find the target object and put it in front of the robot, so as to remotely and directly drive the robot forward and gradually approach the target object. The monitoring sensor 7 is used to collect high-definition images in front, so that the operator can observe the details of the operation in front more clearly and adjust the robot to a suitable position to observe the target object in the working area. The depth camera 12 is used to collect a three-dimensional image of the target object, capture color and depth image data, and generate three-dimensional visual data to create a three-dimensional virtual scene of an unknown working area. The laser scanner 11 is used for high-precision scanning to generate a three-dimensional model of the target object, providing an accurate reference for cutting path planning. The laser scanner 11 has high precision and high stability and can work for a long time in various environments.

[0030] In some embodiments, the depth camera 12 is an RGBD (Red Green Blue Depth) depth camera, and may further be a structured light camera, a TOF depth camera, or a binocular 3D camera.

[0031] In some embodiments, at least one surround camera sensor 6 may be installed. When one surround camera sensor 6 is provided, it is preferably provided at the front side of the robot; when two surround camera sensors 6 are provided, they are preferably provided at the front and rear sides of the robot; when four surround camera sensors 6 are provided, they are preferably provided at the front, back, left, and right sides of the robot, and the four cameras are installed on the support frame 2 to provide 360° panoramic visual data for the robot.

[0032] In some embodiments, the monitoring sensor 7 is installed in front of the support frame 2 on the robot, so as to capture a high-definition image of the target object in front when the remote-controlled robot faces the target object.

[0033] Specifically, the control module 5 is responsible for receiving data information from each sensor, analyzing and processing this information, and generating corresponding control instructions. The control module 5 includes a vehicle-mounted host, including a walking platform control function, a data storage function, a mechanical arm 9 control function, a water jet cutting control function, a vehicle-mounted remote control function, etc., and also includes data collection, processing and transmission functions of a multi-dimensional sensing device.

[0034] The remote control station 14 realizes human-machine interaction, remote video monitoring, remote robot walking and docking, remote scanning of targets, remote water jet cutting operations, chassis status monitoring, operation process monitoring, and remote operation command issuance.

[0035] Exemplarily, the remote control station 14 generally includes a human-machine interaction interface, a remote communication module, a remote control handle, etc. The human-machine interaction interface is used to display real-time video, environmental information and sensor data, and provide an interactive interface for operators to operate. The remote communication module uses a broadband image transmission module, or a 5G module, or a 4G module, or a wifi module or other wireless modules that realize high-speed communication beyond line of sight. The remote control handle includes various control buttons, joysticks and indicator lights, which are used to manually control the actions of various parts of the robot.

[0036] The remote control station 14 is remotely connected to the control module 5 to achieve two-way data transmission between the remote end and the near end. The user sends a remote control command through the human-machine interaction interface at the remote end, and the remote control command is transmitted to the control module 5 at the vehicle end through the image transmission module. The control module 5 finally sends the remote control command to the water cutting system and the robotic arm 9, and controls the water cutting system and the robotic arm 9 through the control module 5 to perform corresponding operations. At the same time, the control module 5 wirelessly transmits the multi-dimensional information (surround view image, front image, three-dimensional image, three-dimensional point cloud image) collected by the multi-dimensional sensing device to the remote image transmission module through the near-end wireless image transmission, and the remote image transmission module transmits the received multi-dimensional information to the human-machine interaction interface for display. Therefore, the operator remotely controls the robot through the remote control station 14, and the robot's control module 5 remotely controls and interacts with the operator through wireless communication. The operator can control the movement and operation process of the robot in a safe position.

[0037] Therefore, in the process of operating the robot for bomb disposal, the execution step of the system provided in this embodiment can be applied to the remote control station 14 in the specific use process, and the walking of the robot, the movement of the mechanical arm 9, the visual perception control and processing of the multi-dimensional perception device, the opening and closing of the high-pressure pump and the high-pressure mortar tank of the water cutting system can be remotely controlled by the remote control station 14. The execution step can also be applied to the control module 5, which can operate the robot to perform corresponding operations by responding to the remote control instructions output by the remote control station 14.

[0038] This embodiment uses the remote control station 14 as the control subject to illustrate the method of using the robot system to perform bomb disposal operations.

[0039] See also Figure 2 , Figure 2 The following is a flow chart of the steps of an intelligent water jetting explosive disposal robot operation method. An intelligent water jetting explosive disposal robot operation method comprises the following steps: S1, obtaining the surround image collected by the surround camera sensor 6, and determining the walking direction of the walking platform; The remote control station 14 and the control module 5 form a two-way communication link. The bomb disposal mission is officially started, and the image of the surrounding environment is captured by the surround camera sensor 6. The operator observes the collected surround image on the human-computer interaction interface on the remote control station 14, finds the target object or the transfer station required to reach the target object, and thus determines the direction of movement of the walking platform. The walking direction of the walking platform is the direction of the target object or the transfer station required to reach the target object, so the chassis is controlled to move in the direction of the target object or the transfer station required to reach the target object.

[0040] S2, remotely send walking instructions to control the walking platform to move to the working area according to the walking direction; After finding the target object, the operator controls the mobile chassis 1 to move forward, backward, turn left or right, etc. through the remote control handle, moves to the working area where the target object is located, and approaches the explosive to observe the explosive.

[0041] Specifically, during the movement, the robot chassis rotates 360 degrees in situ through the collected surround images to place the discovered target in front of the crawler robot, and the robot's walking direction is adjusted by the left and right control buttons to always face the explosive disposal object, and the robot can be driven forward directly to get closer to the explosive disposal object. The surround images provided by the surround camera sensor 6 continuously provide information about the robot's surrounding conditions along the way.

[0042] S3, obtaining the front image collected by the monitoring sensor 7, and determining the target object located in the working area; When the robot is near and facing the explosive, the monitoring sensor 7 provides a high-definition image in front of it to determine the video image of the target object that needs to be defuse, and adjusts the robot to a suitable position through the front image to meet the subsequent operating space of the robot arm 9.

[0043] Further, step S3 includes: S31, obtaining the front image captured by the monitoring sensor 7 and the two-dimensional image captured by the depth camera 12, and determining the position information of the target object in the working area; S32. Based on the position information, control the walking platform to move to the target position.

[0044] The chassis position of the robot is adjusted by the monitoring sensor 7 and the depth camera 12. The monitoring sensor 7 fixed in front of the bomb disposal robot cannot see and judge the distance and relative position relationship between the target object and the bomb disposal robot. In this embodiment, the depth camera 12 fixed on the robotic arm has a two-dimensional visual function. By controlling the robotic arm to be in a suitable posture, the two-dimensional image collected by the depth camera 12 can observe the relative position relationship between the target object and the bomb disposal robot. According to the acquired two-dimensional image, the bomb disposal robot can then be driven closer to the target object until the robot is in the target position. Among them, the target position can be understood as the most suitable position between the current robot and the target object. When the robotic arm 9 is operated at this position, the target object is just within the reachable range of the nozzle fixed to the robotic arm 9, and the robotic arm 9 and the nozzle fixed to the robotic arm 9 do not interfere with the target object and other parts of the robot body.

[0045] Therefore, the image of the target explosive and the information of the depth camera 12 are presented on the human-computer interaction interface through the front image collected by the monitoring sensor 7 and the two-dimensional image collected by the depth camera 12. The chassis 1 is moved to the most suitable target position by the remote control handle, so that the monitoring sensor 7 can observe the panoramic view of the target object and the depth information provided by the depth camera 12 scans to meet the working space of the robot arm 9 for cutting the target object.

[0046] In some scenarios, while the robot is moving, the operator can simultaneously operate the robotic arm 9 through the remote control handle on the remote control station 14, so as to conduct omnidirectional close observation of any suspicious objects in the surrounding area through the depth camera 12 installed thereon, and further identify the target object in the working area.

[0047] S4, obtaining a three-dimensional image of the target object captured by the depth camera 12; After the target object is determined and the bomb disposal robot is in a suitable target position with the target object, a binocular depth camera 12 is used to perform a rough scan to generate a three-dimensional image of the bomb disposal object. In this embodiment, the depth camera 12 is remotely operated to perform a rough scan of the target object, and the three-dimensional image obtained by the rough scan is processed by RGBD, etc., and the final result is presented on the human-computer interaction interface.

[0048] Among them, the depth camera 12 can switch from two-dimensional vision to three-dimensional scanning, and use the three-dimensional scanning function to capture the three-dimensional data of the target object from multiple angles, and then perform three-dimensional stitching on the multi-angle data, and perform three-dimensional reconstruction on the stitched point cloud data, establish a three-dimensional virtual scene with the robot as the coordinate system, restore the approximate contour shape of the target object, and finally display the reconstructed three-dimensional image on the remote human-computer interaction interface. Therefore, the three-dimensional image represents the approximate three-dimensional model of the target object, obtains the approximate outer contour and depth information of the target object, and thus obtains the working space and anti-collision boundary of the subsequent laser scanner 11, providing visualization data for the subsequent manual selection of the fine local scanning area of ​​the laser scanner 11.

[0049] S4, in response to a scanning instruction for a target area of ​​the three-dimensional image, obtaining a three-dimensional point cloud image obtained by scanning the target area by the laser scanner 11; Since the depth camera 12 is not accurate enough, only using the work space information obtained by the three-dimensional image to cut the robot arm 9 may cause the cutting head to collide with the suspected EOD material (there is a risk of detonation) or to be too far away from the suspected EOD material (there is a risk of not cutting through). Therefore, the laser scanner 11 is switched to obtain more accurate three-dimensional information. The operator selects the laser scanning area according to the three-dimensional image of the work area seen on the remote operation station, completes the laser scanning and processing of the selected area, and obtains a three-dimensional point cloud image by scanning in the target area through the laser scanner 11, and presents the processed three-dimensional point cloud image through the human-computer interaction interface so that the operator can confirm the center to be cut.

[0050] Specifically, the user selects the target area to be cut, such as a point or a surface, on the three-dimensional image obtained by the rough scan on the touch screen, and plans the scanning path of the laser scanner 11 based on the target area selected by the user and the three-dimensional image. After confirmation, the scanning instruction of the robot arm 9 is edited and generated to start the scan. The system simultaneously records the movement trajectory of the robot arm 9 (the end position of the robot arm 9) and the laser scanning data, obtains the accurate three-dimensional model of the target object through three-dimensional reconstruction, and stores it in the vehicle-mounted host in the file format of point cloud.

[0051] S5, in response to a cutting instruction for a cutting center of the three-dimensional point cloud image, obtaining a preset operation trajectory with the cutting center as a starting point; The operator selects the center to be cut through the three-dimensional point cloud image, obtains the coordinates of the starting point of the cutting path, and obtains the normal vector of the cutting surface at this point by fitting the three-dimensional point cloud data, thereby completing the cutting path planning of the nozzle 10 and realizing the cutting operation of the selected cutting center. Among them, after the three-dimensional model point cloud file is preprocessed, the operator determines the surface to be cut through the current viewing angle of the laser scanner 11 on the screen of the image transmission module, and transmits the point cloud data of the laser scanner 11 captured in the limited area to the CAM (Computer-Aided Manufacturing) system. The operator selects the center point of the cutting hole on the CAM system and feeds it back to the vehicle-mounted system. The CAM software on the vehicle-mounted host automatically generates a hole-shaped cutting program based on the cutting center point position and normal vector data, and can perform simulation. The script program is converted into a robot motion control program format that conforms to the robot arm 9 system, that is, the preset operation trajectory of the water jet cutting is obtained.

[0052] S7, control the robot arm 9 to transfer the nozzle 10 to the cutting center, and control the nozzle 10 to cut the target object along a preset operation trajectory, and at the same time obtain the front image collected by the monitoring sensor 7 during the cutting process.

[0053] After selecting the cutting center point, a preset operation trajectory with the cutting center point as the starting point is obtained. The control module 5 moves the mechanical arm 9 to the cutting center based on the preset operation trajectory, starts the engine through the control of the water cutting system, and then starts the high-pressure pump to output a high-pressure water jet from the nozzle 10. When the pressure reaches the preset value (the monitoring sensor 7 observes that the water mist is large), the valve of the high-pressure mortar tank is opened, and the nozzle 10 is controlled to move along the preset operation trajectory to achieve the cutting operation of the selected area. During the cutting process, the operator obtains the real-time video image of the vehicle-mounted terminal through the image communication between the remote terminal and the vehicle-mounted terminal, presents the cutting process through the video image collected by the monitoring sensor 7, monitors the cutting status, and remotely pauses or stops the cutting operation when a fault condition is found.

[0054] In summary, the surround camera sensor 6 and the monitoring sensor 7 provide a wide field of view to determine the position and direction of travel of the target object. The depth camera 12 performs a rough scan to quickly generate a large-scale three-dimensional image, determine the general outline of the target object, lock the position, distance and size of the target object, plan the working space and anti-collision boundary of the subsequent laser scanner 11, and provide the target area scanning range for the laser scanner 11. The laser scanner can capture details with high precision in a small range, perform fine scanning in the target area calibrated by the depth camera 12, and obtain an accurate three-dimensional model of the target object through local fine scanning, thereby ensuring accurate analysis and cutting of the target object. In this way, the robot is provided with good visual sensing for travel and operation through panoramic photography and omnidirectional vision, so that the robot can obtain multi-dimensional visual information in a complex and unknown environment, and realize dynamic, all-round target recognition and fine operation positioning. In addition, the functions of multiple modules in different dimensions work together to realize the gradual refinement of the perception process, so that the collection of subsequent modules can be more centralized and efficient, and the overall operation efficiency is improved. The multi-dimensional sensing device enables the robot to have environmental perception capabilities from wide-area to microscopic when performing tasks, effectively improving the accuracy and flexibility of identification, positioning and operation, as well as safety.

[0055] Therefore, bomb disposal robots are expected to achieve higher intelligence and precision, and be able to cope with the bomb disposal mission requirements of various and uncertain obstacles (such as obstacles in the environment, hidden trenches, other dangerous substances, etc.) and irregular terrain (such as urban streets, rugged outdoor terrain, small spaces, etc.).

[0056] As a specific explanation of this embodiment, there is no omnidirectional terminal mechanism with high-precision detection and operation in the related art. Therefore, even if the bomb disposal robot is equipped with a camera or other types of visual sensors, the detection equipment and the operating mechanism are independent and separated from each other in space, which will cause the working position to shift, and the independent detection equipment is not only limited to a fixed installation position or viewing angle, and cannot flexibly follow the operating mechanism for real-time adjustment, but also in a complex or narrow space, the detection equipment cannot fully observe and obtain detailed information on the explosive disposal materials, which increases the difficulty of operation and limits the flexibility of the robot.

[0057] In view of this, another object of the present invention is to realize an end operation mechanism based on a high-precision detection and operation integrated with a robot arm 9 to perform omnidirectional operations on dynamic targets. The laser scanner 11 and the depth camera 12 are installed at the end of the robot arm 9 in a first direction, and the nozzle 10 is installed at the end of the robot arm 9 in a second direction, and the first direction and the second direction intersect.

[0058] Specifically, the robot arm 9 is installed at the front end of the robot, and has six or more degrees of freedom. The laser scanner 11, the depth camera 12 and the hose telescopic endoscope 13 are installed in the first direction of the end of the robot arm 9, and the nozzle 10 is installed in the second direction of the end of the robot arm 9 that deviates from the first direction. Therefore, the multi-degree-of-freedom robot arm 9 can be flipped and used alternately (i.e., switch between the first direction and the second direction), perform multi-dimensional recognition before cutting, and perform water cutting after recognition is completed, ensuring that the multi-dimensional recognition and cutting functions do not interfere with each other.

[0059] In this embodiment, when the first direction is toward the target object, multi-dimensional recognition is performed by the laser scanner 11, the depth camera 12 and the hose telescopic endoscope 13. When the second direction is toward the target object, the water cutting operation of the nozzle 10 is performed.

[0060] Therefore, the components fixed at the end of the robot arm 9 include a depth camera 12, a laser scanner 11, and a nozzle 10. After the remotely operated mobile chassis 1 reaches the target position, the movement of the robot arm 9 is operated to realize the required movement of the components fixed at the end of the robot arm 9 by virtue of the movement of the end of the robot arm 9.

[0061] The end of the robot arm 9 needs to be equipped with a depth camera 12, a laser scanner 11 and a nozzle 10 at the same time, and the depth camera 12 and the laser scanner 11 are opposite to the nozzle 10. When the nozzle 10 is working, the high-pressure water flow will splash everywhere when cutting objects, and will not splash onto the sensing device. Therefore, the depth camera 12, the laser scanner 11, and the nozzle 10 form a turret structure on the robot arm 9 in a specific direction. When the equipment in one direction is working, the equipment in the other direction is in a relatively isolated position. This can ensure that the equipment does not interfere with each other when working, and can also be well isolated and protected from each other.

[0062] In some embodiments, the depth camera 12 and the laser scanner 11 are also protected from water.

[0063] Furthermore, the multi-dimensional sensing device also includes a hose telescopic endoscope 13 arranged in the first direction of the robot arm 9. In this embodiment, the hose telescopic endoscope 13 detects areas that are not easy to directly observe or hidden objects in a small space, so the operation effect can be inspected by endoscopy. Therefore, after the cutting operation is completed, the water jet generator 8 is turned off, the end of the robot arm 9 is reversed, the RGBD depth camera and the hose telescopic endoscope 13 are facing the cutting object, and the operator operates the robot arm 9 through the remote control station 14 to observe the cutting condition at close range, or extends the hose telescopic endoscope 13 into the cutting hole for endoscopy.

[0064] Therefore, after the cutting is completed, the robot arm 9 is controlled to drive the hose telescopic endoscope 13 to extend into the hole after the cutting, obtain the inspection information collected by the hose telescopic endoscope 13, and output the cutting result; and / or, after the cutting of the target object is completed, the nozzle 10 at the end of the robot arm 9 is controlled to flip, drive the depth camera 12 to face the hole after the cutting, obtain the three-dimensional image collected by the depth camera 12, and output the cutting result.

[0065] This embodiment integrates the multi-dimensional sensing device at the end of the robot arm 9 to achieve the integration of detection and operation. By integrating and installing the laser scanner 11, RGBD depth camera and hose telescopic endoscope 13 at the end in a specific direction, the multi-dimensional sensing device can move with the robot arm 9 in a complex or narrow environment, and use the multi-degree-of-freedom movement of the robot arm 9 to directly move the detection equipment close to the observed object to accurately locate the detailed features of the suspicious object. In addition, since the high-precision detection equipment and the robot arm 9 share the same coordinate system, the coordinate conversion error between detection and operation is reduced, the offset of the operation position is avoided, the accuracy of detection positioning and the accuracy of cutting operations are greatly improved, and the needs of high-precision bomb disposal operations are met.

[0066] In a further technical solution, the remote control station 14 includes a robot arm simulation module, which is configured to simulate whether the robot arm 9 collides with the target object when moving along different preset trajectories, and send a motion instruction to the control module 5 to control the robot arm 9 to move along the preset trajectory when there is no collision; wherein the motion instruction includes a first motion instruction, a second motion instruction and a third motion instruction; wherein, The robotic arm simulation module is used to: send a first motion instruction to control the robotic arm 9 to flip along a first preset trajectory to obtain a three-dimensional image; send a second motion instruction to control the robotic arm 9 to move along a second preset trajectory to obtain a three-dimensional point cloud image; send a third motion instruction to control the robotic arm 9 to cut the target object along a preset operating trajectory.

[0067] In this embodiment, considering that the operator is not on site, remote operation of the robot arm 9 is prone to collision. Once it collides with a suspected explosive disposal object, there is a risk of detonation. The remote control station 14 implements the simulation function of the robot arm 9 through the CAM software. In the simulation, the system simulates the movement operation that the robot arm 9 will perform, verifies whether its movement trajectory meets the requirements and does not interfere with the target object, and only after confirming that there is no interference and collision through robot simulation, the movement of the robot arm 9 is executed.

[0068] In this embodiment, when the depth camera 12 installed at the end of the robot arm 9 starts to roughly scan the target object, the operator adjusts the posture of the robot arm 9 through the remote control system, and uses the robot arm simulation module of the system's CAM software to confirm that there is no interference or collision before the actual movement through robot simulation, and then sends a first movement instruction to the control module 5, and then executes the movement of the robot arm 9 corresponding to the first movement instruction, so that the robot arm 9 is turned over so that the depth camera 12 faces the target object, and then uses the depth camera 12 to obtain a three-dimensional image of the target object.

[0069] When the target object is precisely scanned by the laser scanner 11 installed at the end of the robot arm 9, the robot arm simulation module of the CAM software of the system is also used to confirm that there is no interference collision through robot simulation, and then the second movement instruction is sent to the control module 5, and the movement of the robot arm 9 corresponding to the second movement instruction is executed.

[0070] Through the second motion instruction, the robot arm 9 is controlled to adjust the position of the laser scanner 11, and the laser scanner 11 is used to scan the target area with a second preset trajectory to obtain a more accurate three-dimensional point cloud image.

[0071] Exemplarily, the second motion instruction is an instruction to perform high-precision autonomous scanning of the selected target area in a uniform linear motion. Therefore, in step S5, the robotic arm 9 is moved by the second motion instruction, so that the robotic arm 9 drives the laser scanner 11 to move along a certain trajectory in the target area, so that the laser scanner 11 obtains a more accurate three-dimensional point cloud image in the target area.

[0072] When the target object is cut by the nozzle 10 installed at the end of the robot arm 9. The robot arm simulation module of the CAM software of the system is also used. After confirming that there is no interference collision through robot simulation, the third motion instruction is sent to the control module 5, and the movement of the robot arm 9 corresponding to the third motion instruction is executed. The third control instruction is an instruction to control the robot arm 9 to move according to the preset operation trajectory. After the end of the robot arm 9 is flipped, the nozzle 10 is located at the cutting center, that is, the starting point of the preset cutting motion trajectory, and the water jet generator 8 is started to generate a high-pressure sand-containing water flow to the nozzle 10, and the nozzle 10 is further controlled to perform the cutting operation according to the planned operation trajectory.

[0073] Exemplarily, the third motion instruction is a circular motion instruction for controlling the robot arm 9 starting from the cutting center. Therefore, in step S7, the robot arm 9 is moved by the third motion instruction, so that the robot arm 9 drives the nozzle 10 to move along a preset operation trajectory, and a circular hole is cut. Due to the characteristics of nozzle cutting, there is often a small amount of residual material on the bottom surface of the target sheet material where the circular hole is cut out, which is not completely cut off. In order to ensure that the material in the circular hole is completely separated and falls off, and to facilitate subsequent peeping and observation of the hole, the circular cutting movement is planned to be greater than 360° (for example, 370°). This extra 10° of circular cutting movement can ensure that the material in the circular hole is completely separated.

[0074] It is understandable that it is possible to select whether to use the robot arm simulation module to simulate each time the robot arm 9 needs to be moved according to actual conditions before operating the robot arm 9. For example, after confirming that there is no interference or collision through robot simulation, the fourth motion instruction is sent to the control module 5 to operate the robot arm 9 to perform omnidirectional close observation of any suspicious objects in the surrounding area. The fourth motion instruction can be an instruction to control the robot arm 9 to move forward, backward, turn left or right, etc.

[0075] In another technical solution, another object of the present invention is to construct a modular platform that can adapt to a variety of environments and objectives, and to improve the adaptability and stability of the robot in complex terrain and diversified tasks. Telescopic legs 3 are also provided on the four sides of the mobile chassis 1; wherein the control module 5 is configured with a level sensor, and the control module 5 is connected to each telescopic leg 3 to control the telescopic height of each telescopic leg 3.

[0076] In this embodiment, the telescopic legs 3 include four telescopic balancing legs driven by hydraulic cylinders or electric cylinders installed at the four corners of the support frame 2. Therefore, when high-precision cutting operations are required, the walking platform can be moved to the target position of the target object, and after the robot moves to the target position, the chassis remains fixed at the current position. On uneven or sloping terrain, the telescopic legs 3 are automatically adjusted by an electric push rod or hydraulic cylinder to keep the robot balanced on the unstable terrain. After being fixed, the depth camera 12 installed at the end of the robotic arm 9 is used to start scanning the target object. Therefore, moving to the target position and then fixing the working position not only facilitates the global scanning of the multi-dimensional sensing device, but also reduces the vibration impact of the water jet system on the cutting operation.

[0077] The control module 5 is connected to each electric telescopic leg 3, and the four telescopic legs 3 can adjust the height independently. The control module 5 is equipped with a horizontal sensor to detect the horizontal state of the robot in real time. When the robot is in a suitable target position, the four legs of the telescopic legs 3 are released, and the horizontal sensor is used to detect whether the robot is horizontal. If the walking platform is judged to be non-horizontal, the control system will instruct each telescopic leg 3 to extend with a different extension amount until it is horizontal. If the walking platform has been judged to be horizontal, the system will control all telescopic legs 3 to extend with the same extension amount until they reach a preset height.

[0078] For example, the inclination angles of the four telescopic legs 3 are different. After receiving these data, the control module 5 calculates the height that each leg needs to be adjusted according to the inclination angle. After the telescopic legs 3 are adjusted, the control module 5 will read the horizontal information of the horizontal sensor again to confirm the adjustment effect. If the tilt is still found after the adjustment, the control module 5 will continue to make fine adjustments. If it is level, the adjustment will stop.

[0079] In some embodiments, the hydraulic drive can be connected to a hydraulic station. The electric cylinder drive can be connected to a battery system that comes with the robot, such as a battery in a walking platform.

[0080] In some embodiments, the telescopic legs 3 are retracted away from the ground when not in use, that is, the bottom of the legs is higher than the bottom of the chassis. When in use, the telescopic legs 3 can be raised to a greater height to lift the mobile chassis 1 off the ground, and the preset height can be set to a height that allows the walking platform to lift off the ground to adapt to uneven terrain or obstacles. Alternatively, the telescopic legs 3 can be raised to a height that is flush with the mobile chassis 1 when the legs land, and the preset height can be set to a height that is flush with the bottom surface of the walking platform to increase the support force on a flat working surface.

[0081] Preferably, the mobile chassis 1 of the bomb disposal robot system is flexibly connected to the support frame 2, and then supported horizontally on the ground by telescopic balancing legs, so that the loading equipment is seismically isolated from the mechanical arm 9 which requires high precision, thereby ensuring the requirements of high-precision scanning of the laser scanner 11 and high-precision operation of the nozzle 10.

[0082] As a further explanation of this embodiment, the control module 5 includes a first controller and a second controller. The first controller is responsible for the control of the mobile chassis 1 and the mechanical arm 9, the visual image processing and remote transmission of the surround camera sensor 6, the RGBD depth camera and the hose telescopic endoscope 13; the second controller is responsible for the creation of a three-dimensional virtual scene, CAM calculation conversion, path optimization, etc. Therefore, each task can be executed separately and independently, avoiding control delays caused by information overload.

[0083] In summary, the intelligent water jet bomb disposal robot system provided by the present invention includes: a mobile chassis 1, a support frame 2, a telescopic support leg 3, a water tank 4, a control module 5, a surround camera sensor 6, a monitoring sensor 7, a water jet generator 8, a robotic arm 9, a nozzle 10, a laser scanner 11, an RGBD depth camera, a hose telescopic endoscope 13, and a remote control station 14. The robot has an IP65 protection level and is suitable for field operations. The innovative design of the omnidirectional visual sensor, the retractable support leg provides a stable working platform, the high-precision detection and operation-integrated terminal operation mechanism based on the robotic arm 9, and the remote interactive three-dimensional visual detection and planning realize intelligent and flexible cutting operations, so that the bomb disposal robot can perform cutting operations more accurately and comprehensively in a complex and dynamic bomb disposal environment.

[0084] It can be known that the structure of the bomb disposal robot is usually more complex and sophisticated, including several modules and subsystems. For the bomb disposal robot system, such as the walking platform, water cutting system, terminal operation mechanism, control module 5 and remote control station 14, more functional modules are usually included. These modules are relatively mature in the prior art in the relevant field, and their basic functions and composition are well known in the industry. For the sake of clarity and simplicity, they are not described in detail in this technical solution. For example, the walking platform usually includes a power system, an obstacle avoidance module, a positioning and navigation system, etc. For example, the water cutting system also includes an abrasive tank connected to the high-pressure plunger pump through a pipeline, a regulation system, a generator and a battery, etc. For example, the terminal operation mechanism includes a terminal tool interface, a multi-joint structure, a multi-joint control system that provides posture adjustment, etc. For example, the control module 5 also includes a remote communication module, a power module, a fault detection and diagnosis system, etc. For example, the remote control station 14 includes a power supply, an audio prompt system, etc.

[0085] Correspondingly, for the second aspect, please refer again Figure 2 The present invention also provides an intelligent water jetting explosive disposal robot operation method, which is executed by a remote control station 14 and is used in the intelligent water jetting explosive disposal robot system provided in the first aspect of the present invention. The method includes the following steps: S1, obtaining the surround image collected by the surround camera sensor 6, and determining the walking direction of the walking platform; S2, remotely send walking instructions to control the walking platform to move to the working area according to the walking direction; S3, obtaining the front image collected by the monitoring sensor 7, and determining the target object located in the working area; S4, obtaining a three-dimensional image of the target object captured by the depth camera 12; S5, in response to a scanning instruction for a target area of ​​the three-dimensional image, obtaining a three-dimensional point cloud image obtained by scanning the target area by the laser scanner 11; S6, in response to a cutting instruction for a cutting center of the three-dimensional point cloud image, obtaining a preset operation trajectory with the cutting center as a starting point; S7, control the robot arm 9 to transfer the nozzle 10 to the cutting center, and control the nozzle 10 to cut the target object along a preset operation trajectory, and at the same time obtain the front image collected by the monitoring sensor 7 during the cutting process.

[0086] Further, step S3 includes: S31, obtaining the front image captured by the monitoring sensor 7 and the two-dimensional image captured by the depth camera 12, and determining the position information of the target object in the working area; S32. Based on the position information, control the walking platform to move to the target position.

[0087] Further, step S4 includes: S41, sending a first motion instruction to control the robot arm 9 to flip along a first preset trajectory so that the depth camera 12 faces the target object; the first motion instruction is characterized by simulating that the robot arm 9 does not collide with the target object when flipping along the first preset trajectory; S42: Acquire a three-dimensional image of the target object captured by the depth camera 12.

[0088] Step S5 includes: S51, sending a second motion instruction to control the robot arm 9 to move along a second preset trajectory in the target area; the second motion instruction is characterized by simulating that the robot arm 9 does not collide with the target object when moving along the second preset trajectory; S52 , obtaining a three-dimensional point cloud image obtained by scanning the target object in the target area by the laser scanner 11 moving with the robot arm 9 .

[0089] Step S7 includes: S71 , sending a third motion instruction to control the robot arm 9 to move along a preset working trajectory in the target area; the third motion instruction is characterized by simulating that the robot arm 9 does not collide with the target object when moving along the preset working trajectory.

[0090] Further, after step S32, the following steps are included: S33, obtaining horizontal information of the walking platform, and determining whether the walking platform is horizontal; S34, when the walking platform is not horizontal, controlling the multiple telescopic legs 3 to extend to their respective corresponding telescopic heights; S35. When the walking platform is horizontal, control the multiple telescopic legs 3 to extend to the same preset height.

[0091] Further, after step S7, the following steps are included: S8, after the cutting of the target object is completed, the robot arm 9 is controlled to drive the flexible tube telescopic endoscope 13 to extend into the hole after the cutting, obtain the inspection information collected by the flexible tube telescopic endoscope 13, and output the cutting result; and / or, S9. After the cutting of the target object is completed, the nozzle 10 at the end of the robot arm 9 is controlled to flip, driving the depth camera 12 toward the cut hole, obtaining the three-dimensional image captured by the depth camera 12, and outputting the cutting result.

[0092] Further, after step S8 or S9, the following steps are included: S10, the robot resets and evacuates. After the bomb disposal operation is completed, the operator controls the bomb disposal robot to return through the remote control station 14.

[0093] In summary, through the gradually refined perception process, the system can effectively reduce unnecessary scanning and calculation time and improve overall work efficiency. The spatial information provided by the depth camera 12 when establishing a three-dimensional virtual scene makes the target locking of the laser scanner more accurate and reduces errors. The laser scanner can focus on smaller and more precise areas for detailed scanning to ensure that the subsequent water cutting process is carried out on the optimal path. The specific perception device combination operation method enables the robot to have environmental perception capabilities from wide area to microscopic when performing tasks, effectively improving the accuracy and flexibility of identification, positioning and operation, and ensuring the speed, accuracy and safety of bomb disposal operations.

[0094] It should be noted that, for the method embodiments, the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously.

[0095] As for the above method embodiment, since it is basically similar to the system embodiment, the description is relatively simple, and the relevant parts can continue to refer to the partial description of the system embodiment.

[0096] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0097] It should also be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations, nor can they be understood as indicating or implying relative importance. Moreover, the term "include" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or terminal device.

[0098] The above is a detailed introduction to an intelligent water jetting bomb disposal robot system and method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the present application, and the content of this specification should not be understood as a limitation on the present application. At the same time, for those of ordinary skill in the art, according to the present application, there will be different forms of changes in the specific implementation methods and application scopes. It is not necessary and impossible to list all the implementation methods here, and the obvious changes or modifications derived from this are still within the scope of protection of the present application.

Claims

1. An intelligent water jetting bomb disposal robot system, characterized in that: The system comprises: a walking platform and: The water cutting system comprises a water tank, a water jet generator and a nozzle which are connected in sequence; a terminal operation mechanism comprises a mechanical arm which is connected to the nozzle; a multi-dimensional sensing device comprises a surround camera sensor, a monitoring sensor, a depth camera and a laser scanner; a control module which is respectively connected to the walking platform, the water cutting system, the terminal operation mechanism and the multi-dimensional sensing device; and a remote control station which is remotely connected to the control module in communication; Wherein, the remote control station is used for: Acquire the surround view image collected by the surround view camera sensor to determine the walking direction of the walking platform; Remotely send a walking instruction to control the walking platform to move to the working area according to the walking direction; Acquire the front image collected by the monitoring sensor to determine the target object located in the operation area; Acquire a three-dimensional image of the target object acquired by the depth camera; In response to a scanning instruction for a target area of ​​the three-dimensional image, acquiring a three-dimensional point cloud image obtained by scanning the target area with the laser scanner; In response to a cutting instruction for a cutting center of the three-dimensional point cloud image, a preset operation trajectory with the cutting center as a starting point is obtained; The robot arm is controlled to transfer the nozzle to the cutting center, and the nozzle is controlled to cut the target object along the preset operation trajectory, while obtaining the front image collected by the monitoring sensor during the cutting process.

2. The intelligent water jetting bomb disposal robot system according to claim 1 is characterized in that: At least one of the surround camera sensors is disposed on at least one side of the walking platform, and the monitoring sensor is disposed directly in front of the walking platform; The laser scanner and the depth camera are installed at a first direction of the end of the robotic arm, and the nozzle is installed at a second direction of the end of the robotic arm, and the first direction and the second direction intersect.

3. The intelligent water jetting bomb disposal robot system according to claim 2 is characterized in that: in, The remote control station includes a robot arm simulation module, the robot arm simulation module is configured to simulate whether the robot arm collides with the target object when moving along different preset trajectories, and send a motion instruction to the control module to control the robot arm to move along the preset trajectory when there is no collision; wherein the motion instruction includes a first motion instruction, a second motion instruction and a third motion instruction; The robotic arm simulation module is used for: Sending the first motion instruction to control the robotic arm to flip along a first preset trajectory to obtain the three-dimensional image; Sending the second motion instruction to control the robotic arm to move along a second preset trajectory to obtain the three-dimensional point cloud image; The third motion instruction is sent to control the robot arm to cut the target object along the preset operation trajectory.

4. The intelligent water jetting bomb disposal robot system according to claim 2 is characterized in that: The multi-dimensional sensing device further comprises a hose telescopic endoscope arranged in the first direction of the mechanical arm, and the remote control station is further used for: After the cutting of the target object is completed, the flexible tube telescopic endoscope is inserted into the cut hole, the inspection information collected by the flexible tube telescopic endoscope is obtained, and the cutting result is output; and / or, After the cutting of the target object is completed, the nozzle at the end of the robot arm is controlled to flip, driving the depth camera to face the cut hole, obtaining a three-dimensional image captured by the depth camera, and outputting the cutting result.

5. The intelligent water jetting bomb disposal robot system according to claim 1 is characterized in that: The walking platform comprises a mobile chassis, and the four sides of the mobile chassis are respectively provided with telescopic legs; Wherein, the control module is configured with a horizontal sensor, and the control module is connected to each of the telescopic legs to control the telescopic height of each of the telescopic legs.

6. An intelligent water jetting bomb disposal robot operation method, characterized in that: The method comprises: Obtain the surround images collected by the surround camera sensor to determine the walking direction of the walking platform; Remotely send a walking instruction to control the walking platform to move to the working area according to the walking direction; Acquire the front image collected by the monitoring sensor to determine the target object located in the operation area; Acquire a three-dimensional image of the target object acquired by a depth camera; In response to a scanning instruction for a target area of ​​the three-dimensional image, acquiring a three-dimensional point cloud image obtained by scanning the target area with a laser scanner; In response to a cutting instruction for a cutting center of the three-dimensional point cloud image, a preset operation trajectory with the cutting center as a starting point is obtained; The robot arm is controlled to transfer the nozzle to the cutting center, and the nozzle is controlled to cut the target object according to the preset operation trajectory, and at the same time, the front image collected by the monitoring sensor during the cutting process is obtained.

7. The intelligent water jetting bomb disposal robot operation method according to claim 6 is characterized in that: The acquiring of the front image collected by the monitoring sensor and determining the target object located in the operation area includes: Acquire the front image captured by the monitoring sensor and the two-dimensional image captured by the depth camera to determine the position information of the target object located in the working area; Based on the position information, the walking platform is controlled to move to a target position.

8. The intelligent water jetting bomb disposal robot operation method according to claim 6 is characterized in that: Acquiring a three-dimensional image of the target object acquired by the depth camera, comprising: Sending a first motion instruction to control the robotic arm to flip along a first preset trajectory; the first motion instruction is characterized by simulating that the robotic arm does not collide with the target object when flipping along the first preset trajectory; Acquire a three-dimensional image of the target object acquired by the depth camera; The step of acquiring a three-dimensional point cloud image obtained by scanning the target area with the laser scanner includes: Sending a second motion instruction to control the robotic arm to move along a second preset trajectory within the target area; the second motion instruction is characterized by simulating that the robotic arm does not collide with the target object when moving along the second preset trajectory; Acquire a three-dimensional point cloud image obtained by scanning the target object in the target area by the laser scanner moving with the mechanical arm; The controlling the nozzle to cut the target object along the preset operation trajectory includes: A third motion instruction is sent to control the robot arm to move along the preset operation trajectory in the target area; the third motion instruction is characterized by simulating that the robot arm does not collide with the target object when moving along the preset operation trajectory.

9. The intelligent water jetting bomb disposal robot operation method according to claim 7 is characterized in that: Based on the position information, controlling the walking platform to move to the target position, then comprising: Obtaining level information of the walking platform to determine whether the walking platform is level; When the walking platform is not horizontal, controlling the plurality of telescopic legs to extend at their respective corresponding telescopic heights; When the walking platform is horizontal, the plurality of telescopic legs are controlled to extend at the same preset height.

10. The intelligent water jetting bomb disposal robot operation method according to claim 6, characterized in that: The controlling the robot arm to transfer the nozzle to the cutting center and controlling the nozzle to cut the target object along the preset operation trajectory, then includes: After the cutting of the target object is completed, the robot arm is controlled to drive the flexible tube telescopic endoscope to extend into the cut hole, obtain the inspection information collected by the flexible tube telescopic endoscope, and output the cutting result; and / or, After the cutting of the target object is completed, the nozzle at the end of the robot arm is controlled to flip, driving the depth camera to face the cut hole, obtaining a three-dimensional image captured by the depth camera, and outputting the cutting result.

Images