Inspection path planning method and device, storage medium and inspection robot

Abstract

The invention discloses an inspection path planning method and device, a storage medium and an inspection robot. The method comprises the following steps: in response to triggering of an inspection event, acquiring the current position of an inspection robot and environment data of an area where the inspection robot is located; processing the environment data based on the large path decision model to obtain a decision instruction; wherein the decision instruction comprises a first final position of the inspection robot in a current drivable area; determining a first path according to the current position and the decision instruction, and controlling the inspection robot to inspect based on the first path; in response to an event that the first end point position is inconsistent with a second end point position corresponding to the inspection event, returning to execute acquisition of the current position and the environment data; and in response to an event that the first end point position is consistent with the second end point position or the first path comprises the second end point position, controlling the inspection robot to inspect to the second end point position according to the updated first path. According to the technical scheme, accurate and effective planning of the inspection path of the robot is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a patrol path planning method, device, storage medium, and patrol robot. Background Art

[0002] During the operation of a nuclear power plant or in the area where a nuclear accident occurs, there is usually a high level of radiation and a complex environment. It is necessary to inspect numerous devices in the area to confirm the operating status of the nuclear power plant equipment and promptly discover potential safety hazards.

[0003] Nuclear power plants are characterized by unique environments and radiation risks, making manual inspections difficult to conduct effectively. Robotic inspections are often required. Currently, robots typically follow pre-planned, fixed routes. If an unexpected situation or difficult road conditions occur while the robot is traveling along this fixed route, it will be hindered, resulting in reduced inspection efficiency.

[0004] Therefore, how to effectively plan the robot's inspection path to improve inspection efficiency is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] This application provides a patrol path planning method, device, storage medium and patrol robot, which can plan the path segment by segment based on the current drivable area of the patrol robot to avoid the patrol robot from

[0006] According to one aspect of the present application, a patrol path planning method is provided, which is applied to a patrol robot. The method includes:

[0007] In response to a patrol event being triggered, obtaining a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located;

[0008] Processing each of the environmental data based on the path decision model to obtain a decision instruction; wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area;

[0009] Performing path planning according to the current position and the decision instruction to determine a first path, and controlling the inspection robot to perform an inspection task based on the first path;

[0010] In response to an event that the first end position is inconsistent with a second end position corresponding to the inspection event, returning to execute an operation of obtaining a current position of the inspection robot and at least one environmental data of an area where the inspection robot is located;

[0011] In response to an event that the first end position is consistent with the second end position or the first path includes the second end position, the inspection robot is controlled to inspect to the second end position along the updated first path.

[0012] According to another aspect of the present application, a patrol route planning device is provided, the device comprising:

[0013] a data acquisition module, configured to acquire, in response to a patrol event being triggered, a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located;

[0014] A path decision module, configured to process each of the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area;

[0015] a path planning module, configured to perform path planning to determine a first path according to the current position and the decision instruction, and control the inspection robot to perform an inspection task based on the first path;

[0016] a first event response module, configured to, in response to an event that the first end point position is inconsistent with a second end point position corresponding to the inspection event, return to execute an operation of obtaining a current position of the inspection robot and at least one piece of environmental data of an area where the inspection robot is located;

[0017] The second event response module is used to control the inspection robot to patrol to the second end position along the updated first path in response to the event that the first end position is consistent with the second end position or the first path includes the second end position.

[0018] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection path planning method described in any embodiment of the present application when executed.

[0019] According to another aspect of the present application, a patrol robot is provided, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the patrol path planning method described in any embodiment of the present application.

[0020] The technical solution provided by this application obtains the current position of the inspection robot and the environmental data of the area in which it is located in response to a patrol event being triggered; processes the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction includes the first terminal position of the inspection robot in the current drivable area; determines a first path based on the current position and the decision instruction, and controls the inspection robot to perform inspections based on the first path; in response to an event in which the first terminal position is inconsistent with the second terminal position corresponding to the patrol event, returns to execute the acquisition of the current position and environmental data; in response to an event in which the first terminal position is consistent with the second terminal position or the first path includes the second terminal position, controls the inspection robot to patrol along the updated first path to the second terminal position. This technical solution achieves accurate and effective planning of the robot's patrol path and improves the robot's patrol efficiency.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a flowchart of a patrol route planning method provided in Example 1 of the present application.

[0024] Figure 2 This is a flowchart of a patrol route planning method provided in Example 2 of the present application.

[0025] Figure 3 This is a flowchart of a patrol route planning method provided in Example 3 of the present application.

[0026] Figure 4 This is a structural diagram of an inspection path planning device provided in Example 4 of the present application.

[0027] Figure 5 It is a structural diagram of an inspection robot implementing an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0029] It should be noted that the terms "current", "target", "total", "first", "second", "candidate", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Example 1

[0031] Figure 1 This is a flowchart of a patrol path planning method provided in the first embodiment of the present application. This embodiment is applicable to the case of real-time planning of the robot's path. The method can be executed by a patrol path planning device. The patrol path planning device can be implemented in the form of hardware and / or software. The patrol path planning device can be configured in a device with data processing capabilities. Figure 1 As shown, the method includes the following steps.

[0032] S110 : In response to a patrol event being triggered, obtaining a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located.

[0033] Inspection events can be understood as conditions or scenarios that trigger inspection robots to initiate specific response processes. Predefined rules or real-time sensing can be used to identify anomalies or task requirements, driving the inspection robots to execute corresponding actions. Inspection events can involve inspections of specific endpoints, specific areas, or specific targets. Inspection event triggers can be time-based, threshold-based, remotely commanded, or autonomously identified.

[0034] For example, the type of the inspection event is a remote instruction. The user sends an instruction to the inspection robot through the client to inspect device A. The inspection robot starts to run after receiving the remote instruction.

[0035] The current location can be the location parameter of the inspection robot when the inspection event is triggered. Specifically, in outdoor scenarios, the latitude and longitude coordinates of the inspection robot can be obtained through the GPS module; in indoor scenarios, the current location of the inspection robot can be obtained through SLAM (Simultaneous Localization and Mapping) technology combined with LiDAR IMU (Inertial Measurement Unit). In addition, the UWB (Ultra Wide Band) base station can be used to assist in correcting the positioning error of the inspection robot.

[0036] It is understandable that the inspection robot is pre-installed with multiple sensors, such as a gas concentration sensor, a temperature sensor, a radiation dosimeter, a visual sensor, etc., which are used to perceive the environment of the area where the inspection robot is located.

[0037] Environmental data can be quantifiable parameters acquired in real time by the inspection robot using a sensor array that reflect the physical or chemical state of the area it is in. For example, environmental data can include temperature and humidity data, gas concentration data, noise data, optical data, air quality data, electromagnetic field data, terrain data, obstacle data, and so on.

[0038] S120: Process the environmental data based on the path decision model to obtain a decision instruction, wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area.

[0039] A large model refers to an AI model based on a Transformer or similar neural network architecture, trained with massive amounts of data and ultra-large parameters. A large routing decision model can be a model with intelligent decision-making capabilities, trained using a large number of routing decision training samples to train an initial large model. Multimodal environmental data is input into the large routing decision model to generate decision instructions.

[0040] Decision instructions can be understood as instructions from the path decision model to the inspection robot regarding its next action in the current environment. These instructions can include the robot's speed, such as increasing it in open and safe areas and reducing it in crowded areas or under complex road conditions. They can also be action instructions, such as reaching a specific location to take photos, collect samples, or turn certain equipment on or off. They can also be priority assignments for inspection tasks, such as assigning priorities to multiple inspection tasks simultaneously within an inspection event, so that the inspection robot prioritizes the highest-priority task.

[0041] In this embodiment, the decision-making instruction may be an instruction that carries the first terminal position of the inspection robot in the current drivable area. The first terminal position may be the location within the current drivable area that is closest to the terminal position corresponding to the inspection event, or may be the terminal position corresponding to the inspection event. Specifically, the path decision model may determine the current drivable area of the inspection robot based on environmental data. If the terminal position corresponding to the inspection event exists within the current drivable area, this location is used as the first terminal position of the inspection robot in the current drivable area, and this information is sent to the inspection robot in the decision-making instruction.

[0042] Optionally, the path decision model includes an encoder, a feature extraction network and a prediction network; the processing of each environmental data based on the path decision model to obtain a decision instruction includes: based on the encoder in the path decision model, encoding each environmental data separately to obtain encoded data corresponding to each environmental data; based on the feature extraction network in the path decision model, performing feature extraction and fusion processing on each encoded data to obtain high-dimensional data; based on the prediction network in the path decision model, performing prediction processing on the high-dimensional data to obtain a decision instruction.

[0043] It is understandable that the large model is capable of simultaneously processing data collected by various sensors, including visual, temperature, radiation intensity, and gas concentration data. Specifically, the large path decision model performs data encoding, feature extraction, and information fusion on the data collected by the sensors, thereby enhancing the inspection robot's perception capabilities. In this embodiment, multimodal fusion perception can enhance the inspection robot's environmental understanding capabilities, enabling real-time analysis of complex environmental data in extreme environments such as nuclear power plants, and thus providing decision instructions that match the complex environment.

[0044] Environmental data can be categorized as image data, linear data, and interactive command information issued by users to inspection robots, depending on the type of data collected by the sensors. To facilitate unified processing of environmental data of various modalities, this application uses an encoder to encode environmental data of different modalities separately to obtain encoded data corresponding to each environmental data type.

[0045] Based on the characteristics of different modal data, it is necessary to select an appropriate encoder. For example, for image data, convolutional neural networks are usually used as encoders because they can effectively extract the spatial features of images. For audio data, encoders such as Mel-frequency cepstral coefficients or long short-term memory networks are more commonly used. Mel-frequency cepstral coefficients can extract the characteristic parameters of audio, while long short-term memory networks can process the timing information in audio signals. For sensor measurement data, such as numerical data such as temperature and humidity, a simple fully connected neural network can be used for encoding and mapping it to an appropriate feature space. The advantage of this setting is that it improves the perception accuracy and generalization ability of the large path decision model by decoupling the features of heterogeneous data.

[0046] The feature extraction network can be used to extract and fuse features from each encoded data. It is understandable that the encoded data after being encoded by the encoder may contain a large amount of redundant information or may not be the most representative features of the modal data. Therefore, further feature extraction is required. For example, a convolutional neural network layer can be used to extract feature maps, or an attention mechanism can be used to capture important features.

[0047] It should be noted that before extracting and fusing the encoding features, the consistency of multimodal data can be ensured through temporal or spatial alignment, such as using sensor synchronization or spatial transformation networks to adjust the position of feature maps.

[0048] After feature extraction, features can be fused to produce fused features. Specifically, a cross-modal attention mechanism can be used to dynamically calculate the weights of features from each modality to generate context-aware fused features. For example, visual features and LiDAR point clouds can complement each other's spatial information through cross-attention. Features can also be fused at different levels of abstraction, for example, by concatenating low-level features and feeding them into a shared network, and then fusing high-level features through weighted averaging.

[0049] Furthermore, high-dimensional features with a significant impact on path planning can be extracted from the fused features, allowing the large-scale path decision model to predict decision instructions based on these high-dimensional features. Specifically, the fused features can be fed into a multi-layer fully connected network or a deep convolutional neural network to extract high-level abstract features. For example, a multi-layer perceptron can be used to enhance feature representation. Self-attention mechanisms can also be used to strengthen important feature channels, such as by calculating feature channel weights to amplify the influence of high-variance dimensions.

[0050] In the path decision model, the prediction network is used to map high-dimensional features into specific decision instructions, such as end point position, steering angle, speed, braking, etc.

[0051] S130 , performing path planning according to the current position and the decision instruction to determine a first path, and controlling the inspection robot to perform an inspection task based on the first path.

[0052] It is understood that the first path can be classified as an intermediate path or a final path based on whether the first endpoint position in the decision instruction is consistent with the endpoint position corresponding to the inspection event. If the first path is an intermediate path, it indicates that multiple path planning and multiple inspection tasks are required to complete the response to the inspection event. If the first path is a final path, it indicates that the response to the inspection event is completed after executing the inspection task.

[0053] In the present application, since the decision instruction carries the first terminal position of the inspection robot in the current drivable area, path planning can be performed in combination with the current position of the inspection robot to obtain the first path in the current drivable area where the starting position of the inspection robot is the current position and the end position is the first end position.

[0054] Specifically, the first path can be planned using a classic graph search algorithm, for example, based on the Dijkstra algorithm, gradually expanding the set of shortest paths starting from the current position until the first end position, or based on the A* algorithm, optimizing the search direction through a heuristic cost function to obtain the first path; the first path can also be planned using a genetic algorithm or an ant colony algorithm.

[0055] The inspection robot's path in performing inspection tasks is planned once or multiple times through decision-making instructions to achieve flexible planning of the robot's inspection path.

[0056] S140. In response to an event that the first end position is inconsistent with a second end position corresponding to the inspection event, return to executing an operation of obtaining the current position of the inspection robot and at least one environmental data of the area where the inspection robot is located.

[0057] The second endpoint position can be understood as the arrival location specified for the inspection robot in the inspection event. Specifically, if the inspection event directly includes the second endpoint position, it can be directly obtained from the inspection event. If the inspection event does not include the second endpoint position, the inspection event can be parsed and the corresponding second endpoint position can be retrieved from the database based on the parsed results. For example, if the inspection event is taking a photo of device X, the second endpoint position corresponding to the inspection event is a preset photo location around device X.

[0058] In this application, an event in which the first end position is inconsistent with the second end position corresponding to the inspection event can be understood as the inspection robot does not have a second end position corresponding to the inspection event in the current drivable area, and the inspection robot needs to go through multiple path planning to reach the second end position.

[0059] Exemplarily, the environmental data of the area where the patrol robot is located are processed based on the path decision model, and the first terminal position of the patrol robot in the current drivable area is obtained as A, and the second terminal position corresponding to the patrol event is B. The patrol robot is first controlled to patrol along the first path to the first terminal position A, and then returns to execute step S110 to re-acquire the current position of the patrol robot at the first terminal position A and the environmental data of the area where it is located, until the patrol robot reaches the second terminal position B corresponding to the patrol event.

[0060] S150 : In response to an event that the first end position is consistent with the second end position or the first path includes the second end position, control the inspection robot to inspect to the second end position along the updated first path.

[0061] In this application, an event in which the first end position is consistent with the second end position or an event in which the first path includes the second end position can be understood as the inspection robot having a second end position corresponding to the inspection event within the current drivable area, and the inspection robot completes the inspection event by executing this inspection task.

[0062] For example, based on the path decision model, the environmental data of the area where the inspection robot is located are processed to obtain the first terminal position of the inspection robot in the current drivable area as C, and the second terminal position corresponding to the inspection event is also C. Then the inspection robot is directly controlled to patrol to position C along the first path.

[0063] As another example, the environmental data of the area where the inspection robot is located are processed based on the path decision model, and it is obtained that the first path of the inspection robot contains a second terminal position C corresponding to the inspection event. The first path is then re-determined based on the first path and the second terminal position C, and the inspection robot is controlled to patrol to the second terminal position C according to the updated first path.

[0064] An embodiment of the present invention provides a patrol path planning method. The method obtains the current position of a patrol robot and environmental data of the area in which it is located in response to a patrol event being triggered; processes the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction includes the first terminal position of the patrol robot in the current drivable area; determines a first path based on the current position and the decision instruction, and controls the patrol robot to perform patrols based on the first path; in response to an event in which the first terminal position is inconsistent with the second terminal position corresponding to the patrol event, returns to execute the acquisition of the current position and environmental data; in response to an event in which the first terminal position is consistent with the second terminal position or the first path includes the second terminal position, controls the patrol robot to patrol along the updated first path to the second terminal position. This technical solution realizes accurate and effective planning of the robot's patrol path.

[0065] Example 2

[0066] Figure 2 This is a flow chart of a patrol path planning method provided in the second embodiment of the present application. This embodiment is optimized based on the above embodiment, specifically optimizing the process of determining the first path. Figure 2 As shown, the method of this embodiment specifically includes the following steps.

[0067] S210 : In response to a patrol event being triggered, obtaining a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located.

[0068] S220: Process the environmental data based on the path decision model to obtain a decision instruction, wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area.

[0069] S230: Taking the current position as the root node of a random tree, randomly sampling within the current drivable area of the inspection robot according to the environmental data and preset constraints to determine path position points, and updating the random tree according to the path position points.

[0070] Among them, the random tree can be understood as a tree or tree diagram created by a random process. The preset constraints can be the constraints that the inspection robot needs to meet during the overall path planning process, or the constraints that the inspection robot needs to meet during each random sampling process. Exemplarily, the preset constraints can be the maximum turning radius, maximum speed, minimum speed, dynamic obstacle avoidance requirements, etc. of the inspection robot. The path position points can be understood as the tree nodes generated by the random tree during each random sampling process, which are used to expand the random tree and gradually construct the first path. The random sampling method can be uniform sampling, that is, random sampling within the drivable area; it can also be based on the target bias strategy to increase the sampling probability of positions near the first terminal position to accelerate the rapid convergence of the first path.

[0071] For example, the random tree can be initialized first, and the current position is used as the root node of the random tree, that is, the current position is used as the starting point of path planning; random sampling is performed within a preset distance range of the current position according to environmental data and preset constraints to generate random sampling points; the node closest to the random sampling point is found in the random tree as the path position point, and the random tree is updated according to the path position point.

[0072] Optionally, based on the environmental data and preset constraints, random sampling is performed within the current drivable area of the inspection robot to determine the path position points, including: determining the current drivable area of the inspection robot based on the environmental data, and performing random sampling within the current drivable area to determine candidate position points; performing collision detection based on the candidate position points and preset constraints, and if no collision object is detected, determining the candidate position points as path position points.

[0073] The current drivable area can be understood as an obstacle-free area that the inspection robot can move in. Specifically, static obstacles and dynamic obstacles in the surrounding environment of the inspection robot can be identified through environmental data to determine the current drivable area of the inspection robot.

[0074] For example, fixed obstacles such as walls and equipment bases can be marked through SLAM or preset maps based on lidar point cloud data, camera image data or ultrasonic sensor data, and moving objects such as people and other robots can be tracked using Kalman filtering or deep learning models; the identified obstacle information is converted into a geometric representation that can be understood by the inspection robot, such as a grid map, occupancy map, etc.; finally, the current drivable area of the inspection robot is calculated, for example, the current area can be divided into a two-dimensional grid, and the grid where the obstacle is located is marked as "inaccessible", and the remaining grids are marked as "free space". For example, based on the geometric shape of the obstacle, its circumscribed area can be calculated and excluded.

[0075] Collision detection can be understood as determining the spatial relationship between candidate locations and obstacles to ensure that the inspection robot's path from the current position to the candidate location does not physically contact obstacles in the environment. Specifically, this can be done by determining whether the straight path between the current position and the candidate location overlaps with any obstacles. If the candidate location passes collision detection, it is added to the random tree. If the candidate location fails collision detection, the sampling range can be expanded or the sampling strategy can be adjusted to re-randomly sample.

[0076] For example, if the inspection robot needs to move from point A (0, 0) to candidate point B (2, 2), and the obstacle is located in a circular area with a radius of 0.5m at (1, 1), path AB will pass through the obstacle, triggering a collision detection failure.

[0077] The advantage of the above technical solution is that it can not only avoid collisions of inspection robots during the inspection path and ensure the safety of equipment and personnel, but also generate feasible paths in complex scenarios such as narrow passages or multiple obstacles.

[0078] S240. In response to an event that the path position point is inconsistent with the first terminal position, the path position point is used as the parent node of the random tree, and the operation of randomly sampling and determining the path position point within the current drivable area of the inspection robot according to the environmental data and preset constraints is returned.

[0079] The event that the path position point is inconsistent with the first end position can be understood as a situation where the distance between the path position point and the first end position exceeds a preset threshold and the first path planning has not yet been completed.

[0080] In the present application, the Euclidean distance between the path position point and the first terminal position can be calculated first. If it is greater than a preset threshold, the random tree expansion operation is triggered, that is, the path position point is used as the parent node of the random tree to continue the iterative process of path planning until the path position point is consistent with the first terminal position.

[0081] S250 . In response to an event that the path position point is consistent with the first end position, determine a first path according to the updated random tree, and control the inspection robot to perform an inspection task based on the first path.

[0082] The event that the path position point is consistent with the first end position can be understood as a situation where the distance between the path position point and the first end position is less than a preset threshold, and the first path planning has been completed.

[0083] In the present application, the random tree can be traversed in reverse along the parent node pointer from the final path position to the root node, that is, the current position, and the node sequence obtained by backtracing is reversed to generate the first path from the current position to the first terminal position.

[0084] It is understandable that after the first path is generated based on each path position point, a spline curve or a Bezier curve may be used to smooth the first path to reduce wear on the mechanical structure of the inspection robot.

[0085] S260: In response to the event that the first end position is inconsistent with the second end position corresponding to the inspection event, return to the operation of obtaining the current position of the inspection robot and at least one environmental data of the area where the inspection robot is located.

[0086] S270: In response to an event that the first end position is consistent with the second end position or the first path includes the second end position, control the inspection robot to inspect to the second end position along the updated first path.

[0087] An embodiment of the present invention provides a patrol path planning method, which obtains the current position of the patrol robot and at least one environmental data of the area where the patrol robot is located in response to a patrol event being triggered; uses the current position as the root node of a random tree, and randomly samples the current drivable area of the patrol robot according to the environmental data and preset constraints to determine the path position point, and updates the random tree according to the path position point; in response to an event that the path position point is inconsistent with the first end point position, uses the path position point as the parent node of the random tree, returns to execute the random sampling in the current drivable area of the patrol robot according to the environmental data and preset constraints to determine the path point. The present invention includes the following steps: first, determining a path point; in response to an event in which a path point is consistent with a first end point, determining a first path based on an updated random tree; performing path planning based on the current position and the decision instruction to determine the first path, and controlling the inspection robot to perform the inspection task based on the first path; in response to an event in which the first end point is inconsistent with a second end point corresponding to the inspection event, returning to the operation of obtaining the current position of the inspection robot and at least one piece of environmental data of the area in which the inspection robot is located; in response to an event in which the first end point is consistent with the second end point or the first path includes the second end point, controlling the inspection robot to patrol along the updated first path to the second end point. This technical solution improves the efficiency and accuracy of path planning by generating a random tree to plan the path.

[0088] Example 3

[0089] Figure 3This is a flow chart of a patrol path planning method provided in the second embodiment of the present application. This embodiment is optimized based on the above embodiment, specifically for the case where the patrol robot is a multi-habitat robot. Figure 3 As shown, the method of this embodiment specifically includes the following steps.

[0090] S310: In response to a patrol event being triggered, obtaining the current position of the patrol robot and at least one piece of environmental data of the area where the patrol robot is located.

[0091] A multi-robot system is one that can flexibly switch between motion modes and perform tasks in at least two physical environments. For example, an inspection robot can be a land-air robot, a water-land robot, or a water-air robot.

[0092] S320: Process the environmental data based on the path decision model to obtain a decision instruction, wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area and the first operating mode of the inspection robot.

[0093] The first operating mode can be understood as the operating state of the multi-habitat robot in different physical environments. For example, the first operating mode can be a land mode, an air mode, an underwater mode, etc.

[0094] Specifically, the path decision model processes the environmental data and outputs an operation mode switching instruction if it detects that the inspection robot's current operation mode is difficult to pass. The operation mode switching instruction includes the operation mode to be switched to by the inspection robot, that is, the first operation mode.

[0095] For example, the current operating mode of the land-air amphibious inspection robot is land mode, but after the path decision model processes the environmental data of the current position, it determines that the inspection robot can no longer continue to travel in land mode, and then it will output a land-air conversion instruction, that is, the first operating mode of the inspection robot is air mode.

[0096] Optionally, before obtaining the current position of the inspection robot and at least one environmental data of the area where the inspection robot is located, it also includes: obtaining the total sensor list of the inspection robot, the current operating mode of the inspection robot and the first sensor list corresponding to the current operating mode; determining the activation status of each sensor in the total sensor list based on the total sensor list and the first sensor list.

[0097] It is understandable that for a multi-habitat robot, the environmental data required in different environments may vary. For example, when planning an aerial path, the environmental data affecting path planning may include flight form, aerial obstacles, wind speed, etc.; while when planning a land path, the environmental data affecting path planning may include terrain, slope, ground obstacles, etc.

[0098] Therefore, different sensor lists can be pre-set for different operating modes, wherein the sensor lists are used to record the sensors corresponding to the operating modes.

[0099] Specifically, the current operating mode of the patrol robot and the first sensor list corresponding to the current operating mode can be obtained to determine the sensors that need to be enabled in the current operating mode of the patrol robot; the total sensor list of the patrol robot can be obtained and combined with the first sensor list to determine the sensors that need to be turned off in the current operating mode of the patrol robot.

[0100] Based on the enabled state of each sensor in the total sensor list, the enabled state of each sensor is tested to ensure that the sensors that should be in the enabled state are running and the sensors that should be in the disabled state are disabled.

[0101] The beneficial effect of the above technical solution is that, for sensors that are not needed in the current operating mode, the shutdown instructions can be used to control the shutdown of the unnecessary sensors, thereby improving the endurance of the inspection robot.

[0102] S330: Perform path planning according to the current position and the decision instruction to determine a first path, and control the inspection robot to perform an inspection task based on the first path in a first operating mode.

[0103] In the present application, it is possible to first determine whether the operating mode of the inspection robot needs to be switched based on whether the first operating mode in the decision instruction is consistent with the current operating mode; if the operating mode of the inspection robot needs to be switched, the inspection robot is controlled to switch the current operating mode to the first operating mode. After switching to the first operating mode, the process can return to step S310 to re-acquire the environmental data under the first operating mode so as to determine a new decision instruction and the first path based on the new environmental data; if the operating mode of the inspection robot does not need to be switched, the inspection robot is controlled to perform the inspection task based on the first path in the first operating mode.

[0104] Optionally, the path planning based on the current position and the decision instruction to determine the first path includes: when the first operating mode is the aerial mode, obtaining the current posture information of the inspection robot; wherein the current posture information includes pitch angle information and yaw angle information; and path planning based on the current position, the current posture information and the decision instruction to determine the first path.

[0105] The current posture information refers to a quantitative description of the inspection robot's position, orientation, and motion state in three-dimensional space. Position can be the robot's coordinates in space, such as two-dimensional (x, y) or three-dimensional (x, y, z), typically referenced to the Earth coordinate system or a local coordinate system. Orientation can be Euler angles describing the robot's rotation around three axes, such as pitch, yaw, and roll. Quaternions can also be used to represent rotations to avoid the "gimbal lock" problem associated with Euler angles. A 3*3 matrix can also be used to describe rotational transformations between coordinate systems. Motion state can be linear velocity, angular velocity, linear acceleration, or angular acceleration.

[0106] Specifically, the current attitude information can be obtained through sensors such as inertial measurement units, magnetometers, visual sensors, GPS and star sensors; it can also be obtained through data fusion algorithms, such as fusing multi-source data such as inertial measurement units, magnetometers, and visual sensors based on Kalman filtering to improve the estimation accuracy of attitude information, or based on complementary filtering, combining the advantages of high-frequency data and low-frequency data to suppress noise.

[0107] In this application, a path planning based on the three-dimensional spatial degrees of freedom of the inspection robot can be performed according to the current position, current posture information, and decision instructions to obtain a first path. The three-dimensional spatial degrees of freedom refer to the ability of the inspection robot to move along three independent directions in three-dimensional space. These three directions generally include translational degrees of freedom and rotational degrees of freedom. Translational degrees of freedom refer to the ability of an object to move along three different axes (such as the x, y, and z axes), while rotational degrees of freedom refer to the ability of an object to rotate around these three axes.

[0108] The benefits of this technical solution are that it increases the flexibility of aerial path planning and improves obstacle avoidance capabilities during the process. Specifically, by leveraging the translational degree of freedom, more complex and efficient flight paths can be planned; while by leveraging the rotational degree of freedom, the inspection robot can adjust its posture mid-air, thereby adjusting its flight direction to avoid obstacles.

[0109] S340: In response to an event that the first end position is inconsistent with the second end position corresponding to the inspection event, return to the operation of obtaining the current position of the inspection robot and at least one environmental data of the area where the inspection robot is located.

[0110] S350: In response to an event that the first end position is consistent with the second end position or the first path includes the second end position, control the inspection robot to inspect to the second end position along the updated first path.

[0111] An embodiment of the present invention provides a patrol path planning method, which obtains the current position of a multi-habitat patrol robot and the environmental data of the area in which it is located in response to a patrol event being triggered; processes the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction includes the first operating mode of the patrol robot and the first terminal position in the current drivable area; determines the first path according to the current position and the decision instruction, and controls the patrol robot to patrol based on the first path in the first operating mode; in response to an event that the first terminal position is inconsistent with the second terminal position corresponding to the patrol event, returns to execute the acquisition of the current position and environmental data; in response to an event that the first terminal position is consistent with the second terminal position or the first path includes the second terminal position, controls the patrol robot to patrol to the second terminal position along the updated first path. This technical solution can improve the patrol robot's patrol capability in different operating environments by identifying the operating mode of the patrol robot.

[0112] Example 4

[0113] Figure 4 This is a schematic diagram of the structure of a patrol route planning device provided in the fourth embodiment of the present application. Figure 4 As shown, the device includes:

[0114] The data acquisition module 410 is configured to acquire, in response to a patrol event being triggered, the current position of the patrol robot and at least one piece of environmental data of the area where the patrol robot is located;

[0115] A path decision module 420 is configured to process the environmental data based on the path decision model to obtain a decision instruction; wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area;

[0116] a path planning module 430, configured to perform path planning to determine a first path according to the current position and the decision instruction, and control the inspection robot to perform an inspection task based on the first path;

[0117] A first event response module 440 is configured to, in response to an event that the first end point position is inconsistent with a second end point position corresponding to the inspection event, return to execute an operation of obtaining the current position of the inspection robot and at least one piece of environmental data of the area where the inspection robot is located;

[0118] The second event response module 450 is used to control the inspection robot to patrol to the second end position along the updated first path in response to the event that the first end position is consistent with the second end position or the first path includes the second end position.

[0119] An embodiment of the present invention provides a patrol path planning device. The device, in response to a patrol event being triggered, obtains the current position of a patrol robot and environmental data of the area in which it is located; processes the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction includes the first terminal position of the patrol robot in the current drivable area; determines a first path based on the current position and the decision instruction, and controls the patrol robot to perform patrols based on the first path; in response to an event in which the first terminal position is inconsistent with the second terminal position corresponding to the patrol event, returns to execute the acquisition of the current position and environmental data; in response to an event in which the first terminal position is consistent with the second terminal position or the first path includes the second terminal position, controls the patrol robot to patrol along the updated first path to the second terminal position. This technical solution achieves accurate and effective planning of the robot's patrol path.

[0120] Furthermore, the path decision model includes an encoder, a feature extraction network, and a prediction network; the path decision module 420 includes:

[0121] A data encoding unit, configured to encode each of the environmental data based on an encoder in the path decision model to obtain encoded data corresponding to each of the environmental data;

[0122] A feature extraction unit is used to extract and fuse the features of each of the encoded data based on the feature extraction network in the path decision model to obtain high-dimensional data;

[0123] The data prediction unit is used to perform prediction processing on the high-dimensional data based on the prediction network in the path decision model to obtain decision instructions.

[0124] Furthermore, the path planning module 430 includes:

[0125] a random sampling unit, configured to use the current position as the root node of a random tree, perform random sampling within the current drivable area of the inspection robot according to the environmental data and preset constraints to determine path position points, and update the random tree according to the path position points;

[0126] a random tree expansion unit, configured to, in response to an event that the path position point is inconsistent with the first end position, use the path position point as a parent node of the random tree and return to executing the operation of randomly sampling and determining the path position point within the current drivable area of the inspection robot based on the environmental data and preset constraints;

[0127] A first path determining unit is configured to determine a first path according to the updated random tree in response to an event that the path position point is consistent with the first end point position.

[0128] Furthermore, the random sampling unit is specifically used to:

[0129] Determine the current drivable area of the inspection robot according to the environmental data, and perform random sampling within the current drivable area to determine candidate location points;

[0130] A collision detection is performed based on the candidate position point and preset constraint conditions. If no collision object is detected, the candidate position point is determined as a path position point.

[0131] Furthermore, the inspection robot is a multi-habitat robot, and the decision instruction further includes a first operating mode of the inspection robot;

[0132] The path planning module 430 is specifically configured to:

[0133] The inspection robot is controlled to perform an inspection task based on the first path in a first operating mode.

[0134] Furthermore, the device further comprises:

[0135] a sensor list acquisition module, configured to acquire a total sensor list of the inspection robot, a current operating mode of the inspection robot, and a first sensor list corresponding to the current operating mode before acquiring the current position of the inspection robot and at least one environmental data of the area where the inspection robot is located;

[0136] A sensor status determination module is configured to determine an activation status of each sensor in the total sensor list according to the total sensor list and the first sensor list.

[0137] Furthermore, the path planning module 430 includes:

[0138] A posture information acquisition unit, configured to acquire current posture information of the inspection robot when the first operating mode is the aerial mode; wherein the current posture information includes pitch angle information and yaw angle information;

[0139] The first path planning unit is configured to perform path planning based on the current position, the current posture information, and the decision instruction to determine a first path.

[0140] A patrol path planning device provided in an embodiment of the present application can execute a patrol path planning method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

[0141] Example 5

[0142] Figure 5 The following is a schematic diagram of the structure of an inspection robot 10 that can be used to implement embodiments of the present application. The inspection path planning device provided by embodiments of the present application can be integrated into the inspection robot 10. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0143] like Figure 5 As shown, the inspection robot 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0144] Several components of the inspection robot 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard and mouse; an output unit 17, such as various types of displays and speakers; a storage unit 18, such as a magnetic disk and optical disk; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0145] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the inspection path planning method.

[0146] In some embodiments, the inspection path planning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the inspection robot 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the inspection path planning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the inspection path planning method in any other appropriate manner (for example, by means of firmware).

[0147] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0151] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0152] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0153] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0154] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A patrol route planning method, characterized in that: Applied to an inspection robot, the method includes: In response to a patrol event being triggered, obtaining a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located; Processing each of the environmental data based on the path decision model to obtain a decision instruction; wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area; Performing path planning according to the current position and the decision instruction to determine a first path, and controlling the inspection robot to perform an inspection task based on the first path; In response to an event that the first end position is inconsistent with a second end position corresponding to the inspection event, returning to execute an operation of obtaining a current position of the inspection robot and at least one environmental data of an area where the inspection robot is located; In response to an event that the first end position is consistent with the second end position or the first path includes the second end position, the inspection robot is controlled to inspect to the second end position along the updated first path.

2. The method according to claim 1, characterized in that The path decision model includes an encoder, a feature extraction network, and a prediction network; the process of processing the environmental data based on the path decision model to obtain a decision instruction includes: Based on the encoder in the path decision model, each of the environmental data is encoded respectively to obtain encoded data corresponding to each of the environmental data; Based on the feature extraction network in the path decision model, feature extraction and fusion processing are performed on each of the encoded data to obtain high-dimensional data; Based on the prediction network in the path decision model, the high-dimensional data is predicted and processed to obtain decision instructions.

3. The method according to claim 1, characterized in that The performing path planning according to the current position and the decision instruction to determine the first path includes: Taking the current position as the root node of a random tree, randomly sampling within the current drivable area of the inspection robot according to the environmental data and preset constraints to determine path position points, and updating the random tree according to the path position points; In response to an event that the path location point is inconsistent with the first end location, taking the path location point as a parent node of the random tree, and returning to execute the operation of randomly sampling and determining the path location point within the current drivable area of the inspection robot based on the environmental data and preset constraints; In response to an event that the path location point is consistent with the first end location, a first path is determined according to the updated random tree.

4. The method according to claim 3, characterized in that According to the environmental data and preset constraints, random sampling is performed within the current drivable area of the inspection robot to determine a path position point, including: Determine the current drivable area of the inspection robot according to the environmental data, and perform random sampling within the current drivable area to determine candidate location points; A collision detection is performed based on the candidate position point and preset constraint conditions. If no collision object is detected, the candidate position point is determined as a path position point.

5. The method according to claim 1, characterized in that The inspection robot is a multi-habitat robot, and the decision instruction further includes a first operating mode of the inspection robot; The inspection robot performs an inspection task based on the first path, including: The inspection robot is controlled to perform an inspection task based on the first path in a first operating mode.

6. The method according to claim 5, characterized in that Before obtaining the current position of the inspection robot and at least one piece of environmental data of the area where the inspection robot is located, the method further includes: Obtaining a total sensor list of the inspection robot, a current operating mode of the inspection robot, and a first sensor list corresponding to the current operating mode; An activation state of each sensor in the total sensor list is determined according to the total sensor list and the first sensor list.

7. The method according to claim 5, characterized in that The performing path planning according to the current position and the decision instruction to determine the first path includes: When the first operating mode is the aerial mode, obtaining current posture information of the inspection robot; wherein the current posture information includes pitch angle information and yaw angle information; Path planning is performed according to the current position, the current posture information, and the decision instruction to determine a first path.

8. A patrol route planning device, characterized in that: Applied to an inspection robot, the device comprises: a data acquisition module, configured to acquire, in response to a patrol event being triggered, a current position of the patrol robot and at least one piece of environmental data of an area where the patrol robot is located; A path decision module, configured to process each of the environmental data based on a path decision model to obtain a decision instruction; wherein the decision instruction carries the first terminal position of the inspection robot in the current drivable area; a path planning module, configured to perform path planning to determine a first path according to the current position and the decision instruction, and control the inspection robot to perform an inspection task based on the first path; a first event response module, configured to, in response to an event that the first end point position is inconsistent with a second end point position corresponding to the inspection event, return to execute an operation of obtaining a current position of the inspection robot and at least one environmental data of an area where the inspection robot is located; The second event response module is used to control the inspection robot to patrol to the second end position along the updated first path in response to the event that the first end position is consistent with the second end position or the first path includes the second end position.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection path planning method according to any one of claims 1 to 7 when executed.

10. A patrol robot, characterized in that: It includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the inspection path planning method described in any one of claims 1 to 7.

Images