Inspection control method, device and equipment of inspection robot and storage medium

Abstract

The invention relates to the technical field of power plant inspection, in particular to an inspection control method, device and equipment of an inspection robot and a storage medium. Determining an environment category of an area where the inspection robot is located; path planning is carried out through a target planning model corresponding to the environment category based on the inspection information, and a target advancing path is obtained; and controlling the inspection robot to carry out mobile inspection based on the target advancing path, so that path planning in different types of environments is realized, inspection control of the inspection robot in different types of environments is realized, and the technical problem of relatively low inspection efficiency of equipment in a power plant in the prior art is avoided.

Description

Technical Field

[0001] The present invention relates to the field of power plant inspection technology, and in particular to an inspection control method, device, equipment and storage medium of an inspection robot. Background Art

[0002] During the operation of a power plant, it is necessary to frequently inspect the equipment in the power plant to monitor the operating status of each device and avoid equipment failure. The traditional means of monitoring power plant equipment is mainly through manual inspection to record the operating data of each device. However, this method has a large workload and cannot provide timely operating status feedback for each device, affecting the inspection efficiency.

[0003] With the development of intelligent robots, there are also examples of patrol inspections by robots in the existing technology. However, during the patrol inspection process, the robot can only patrol according to the set route, cannot adapt to the relatively loaded indoor environment and the relatively wide outdoor environment, and the effect is single, generally only for the purpose of collecting data. It is impossible to self-check the status of the equipment.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an inspection control method, device, equipment and storage medium for an inspection robot, aiming to solve the technical problem of low efficiency in inspecting equipment in power plants in the prior art.

[0006] To achieve the above object, the present invention provides an inspection control method of an inspection robot, the method comprising the following steps:

[0007] Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located;

[0008] Determining the environmental category of the area where the inspection robot is located;

[0009] Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path;

[0010] The inspection robot is controlled to perform mobile inspection based on the target travel path.

[0011] Optionally, the inspection information includes at least: image information and radar positioning information;

[0012] The path planning is performed based on each inspection information through the target planning model corresponding to the environment category to obtain the target travel path, including:

[0013] When the environment category is an outdoor environment, determining the current position of the inspection robot according to the image information and the radar positioning information;

[0014] Obtaining an inspection task list of the inspection robot to determine a device to be inspected and a target position of the device to be inspected;

[0015] According to the current location and the target location, a theoretical map is searched through a preset database;

[0016] Generate an initial travel path according to the current position, the target position and the theoretical map;

[0017] When there are obstacles in the initial travel path, the path is replanned using an outdoor path planning model to obtain a target travel path.

[0018] Optionally, generating an initial travel path according to the current position, the target position and the theoretical map includes:

[0019] Divide the theoretical map into cells to obtain a unit map, wherein the size of the divided cells is the same as the size of the inspection robot;

[0020] marking a straight line between the current position and the target position in the unit map;

[0021] Extract the target cell through which the straight line passes in the cell map;

[0022] An initial travel path is generated based on the target cell.

[0023] Optionally, re-planning the path using an outdoor path planning model to obtain a target travel path includes:

[0024] marking the obstacle cell occupied by the obstacle in the cell map;

[0025] Determine the adjacent cells of the cell where the current position is located;

[0026] Generate at least two derived travel paths according to the adjacent cells and the initial travel path, wherein the cells in the derived travel paths are adjacent cells of each cell in the initial travel path in a direction along a straight line between the current position and the target position;

[0027] calculating a Manhattan distance of the derived travel path and a straight-line Euclidean distance between the current position and the target position;

[0028] The target travel path in each derived travel path is determined according to the deviation between the Manhattan distance and the Euclidean distance.

[0029] Optionally, the inspection information further includes: laser point cloud information;

[0030] The inspection control method of the inspection robot further includes:

[0031] When the environment category is an indoor environment, the laser point cloud information is subjected to dimensionality reduction processing to obtain a ground point cloud depth image;

[0032] Calculating a smoothing value of the ground point cloud depth image;

[0033] Segmenting the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images;

[0034] Feature extraction is performed on each depth sub-image of the ground point cloud to obtain multiple target point cloud sets;

[0035] Path planning is performed according to the target point cloud set and a preset map of the indoor environment to obtain a target travel path.

[0036] Optionally, controlling the inspection robot to perform mobile inspection based on the target travel path includes:

[0037] When the environment category is an indoor environment, acquiring image information at consecutive time intervals;

[0038] Perform obstacle judgment on the image information according to the multiple target point cloud sets, and update the target travel path in real time according to the judgment result;

[0039] The inspection robot is controlled to perform mobile inspection according to the updated target travel path.

[0040] Optionally, after controlling the inspection robot to perform mobile inspection based on the target travel path, the method further includes:

[0041] Collecting images of equipment during the mobile inspection process of the inspection robot;

[0042] Performing instrument detection on the device image through a target detection model, and segmenting the instrument area image in the device image according to the detection result;

[0043] Correcting the instrument area image to obtain a target instrument area image;

[0044] Identifying equipment instrument information in the target instrument area image;

[0045] Perform equipment self-check based on the equipment instrument information.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also proposes an inspection control device of an inspection robot, the inspection control device of the inspection robot comprising:

[0047] An acquisition module, used to acquire device images during the mobile inspection process of the inspection robot;

[0048] A segmentation module, used to perform instrument detection on the device image through a target detection model, and segment the instrument area image in the device image according to the detection result;

[0049] A correction module, used for correcting the instrument area image to obtain a target instrument area image;

[0050] An identification module, used for identifying equipment instrument information in the target instrument area image;

[0051] The self-check module is used to perform equipment self-check based on the equipment instrument information.

[0052] In addition, to achieve the above-mentioned objectives, the present invention also proposes a patrol control device of a patrol robot, the patrol control device of the patrol robot comprising: a memory, a processor, and a patrol control program of the patrol robot stored in the memory and executable on the processor, the patrol control program of the patrol robot being configured to implement the steps of the patrol control method of the patrol robot as described above.

[0053] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a patrol control program of a patrol robot is stored. When the patrol control program of the patrol robot is executed by a processor, the steps of the patrol control method of the patrol robot as described above are implemented.

[0054] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the inspection control method of the inspection robot as described above.

[0055] One or more technical solutions proposed in the present invention have at least the following technical effects: the present invention obtains inspection information of a patrol robot in multiple dimensions of the area where it is located; determines the environmental category of the area where the patrol robot is located; performs path planning based on each inspection information through a target planning model corresponding to the environmental category to obtain a target travel path; controls the patrol robot to perform mobile inspection based on the target travel path, realizes path planning in different types of environments, and thus realizes patrol control of the patrol robot in different types of environments, avoiding the technical problem of low efficiency in patrolling equipment in power plants in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0058] Figure 1 It is a flow chart of a first embodiment of the inspection control method of the inspection robot of the present invention;

[0059] Figure 2 It is a flow chart of a second embodiment of the inspection control method of the inspection robot of the present invention;

[0060] Figure 3 A schematic diagram of initial path planning of an inspection control method of an inspection robot according to an embodiment of the present invention;

[0061] Figure 4 A schematic diagram of derived path planning of an inspection control method of an inspection robot according to an embodiment of the present invention;

[0062] Figure 5 It is a flow chart of a third embodiment of the inspection control method of the inspection robot of the present invention;

[0063] Figure 6 It is a flowchart of a fourth embodiment of the inspection control method of the inspection robot of the present invention;

[0064] Figure 7 It is a structural block diagram of a first embodiment of the inspection control device of the inspection robot of the present invention;

[0065] Figure 8 It is a structural schematic diagram of an inspection control device of an inspection robot in a hardware operating environment involved in an embodiment of the present invention.

[0066] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0068] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0069] Based on this, an embodiment of the present invention provides an inspection control method of an inspection robot, referring to Figure 1 , Figure 1 The figure is a flow chart of a first embodiment of a patrol control method of a patrol robot according to the present invention.

[0070] In this embodiment, the inspection control method of the inspection robot includes:

[0071] Step S10: Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located.

[0072] Step S20: Determine the environment category of the area where the inspection robot is located.

[0073] Step S30: Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path.

[0074] Step S40: controlling the inspection robot to perform mobile inspection based on the target travel path.

[0075] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an inspection robot inspection control, etc. The following takes the inspection robot inspection control as an example to illustrate this embodiment and the following embodiments.

[0076] It should be noted that since the installation environment of power plant equipment includes indoor and outdoor environments, in order to improve the mobile inspection capabilities of the inspection robot in different environments and obtain accurate equipment images and self-inspection efficiency, this embodiment will perform path planning in corresponding modes according to the environmental category of the area where the inspection robot is located.

[0077] Among them, the inspection information of multiple dimensions includes at least image information, radar positioning information and laser point cloud information. In outdoor environments, since the outdoor environment is relatively wide and there are fewer obstacles affecting the movement of the inspection robot, the use of image information and radar positioning can improve the accuracy of the inspection robot in the path planning process; in indoor environments, the installation of various equipment occupies a large amount of space, and it is even more necessary to update the inspection robot's travel path in real time to improve inspection efficiency.

[0078] In this embodiment, when determining the environmental category of the area where the inspection robot is located, the position of the inspection robot can be located based on image information and radar positioning, and then the environment of the area where the inspection robot is located can be judged whether it is an indoor environment or an outdoor environment based on the floor plan of the power plant.

[0079] This embodiment obtains inspection information of the inspection robot in multiple dimensions of the area where it is located; determines the environmental category of the area where the inspection robot is located; performs path planning based on each inspection information through a target planning model corresponding to the environmental category to obtain a target travel path; controls the inspection robot to perform mobile inspection based on the target travel path, thereby realizing path planning in different types of environments, thereby realizing inspection control of the inspection robot in different types of environments, avoiding the technical problem of low efficiency in inspecting equipment in power plants in the prior art.

[0080] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S30, comprising:

[0081] Step S301: When the environment category is an outdoor environment, the current position of the inspection robot is determined according to the image information and radar positioning information.

[0082] Step S302: Obtain the inspection task list of the inspection robot to determine the equipment to be inspected and the target position of the equipment to be inspected.

[0083] Step S303: querying a theoretical map through a preset database according to the current position and the target position.

[0084] Step S304: Generate an initial travel path according to the current position, the target position and the theoretical map.

[0085] Step S305: When there are obstacles in the initial travel path, re-plan the path through the outdoor path planning model to obtain the target travel path.

[0086] In the specific implementation, the inspection task list of the inspection robot includes the location and information of each device that needs to be inspected, so that the inspection robot can plan the path in the outdoor environment and realize the inspection of each device in the inspection task list. Devices without collected device information or devices that do not need to be inspected by the inspection robot will be marked as devices to be inspected. The target position of the devices to be inspected can be queried through the devices to be inspected in the inspection task list.

[0087] The preset database stores theoretical maps of various areas within the power plant. The theoretical maps include the layout locations of various equipment and obstacle locations, etc., to facilitate the subsequent generation of an initial travel path and provide an initial travel direction.

[0088] Since the outdoor environment may be affected by factors such as equipment damage and falling obstacles, which may cause obstacles to appear on the initial travel path, when there are obstacles in the initial travel path, the path is re-planned through the outdoor path planning model to obtain the target travel path and improve the inspection efficiency of the inspection robot.

[0089] Further, generating an initial travel path according to the current position, the target position and the theoretical map includes:

[0090] Divide the theoretical map into cells to obtain a unit map, wherein the size of the divided cells is the same as the size of the inspection robot;

[0091] marking a straight line between the current position and the target position in the unit map;

[0092] Extract the target cell through which the straight line passes in the cell map;

[0093] An initial travel path is generated based on the target cell.

[0094] In the specific implementation, refer to Figure 3 , Figure 3 For the unit map divided in this embodiment, the current position of the inspection robot is taken as the starting point, the target position of the equipment to be inspected is taken as the destination point, and the cells involved in the straight line between the starting point and the destination point are taken as the initial travel path. During this process, the inspection robot is allowed to move in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right.

[0095] Furthermore, re-planning the path through the outdoor path planning model to obtain the target travel path includes:

[0096] marking the obstacle cell occupied by the obstacle in the cell map;

[0097] Determine the adjacent cells of the cell where the current position is located;

[0098] Generate at least two derived travel paths according to the adjacent cells and the initial travel path, wherein the cells in the derived travel paths are adjacent cells of each cell in the initial travel path in a direction along a straight line between the current position and the target position;

[0099] calculating a Manhattan distance of the derived travel path and a straight-line Euclidean distance between the current position and the target position;

[0100] The target travel path in each derived travel path is determined according to the deviation between the Manhattan distance and the Euclidean distance.

[0101] In this embodiment, reference Figure 4 , Figure 4 This is a unit map when there are obstacles on the initial travel path. The cells occupied by the obstacles in the unit map are marked. Since the division of cells in this embodiment is based on the size of the inspection robot, if a cell is completely or partially occupied by an obstacle, the inspection robot will be unable to pass.

[0102] Based on this, this embodiment expands at least two derived travel paths adjacent to the initial travel path based on the direction of the initial travel path, thereby reducing the cost of the inspection robot's movement and the workload of re-path planning.

[0103] Among them, Manhattan distance refers to the movement cost of the inspection robot on two adjacent cells when it moves from the initial travel distance to the derived travel path. Euclidean distance is used to characterize the distance cost of the straight line from the current position to the target position. The deviation between Manhattan distance and the Euclidean distance can be used to characterize the movement cost of the inspection robot switching from the initial travel path to the derived travel path. The derived travel path corresponding to the minimum value of the deviation between Manhattan distance and the Euclidean distance is used as the final target travel path. This method not only reduces the workload of re-path planning, but also reduces the path movement cost of the inspection robot.

[0104] Accordingly, the step S30 further includes:

[0105] Step S301: When the environment category is an indoor environment, the laser point cloud information is subjected to dimensionality reduction processing to obtain a ground point cloud depth image.

[0106] Step S302: Calculate the smoothing value of the ground point cloud depth image.

[0107] Step S303: Segment the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images.

[0108] Step S304: Extract features from each ground point cloud depth sub-image to obtain multiple target point cloud sets.

[0109] Step S305`: performing path planning according to the target point cloud set and the preset map of the indoor environment to obtain a target travel path.

[0110] It should be noted that the indoor environment is different from the outdoor environment. There are a large number of electrical equipment installed in the indoor environment of the power plant, and the wiring between the equipment is complicated. In addition, the image information and radar information will be interfered by the equipment and electromagnetic fields, and the travel path cannot be accurately formulated. As a result, simple path planning algorithms are difficult to achieve path planning in indoor environments, affecting the inspection efficiency of the inspection robot.

[0111] It can be understood that laser point cloud information is a data set of spatial points obtained by scanning with a three-dimensional laser radar device. Each point cloud contains three-dimensional coordinates and laser reflection intensity, and the distance information of the target object is measured by the speed of light and the time information from laser emission to return.

[0112] Since laser point cloud information is three-dimensional information, when participating in path planning, point cloud information that is not on the ground will interfere with the path planning of the inspection robot. For example, obstacles located at high altitudes will not affect normal passage during the actual inspection process, but in the laser point cloud information, the obstacle will be displayed as overlapping with the inspection robot, increasing the difficulty of data processing in the path planning process.

[0113] Therefore, in this embodiment, the three-dimensional laser point cloud information is first subjected to dimensionality reduction processing, and only the ground point cloud depth image is retained. Specifically, the dimensionality reduction processing refers to segmenting the ground point cloud information and non-ground point cloud information according to the pitch angle information of each point cloud information, and retaining the ground point cloud information, thereby removing the coordinate information in the spatial dimension and obtaining the ground point cloud depth image.

[0114] The calculation formula of the smoothing value is:

[0115]

[0116] Among them, ri and rj are the Euclidean distances between any two points and the sensor of the inspection robot, and S is a set of point clouds with uniform height in the ground point cloud depth image.

[0117] By calculating the smoothness value between two point clouds, it is determined whether the two points belong to the same subject. If they do, it means that the area cannot be passed. Finally, the extracted point cloud set and the preset map of the indoor environment are combined for path planning to improve the robot's inspection efficiency in complex indoor environments.

[0118] Further, controlling the inspection robot to perform mobile inspection based on the target travel path includes:

[0119] When the environment category is an indoor environment, acquiring image information at consecutive time intervals;

[0120] Perform obstacle judgment on the image information according to the multiple target point cloud sets, and update the target travel path in real time according to the judgment result;

[0121] The inspection robot is controlled to perform mobile inspection according to the updated target travel path.

[0122] It should be noted that, taking into account the complexity and interference of the indoor environment, in this embodiment, when the inspection robot is in an indoor environment, during the inspection process of the robot, the robot can obtain image frames at continuous time intervals and determine whether there are sudden obstacles through adjacent image frames. If so, the robot returns to the step of obtaining inspection information of multiple dimensions of the inspection robot in the area where it is located and re-plans the path.

[0123] This embodiment performs dimensionality reduction processing on the laser point cloud information to obtain a ground point cloud depth image when the environment category is an indoor environment; calculates a smoothing value of the ground point cloud depth image; segments the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images; performs feature extraction on each ground point cloud depth sub-image to obtain a plurality of target point cloud sets; performs path planning according to the target point cloud sets and a preset map of the indoor environment to obtain a target travel path, and improves the inspection efficiency of the inspection robot in the indoor environment by performing path planning in advance and updating the path in real time during the inspection process.

[0124] refer to Figure 5 Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description and will not be described in detail later. In this embodiment, the inspection control method of the inspection robot further includes:

[0125] Step S50: collecting images of equipment during the mobile inspection process of the inspection robot.

[0126] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, information processing of an inspection robot, etc. The following takes the information processing of an inspection robot as an example to illustrate this embodiment and the following embodiments.

[0127] It should be noted that during the inspection process in the power plant, the inspection robot generally moves according to the set target path, thereby meeting the inspection task requirements in the inspection task list, ensuring the normal inspection frequency of various equipment in the power plant, and avoiding equipment damage.

[0128] Equipment images include but are not limited to appearance images and instrument images of various types of electrical equipment. Appearance images are used to detect whether obvious faults have occurred, and instrument images are used to be provided to cloud servers to detect the operating status of various types of electrical equipment, so as to timely inspect and maintain the electrical equipment.

[0129] Step S60: Perform instrument detection on the device image using a target detection model, and segment the instrument area image in the device image according to the detection result.

[0130] It should be understood that the target detection model can be a target detection model based on YOLOv5, machine learning or deep learning, which is mainly used to denoise the collected device images, thereby retaining the instrument area image in the device image and reducing the data extraction interference caused by environmental noise and the workload of information processing.

[0131] After performing instrument detection on the device image through the target detection model, the boundary of the instrument area can be determined, so that the boundary of the instrument area can be framed and marked in the device image, and finally the instrument area image in the device image can be segmented by image cropping.

[0132] Since the inspection robot is affected by the environment in which it is located when collecting equipment images, such as incident light, dust, and electromagnetic interference, in this embodiment, before detecting the equipment image, the equipment image can be preprocessed, that is, the equipment image can be smoothed, filtered, enhanced, and denoised to improve the image recognizability and reduce noise interference.

[0133] Step S70: Correcting the instrument area image to obtain a target instrument area image.

[0134] It is understandable that due to the influence of factors such as shooting angle or dust obstruction, the collected instrument area image may not be directly facing the camera or may be partially obstructed. By correcting the instrument area image, the collected instrument area image can be restored to a straight and complete target instrument area image, so as to improve the efficiency of subsequent identification of device instrument information.

[0135] Furthermore, the correcting the instrument area image to obtain the target instrument area image includes:

[0136] Extracting the characteristic pixel points to be selected in the instrument area image;

[0137] Matching theoretical feature pixel points corresponding to the feature pixel points to be selected in the theoretical equipment instrument image;

[0138] Angle mapping is performed based on the candidate characteristic pixel points and the theoretical characteristic pixel points to obtain a target instrument area image after the instrument area image is corrected.

[0139] In a specific implementation, in order to correct the instrument area image, this embodiment can respectively determine the characteristic pixel points in the actually collected instrument area image and the theoretical characteristic pixel points in the theoretical device instrument image for mapping, and respectively determine several characteristic pixel points or characteristic pixel areas ABC in the instrument image, and then combine them with the theoretical pixel points abc in the theoretical device instrument image, so as to proportionally map the collected instrument image to the theoretical device instrument image, thereby correcting the viewing angle of the collected instrument image.

[0140] Step S80: Identify equipment instrument information in the target instrument area image.

[0141] In a specific implementation, different methods may be used to identify the device instrument information in the instrument area image for different types of instruments. For example, for exponential instruments, the device instrument information may be extracted by text recognition, and for pointer-type instruments, the device instrument information may be identified by calculating the scale at which the pointer is located. The device instrument information is used to characterize at least one operating parameter of the device, such as temperature, humidity, charge load, etc. This embodiment does not impose any specific restrictions on this.

[0142] Step S90: Performing equipment self-check based on the equipment instrument information.

[0143] It should be noted that when performing equipment self-inspection, it is possible to determine whether the operating parameters of the equipment are within the theoretical operating range corresponding to its normal operation based on the instrument information corresponding to each device. If the self-inspection result indicates a fault, a fault warning information can be sent to the cloud server through the inspection robot to prompt maintenance personnel to perform repairs.

[0144] This embodiment collects equipment images during the mobile inspection of the inspection robot, and performs instrument detection based on the collected equipment images, thereby determining the instrument area image in the collected equipment image, reducing the impact of noise in the environment, and then correcting and identifying the instrument area image to extract the instrument information of each equipment image collected during the inspection process, thereby improving the information processing accuracy of the inspection robot during the inspection process, and finally performing equipment self-inspection based on the equipment instrument information, thereby improving the safe operation capability of the equipment in the power plant and avoiding the technical problem of low efficiency in inspecting equipment in the power plant in the prior art.

[0145] Based on the third embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 6, step S80, comprising:

[0146] Step S801: extracting features from the target instrument area image to obtain an appearance feature vector of the instrument area.

[0147] Step S802: Determine the instrument type of the target instrument area according to the appearance feature vector.

[0148] Step S803: When the meter type is an exponential meter, matching is performed based on the appearance feature vector and a theoretical appearance feature vector corresponding to the meter type.

[0149] Step S804: when the match is successful, the device instrument information in the target instrument area image is determined according to the instrument information corresponding to the successfully matched theoretical appearance feature vector.

[0150] It should be noted that different types of instruments have significant differences in appearance. Therefore, when identifying device instrument information, different methods can be used to accurately extract device information in the instrument area image. Specifically, for exponential instruments, they are generally digital instrument panels composed of a collection of LED light tubes, and the general form of representing device instrument information is digital form. Therefore, in this embodiment, it is only necessary to match the theoretical appearance feature vectors of pre-set numbers, letters and other characters with the collected appearance feature vectors on the exponential instrument. If the match is successful, the characters at each position on the exponential instrument can be determined, thereby extracting the device instrument information.

[0151] In addition, since the LED tubes on the exponential meter may be damaged, resulting in deviations in the device information stored therein, in order to avoid subsequent false alarms in self-tests, when the matching fails, error maintenance information is generated and uploaded to the cloud server to provide maintenance assistance information.

[0152] Furthermore, after determining the instrument type of the target instrument area according to the appearance feature vector, the method further includes:

[0153] When the instrument type is a pointer-type instrument, identifying a pointer and scale lines of the target instrument area based on the appearance feature vector;

[0154] Calculating the angle between the pointer and each scale line;

[0155] The equipment instrument information is determined according to the angle.

[0156] In the specific implementation, if the instrument type is a pointer type instrument, the device instrument information can only be determined by the position of the pointer. Specifically, the pointer and scale lines in the instrument image area are determined by the appearance feature vector, and the angle between the pointer and each scale line is calculated to locate the precise position of the pointer, and finally extract accurate instrument operation information.

[0157] This embodiment obtains an appearance feature vector of the instrument area by performing feature extraction on the target instrument area image; determines the instrument type of the target instrument area according to the appearance feature vector; when the instrument type is an exponential instrument, matches the appearance feature vector with a theoretical appearance feature vector corresponding to the instrument type; when the match is successful, determines the equipment instrument information in the target instrument area image according to the instrument information corresponding to the successfully matched theoretical appearance feature vector. This embodiment determines the instrument type through the appearance feature vector of the target instrument area image, thereby adopting different equipment information extraction schemes according to different instrument types, thereby improving the efficiency and robustness of the inspection control of the inspection robot.

[0158] This application also provides an inspection control device for an inspection robot, please refer to Figure 7 , the inspection control device of the inspection robot comprises:

[0159] The acquisition module 10 is used to acquire the inspection information of the inspection robot in multiple dimensions in the area where it is located.

[0160] The determination module 20 is used to determine the environment category of the area where the inspection robot is located.

[0161] The planning module 30 is used to perform path planning based on each inspection information through the target planning model corresponding to the environment category to obtain a target travel path.

[0162] The inspection module 40 is used to control the inspection robot to perform mobile inspection based on the target travel path.

[0163] This embodiment obtains inspection information of the inspection robot in multiple dimensions of the area where it is located; determines the environmental category of the area where the inspection robot is located; performs path planning based on each inspection information through a target planning model corresponding to the environmental category to obtain a target travel path; controls the inspection robot to perform mobile inspection based on the target travel path, thereby realizing path planning in different types of environments, thereby realizing inspection control of the inspection robot in different types of environments, avoiding the technical problem of low efficiency in inspecting equipment in power plants in the prior art.

[0164] In one embodiment, the planning module 30 is also used to determine the current position of the inspection robot according to the image information and the radar positioning information when the environment category is an outdoor environment; obtain the inspection task list of the inspection robot to determine the equipment to be inspected and the target position of the equipment to be inspected; query the theoretical map through a preset database according to the current position and the target position; generate an initial travel path according to the current position, the target position and the theoretical map; when there are obstacles in the initial travel path, re-plan the path through the outdoor path planning model to obtain the target travel path.

[0165] In one embodiment, the planning module 30 is also used to divide the theoretical map into cells to obtain a unit map, wherein the size of the divided cells is the same as the size of the inspection robot; mark the straight line between the current position and the target position in the unit map; extract the target cell through which the straight line passes in the unit map; and generate an initial travel path based on the target cell.

[0166] In one embodiment, the planning module 30 is also used to mark the obstacle cells occupied by the obstacle in the unit map; determine the adjacent cells of the cell where the current position is located; generate at least two derived travel paths based on the adjacent cells and the initial travel path, and the cells in the derived travel paths are all adjacent cells of each cell in the initial travel path along the straight line between the current position and the target position; calculate the Manhattan distance of the derived travel path and the Euclidean distance on the straight line between the current position and the target position; determine the target travel path in each derived travel path based on the deviation between the Manhattan distance and the Euclidean distance.

[0167] In one embodiment, the planning module 30 is also used to, when the environment category is an indoor environment, reduce the dimension of the laser point cloud information to obtain a ground point cloud depth image; calculate the smoothing value of the ground point cloud depth image; segment the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images; perform feature extraction on each ground point cloud depth sub-image to obtain a plurality of target point cloud sets; perform path planning according to the target point cloud set and a preset map of the indoor environment to obtain a target travel path.

[0168] In one embodiment, the planning module 30 is also used to obtain image information at continuous time intervals when the environment category is an indoor environment; perform obstacle judgment on the image information based on the multiple target point cloud sets, and update the target travel path in real time based on the judgment results; and control the inspection robot to perform mobile inspection according to the updated target travel path.

[0169] In one embodiment, the inspection control device of the inspection robot also includes an inspection module 50, and the inspection module 50 is used to collect equipment images during the mobile inspection process of the inspection robot; perform instrument detection on the equipment image through a target detection model, and segment the instrument area image in the equipment image according to the detection result; correct the instrument area image to obtain a target instrument area image; identify equipment instrument information in the target instrument area image; and perform equipment self-inspection based on the equipment instrument information.

[0170] The present application provides an inspection control device for an inspection robot, the inspection control device for the inspection robot comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the inspection control method for the inspection robot in the above-mentioned embodiment one.

[0171] Reference below Figure 8 , which shows a schematic diagram of the structure of the inspection control device of the inspection robot suitable for implementing the embodiment of the present application. The inspection control device of the inspection robot in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The inspection control device of the inspection robot shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0172] like Figure 8As shown, the inspection control device of the inspection robot may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the inspection control device of the inspection robot are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the inspection control device of the inspection robot to communicate with other devices wirelessly or by wire to exchange data. Although the inspection control device of the inspection robot with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0173] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0174] The inspection control device of the inspection robot provided by the present application adopts the inspection control method of the inspection robot in the above embodiment, which can solve the technical problem of inspection control of the inspection robot. Compared with the prior art, the beneficial effects of the inspection control device of the inspection robot provided by the present application are the same as the beneficial effects of the inspection control method of the inspection robot provided by the above embodiment, and the other technical features of the inspection control device of the inspection robot are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0175] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0176] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0177] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the inspection control method of the inspection robot in the above-mentioned embodiment.

[0178] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0179] The computer-readable storage medium may be included in the inspection control device of the inspection robot; or may exist independently without being assembled into the inspection control device of the inspection robot.

[0180] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the inspection control device of the inspection robot, the inspection control device of the inspection robot performs inspection control of the inspection robot.

[0181] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0184] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the inspection control method of the inspection robot, and can solve the technical problems of the inspection control of the inspection robot. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the inspection control method of the inspection robot provided by the above embodiment, and will not be repeated here.

[0185] The present application also provides a computer program product, including a computer program, which implements the steps of the inspection control method of the inspection robot as described above when the computer program is executed by a processor.

[0186] The computer program product provided by this application can solve the technical problem of patrol control of patrol robots. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the patrol control method of patrol robots provided by the above embodiments, which will not be repeated here.

[0187] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A patrol control method for a patrol robot, characterized in that: The inspection control method of the inspection robot comprises: Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located; Determining the environmental category of the area where the inspection robot is located; Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path; The inspection robot is controlled to perform mobile inspection based on the target travel path.

2. The inspection control method of the inspection robot according to claim 1, characterized in that: The inspection information at least includes: image information and radar positioning information; The path planning is performed based on each inspection information through the target planning model corresponding to the environment category to obtain the target travel path, including: When the environment category is an outdoor environment, determining the current position of the inspection robot according to the image information and the radar positioning information; Obtaining an inspection task list of the inspection robot to determine a device to be inspected and a target position of the device to be inspected; According to the current location and the target location, a theoretical map is searched through a preset database; Generate an initial travel path according to the current position, the target position and the theoretical map; When there are obstacles in the initial travel path, the path is replanned using an outdoor path planning model to obtain a target travel path.

3. The inspection control method of the inspection robot according to claim 2, characterized in that: The generating an initial travel path according to the current position, the target position and the theoretical map comprises: Divide the theoretical map into cells to obtain a unit map, wherein the size of the divided cells is the same as the size of the inspection robot; marking a straight line between the current position and the target position in the unit map; Extract the target cell through which the straight line passes in the cell map; An initial travel path is generated based on the target cell.

4. The inspection control method of the inspection robot according to claim 2, characterized in that: The re-planning of the path by using the outdoor path planning model to obtain the target travel path includes: marking the obstacle cell occupied by the obstacle in the cell map; Determine the adjacent cells of the cell where the current position is located; Generate at least two derived travel paths according to the adjacent cells and the initial travel path, wherein the cells in the derived travel paths are adjacent cells of each cell in the initial travel path in a direction along a straight line between the current position and the target position; calculating a Manhattan distance of the derived travel path and a straight-line Euclidean distance between the current position and the target position; The target travel path in each derived travel path is determined according to the deviation between the Manhattan distance and the Euclidean distance.

5. The inspection control method of the inspection robot according to claim 2, characterized in that: The inspection information also includes: laser point cloud information; The inspection control method of the inspection robot further includes: When the environment category is an indoor environment, the laser point cloud information is subjected to dimensionality reduction processing to obtain a ground point cloud depth image; Calculating a smoothing value of the ground point cloud depth image; Segmenting the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images; Feature extraction is performed on each depth sub-image of the ground point cloud to obtain multiple target point cloud sets; Path planning is performed according to the target point cloud set and a preset map of the indoor environment to obtain a target travel path.

6. The inspection control method of the inspection robot according to claim 1, characterized in that: The controlling the inspection robot to perform mobile inspection based on the target travel path includes: When the environment category is an indoor environment, acquiring image information at consecutive time intervals; Perform obstacle judgment on the image information according to the multiple target point cloud sets, and update the target travel path in real time according to the judgment result; The inspection robot is controlled to perform mobile inspection according to the updated target travel path.

7. The inspection control method of the inspection robot according to claim 1, characterized in that: After controlling the inspection robot to perform mobile inspection based on the target travel path, the method further includes: Collecting images of equipment during the mobile inspection process of the inspection robot; Performing instrument detection on the device image through a target detection model, and segmenting the instrument area image in the device image according to the detection result; Correcting the instrument area image to obtain a target instrument area image; Identifying equipment instrument information in the target instrument area image; Perform equipment self-check based on the equipment instrument information.

8. A patrol control device for a patrol robot, characterized in that: The inspection control device of the inspection robot comprises: An acquisition module is used to obtain the inspection information of the inspection robot in multiple dimensions in the area where it is located; A determination module, used to determine the environment category of the area where the inspection robot is located; A planning module, used to perform path planning based on each inspection information through a target planning model corresponding to the environment category to obtain a target travel path; The inspection module is used to control the inspection robot to perform mobile inspection based on the target travel path.

9. An inspection control device for an inspection robot, characterized in that: The inspection control device of the inspection robot comprises: a memory, a processor and a inspection control program of the inspection robot stored in the memory and executable on the processor, wherein the inspection control program of the inspection robot is configured to implement the inspection control method of the inspection robot as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an inspection control program of the inspection robot, and when the inspection control program of the inspection robot is executed by the processor, the inspection control method of the inspection robot according to any one of claims 1 to 7 is implemented.

Images