Information processing method, device and equipment of inspection robot and storage medium

Abstract

The invention relates to the technical field of power plant inspection, in particular to an information processing method and device of an inspection robot, equipment and a storage medium, and the method comprises the steps: collecting an equipment image in a moving inspection process of the inspection robot, carrying out the instrument detection based on the collected equipment image, and determining an instrument region image in the collected equipment image; the method comprises the following steps: reducing noise influence in the environment, correcting and identifying an instrument area image, extracting instrument information of each equipment image acquired in the inspection process, improving the information processing precision of an inspection robot in the inspection process, and finally performing equipment self-inspection according to the equipment instrument information, thereby improving the safe operation capability of equipment in a power plant. The technical problem that in the prior art, the inspection efficiency of equipment in a power plant is low is solved.

Description

Technical Field

[0001] The present invention relates to the field of power plant inspection technology, and in particular to an information processing method, device, equipment and storage medium for an inspection robot. Background Art

[0002] During the operation of a power plant, it is necessary to frequently inspect the equipment in the power plant to monitor the operating status of each device and avoid equipment failure. The traditional means of monitoring power plant equipment is mainly through manual inspection to record the operating data of each device. However, this method has a large workload and cannot provide timely operating status feedback for each device, affecting the inspection efficiency.

[0003] With the development of intelligent robots, there are also examples of patrol inspections by robots in the existing technology. However, during the patrol inspection process, the robot can only patrol according to the set route, cannot adapt to the relatively loaded indoor environment and the relatively wide outdoor environment, and the effect is single, generally only for the purpose of collecting data. It is impossible to self-check the status of the equipment.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an information processing method, device, equipment and storage medium for an inspection robot, aiming to solve the technical problem of low efficiency in inspecting equipment in power plants in the prior art.

[0006] To achieve the above object, the present invention provides an information processing method for an inspection robot, the method comprising the following steps:

[0007] Collecting images of equipment during the mobile inspection process of the inspection robot;

[0008] Performing instrument detection on the device image through a target detection model, and segmenting the instrument area image in the device image according to the detection result;

[0009] Correcting the instrument area image to obtain a target instrument area image;

[0010] Identifying equipment instrument information in the target instrument area image;

[0011] Perform equipment self-check based on the equipment instrument information.

[0012] Optionally, the correcting the instrument area image to obtain a target instrument area image includes:

[0013] Extracting the characteristic pixel points to be selected in the instrument area image;

[0014] Matching theoretical feature pixel points corresponding to the feature pixel points to be selected in the theoretical equipment instrument image;

[0015] Angle mapping is performed based on the candidate characteristic pixel points and the theoretical characteristic pixel points to obtain a target instrument area image after the instrument area image is corrected.

[0016] Optionally, the identifying the equipment instrument information in the target instrument area image includes:

[0017] Extracting features from the target instrument area image to obtain an appearance feature vector of the instrument area;

[0018] Determining the instrument type of the target instrument area according to the appearance feature vector;

[0019] When the meter type is an exponential meter, matching is performed based on the appearance feature vector and a theoretical appearance feature vector corresponding to the meter type;

[0020] When the match is successful, the device instrument information in the target instrument area image is determined according to the instrument information corresponding to the theoretical appearance feature vector that is successfully matched;

[0021] When the matching fails, error repair information is generated and uploaded to the cloud server.

[0022] Optionally, after determining the instrument type of the target instrument area according to the appearance feature vector, the method further includes:

[0023] When the instrument type is a pointer-type instrument, identifying a pointer and scale lines of the target instrument area based on the appearance feature vector;

[0024] Calculating the angle between the pointer and each scale line;

[0025] The equipment instrument information is determined according to the angle.

[0026] Optionally, before collecting the device image during the mobile inspection process of the inspection robot, the method further includes:

[0027] Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located;

[0028] Determining the environmental category of the area where the inspection robot is located;

[0029] Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path;

[0030] The inspection robot is controlled to perform mobile inspection based on the target travel path.

[0031] Optionally, performing path planning according to the inspection information through a target planning model corresponding to the environment category to obtain a target travel path includes:

[0032] When the environment category is an outdoor environment, determining the current position of the inspection robot according to the image information and the radar positioning information;

[0033] Obtaining an inspection task list of the inspection robot to determine a device to be inspected and a target position of the device to be inspected;

[0034] According to the current location and the target location, a theoretical map is searched through a preset database;

[0035] Generate an initial travel path according to the current position, the target position and the theoretical map;

[0036] When there are obstacles in the initial travel path, the path is replanned using an outdoor path planning model to obtain a target travel path.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes an information processing device of an inspection robot, the information processing device of the inspection robot comprising:

[0038] An acquisition module, used to acquire device images during the mobile inspection process of the inspection robot;

[0039] A segmentation module, used to perform instrument detection on the device image through a target detection model, and segment the instrument area image in the device image according to the detection result;

[0040] A correction module, used for correcting the instrument area image to obtain a target instrument area image;

[0041] An identification module, used for identifying equipment instrument information in the target instrument area image;

[0042] The self-check module is used to perform equipment self-check based on the equipment instrument information.

[0043] In addition, to achieve the above-mentioned objectives, the present invention also proposes an information processing device for a patrol robot, which includes: a memory, a processor, and an information processing program for the patrol robot stored in the memory and executable on the processor, and the information processing program for the patrol robot is configured to implement the steps of the information processing method for the patrol robot as described above.

[0044] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which is stored an information processing program of the inspection robot, and when the information processing program of the inspection robot is executed by the processor, the steps of the information processing method of the inspection robot as described above are implemented.

[0045] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the information processing method of the inspection robot as described above.

[0046] One or more technical solutions proposed in the present invention have at least the following technical effects: the present invention collects equipment images during the mobile inspection of the inspection robot, and performs instrument detection based on the collected equipment images, thereby determining the instrument area image in the collected equipment image, reducing the impact of noise in the environment, and then corrects and identifies the instrument area image to extract the instrument information of each equipment image collected during the inspection process, thereby improving the information processing accuracy of the inspection robot during the inspection process, and finally performing equipment self-inspection based on the equipment instrument information, thereby improving the safe operation capability of the equipment in the power plant and avoiding the technical problem of low efficiency in inspecting equipment in the power plant in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 It is a flowchart of a first embodiment of the information processing method of the inspection robot of the present invention;

[0050] Figure 2 A schematic diagram of process instrument image correction according to an embodiment of an information processing method for an inspection robot of the present invention;

[0051] Figure 3 It is a flowchart of a second embodiment of the information processing method of the inspection robot of the present invention;

[0052] Figure 4 It is a flowchart of a third embodiment of the information processing method of the inspection robot of the present invention;

[0053] Figure 5This is a schematic diagram of initial path planning of an embodiment of an information processing method for an inspection robot of the present invention;

[0054] Figure 6 A schematic diagram of derived path planning of an information processing method for an inspection robot according to an embodiment of the present invention;

[0055] Figure 7 It is a structural block diagram of a first embodiment of the information processing device of the inspection robot of the present invention;

[0056] Figure 8 It is a structural schematic diagram of an information processing device of an inspection robot in a hardware operating environment involved in an embodiment of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0059] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0060] Based on this, an embodiment of the present invention provides an information processing method for an inspection robot, referring to Figure 1 , Figure 1 The figure is a flow chart of a first embodiment of an information processing method for an inspection robot according to the present invention.

[0061] In this embodiment, the information processing method of the inspection robot includes:

[0062] Step S10: collecting images of equipment during the mobile inspection process of the inspection robot.

[0063] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, information processing of an inspection robot, etc. The following takes the information processing of an inspection robot as an example to illustrate this embodiment and the following embodiments.

[0064] It should be noted that during the inspection process in the power plant, the inspection robot generally moves according to the set target path, thereby meeting the inspection task requirements in the inspection task list, ensuring the normal inspection frequency of various equipment in the power plant, and avoiding equipment damage.

[0065] Equipment images include but are not limited to appearance images and instrument images of various types of electrical equipment. Appearance images are used to detect whether obvious faults have occurred, and instrument images are used to be provided to cloud servers to detect the operating status of various types of electrical equipment, so as to timely inspect and maintain the electrical equipment.

[0066] Step S20: Perform instrument detection on the device image using a target detection model, and segment the instrument area image in the device image according to the detection result.

[0067] It should be understood that the target detection model can be a target detection model based on YOLOv5, machine learning or deep learning, which is mainly used to denoise the collected device images, thereby retaining the instrument area image in the device image and reducing the data extraction interference caused by environmental noise and the workload of information processing.

[0068] After performing instrument detection on the device image through the target detection model, the boundary of the instrument area can be determined, so that the boundary of the instrument area can be framed and marked in the device image, and finally the instrument area image in the device image can be segmented by image cropping.

[0069] Since the inspection robot is affected by the environment in which it is located when collecting equipment images, such as incident light, dust, and electromagnetic interference, in this embodiment, before detecting the equipment image, the equipment image can be preprocessed, that is, the equipment image can be smoothed, filtered, enhanced, and denoised to improve the image recognizability and reduce noise interference.

[0070] Step S30: Correcting the instrument area image to obtain a target instrument area image.

[0071] It is understandable that due to the influence of factors such as shooting angle or dust obstruction, the collected instrument area image may not be directly facing the camera or may be partially obstructed. By correcting the instrument area image, the collected instrument area image can be restored to a straight and complete target instrument area image, so as to improve the efficiency of subsequent identification of device instrument information.

[0072] Furthermore, the correcting the instrument area image to obtain the target instrument area image includes:

[0073] Extracting the characteristic pixel points to be selected in the instrument area image;

[0074] Matching theoretical feature pixel points corresponding to the feature pixel points to be selected in the theoretical equipment instrument image;

[0075] Angle mapping is performed based on the candidate characteristic pixel points and the theoretical characteristic pixel points to obtain a target instrument area image after the instrument area image is corrected.

[0076] In the specific implementation, refer to Figure 2 , Figure 2 FIG. 1 is a schematic diagram of image correction in this embodiment. In order to correct the instrument area image, this embodiment can respectively determine the characteristic pixel points in the instrument area image actually collected and the theoretical characteristic pixel points in the instrument image of the theoretical device for mapping. Figure 2 For example, Figure 2 In the method, several characteristic pixel points or characteristic pixel areas ABC in the instrument image are determined respectively, and then combined with the theoretical pixel points abc in the theoretical device instrument image, the collected instrument image is proportionally mapped to the theoretical device instrument image, thereby correcting the viewing angle of the collected instrument image.

[0077] Step S40: Identify equipment instrument information in the target instrument area image.

[0078] In a specific implementation, different methods may be used to identify the device instrument information in the instrument area image for different types of instruments. For example, for exponential instruments, the device instrument information may be extracted by text recognition, and for pointer-type instruments, the device instrument information may be identified by calculating the scale at which the pointer is located. The device instrument information is used to characterize at least one operating parameter of the device, such as temperature, humidity, charge load, etc. This embodiment does not impose any specific restrictions on this.

[0079] Step S50: Performing equipment self-check based on the equipment instrument information.

[0080] It should be noted that when performing equipment self-inspection, it is possible to determine whether the operating parameters of the equipment are within the theoretical operating range corresponding to its normal operation based on the instrument information corresponding to each device. If the self-inspection result indicates a fault, a fault warning information can be sent to the cloud server through the inspection robot to prompt maintenance personnel to perform repairs.

[0081] This embodiment collects equipment images during the mobile inspection of the inspection robot, and performs instrument detection based on the collected equipment images, thereby determining the instrument area image in the collected equipment image, reducing the impact of noise in the environment, and then correcting and identifying the instrument area image to extract the instrument information of each equipment image collected during the inspection process, thereby improving the information processing accuracy of the inspection robot during the inspection process, and finally performing equipment self-inspection based on the equipment instrument information, thereby improving the safe operation capability of the equipment in the power plant and avoiding the technical problem of low efficiency in inspecting equipment in the power plant in the prior art.

[0082] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S40, comprising:

[0083] Step S401: extracting features from the target instrument area image to obtain an appearance feature vector of the instrument area.

[0084] Step S402: Determine the instrument type of the target instrument area according to the appearance feature vector.

[0085] Step S403: When the meter type is an exponential meter, matching is performed based on the appearance feature vector and a theoretical appearance feature vector corresponding to the meter type.

[0086] Step S404: when the matching is successful, the device instrument information in the target instrument area image is determined according to the instrument information corresponding to the successfully matched theoretical appearance feature vector.

[0087] It should be noted that different types of instruments have significant differences in appearance. Therefore, when identifying device instrument information, different methods can be used to accurately extract device information in the instrument area image. Specifically, for exponential instruments, they are generally digital instrument panels composed of a collection of LED light tubes, and the general form of representing device instrument information is digital form. Therefore, in this embodiment, it is only necessary to match the theoretical appearance feature vectors of pre-set numbers, letters and other characters with the collected appearance feature vectors on the exponential instrument. If the match is successful, the characters at each position on the exponential instrument can be determined, thereby extracting the device instrument information.

[0088] In addition, since the LED tubes on the exponential meter may be damaged, resulting in deviations in the device information stored therein, in order to avoid subsequent false alarms in self-tests, when the matching fails, error maintenance information is generated and uploaded to the cloud server to provide maintenance assistance information.

[0089] Furthermore, after determining the instrument type of the target instrument area according to the appearance feature vector, the method further includes:

[0090] When the instrument type is a pointer-type instrument, identifying a pointer and scale lines of the target instrument area based on the appearance feature vector;

[0091] Calculating the angle between the pointer and each scale line;

[0092] The equipment instrument information is determined according to the angle.

[0093] In the specific implementation, if the instrument type is a pointer type instrument, the device instrument information can only be determined by the position of the pointer. Specifically, the pointer and scale lines in the instrument image area are determined by the appearance feature vector, and the angle between the pointer and each scale line is calculated to locate the precise position of the pointer, and finally extract accurate instrument operation information.

[0094] This embodiment obtains an appearance feature vector of the instrument area by performing feature extraction on the target instrument area image; determines the instrument type of the target instrument area according to the appearance feature vector; when the instrument type is an exponential instrument, matches the appearance feature vector with a theoretical appearance feature vector corresponding to the instrument type; when the match is successful, determines the equipment instrument information in the target instrument area image according to the instrument information corresponding to the successfully matched theoretical appearance feature vector. This embodiment determines the instrument type through the appearance feature vector of the target instrument area image, thereby adopting different equipment information extraction schemes according to different instrument types, thereby improving the efficiency and robustness of information processing of the inspection robot.

[0095] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 4 , before step S10, further comprising:

[0096] Step S01: Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located.

[0097] Step S02: Determine the environment category of the area where the inspection robot is located.

[0098] Step S03: Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path.

[0099] Step S04: controlling the inspection robot to perform mobile inspection based on the target travel path.

[0100] It should be noted that since the installation environment of power plant equipment includes indoor and outdoor environments, in order to improve the mobile inspection capabilities of the inspection robot in different environments and obtain accurate equipment images and self-inspection efficiency, this embodiment will perform path planning in corresponding modes according to the environmental category of the area where the inspection robot is located.

[0101] Among them, the inspection information of multiple dimensions includes at least image information, radar positioning information and laser point cloud information. In outdoor environments, since the outdoor environment is relatively wide and there are fewer obstacles affecting the movement of the inspection robot, the use of image information and radar positioning can improve the accuracy of the inspection robot in the path planning process; in indoor environments, the installation of various equipment occupies a large amount of space, and it is even more necessary to update the inspection robot's travel path in real time to improve inspection efficiency.

[0102] In this embodiment, when determining the environmental category of the area where the inspection robot is located, the position of the inspection robot can be located based on image information and radar positioning, and then the environment of the area where the inspection robot is located can be judged whether it is an indoor environment or an outdoor environment based on the floor plan of the power plant.

[0103] Further, the inspection information includes at least: image information and radar positioning information;

[0104] The path planning is performed based on each inspection information through the target planning model corresponding to the environment category to obtain the target travel path, including:

[0105] When the environment category is an outdoor environment, determining the current position of the inspection robot according to the image information and the radar positioning information;

[0106] Obtaining an inspection task list of the inspection robot to determine a device to be inspected and a target position of the device to be inspected;

[0107] According to the current location and the target location, a theoretical map is searched through a preset database;

[0108] Generate an initial travel path according to the current position, the target position and the theoretical map;

[0109] When there are obstacles in the initial travel path, the path is replanned using an outdoor path planning model to obtain a target travel path.

[0110] In the specific implementation, the inspection task list of the inspection robot includes the location and information of each device that needs to be inspected, so that the inspection robot can plan the path in the outdoor environment and realize the inspection of each device in the inspection task list. Devices without collected device information or devices that do not need to be inspected by the inspection robot will be marked as devices to be inspected. The target position of the devices to be inspected can be queried through the devices to be inspected in the inspection task list.

[0111] The preset database stores theoretical maps of various areas within the power plant. The theoretical maps include the layout locations of various equipment and obstacle locations, etc., to facilitate the subsequent generation of an initial travel path and provide an initial travel direction.

[0112] Since the outdoor environment may be affected by factors such as equipment damage and falling obstacles, which may cause obstacles to appear on the initial travel path, when there are obstacles in the initial travel path, the path is re-planned through the outdoor path planning model to obtain the target travel path and improve the inspection efficiency of the inspection robot.

[0113] Further, generating an initial travel path according to the current position, the target position and the theoretical map includes:

[0114] Divide the theoretical map into cells to obtain a unit map, wherein the size of the divided cells is the same as the size of the inspection robot;

[0115] marking a straight line between the current position and the target position in the unit map;

[0116] Extract the target cell through which the straight line passes in the cell map;

[0117] An initial travel path is generated based on the target cell.

[0118] In the specific implementation, refer to Figure 5 , Figure 5 For the unit map divided in this embodiment, the current position of the inspection robot is taken as the starting point, the target position of the equipment to be inspected is taken as the destination point, and the cells involved in the straight line between the starting point and the destination point are taken as the initial travel path. During this process, the inspection robot is allowed to move in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right.

[0119] Furthermore, re-planning the path through the outdoor path planning model to obtain the target travel path includes:

[0120] marking the obstacle cell occupied by the obstacle in the cell map;

[0121] Determine the adjacent cells of the cell where the current position is located;

[0122] Generate at least two derived travel paths according to the adjacent cells and the initial travel path, wherein the cells in the derived travel paths are adjacent cells of each cell in the initial travel path in a direction along a straight line between the current position and the target position;

[0123] calculating a Manhattan distance of the derived travel path and a straight-line Euclidean distance between the current position and the target position;

[0124] The target travel path in each derived travel path is determined according to the deviation between the Manhattan distance and the Euclidean distance.

[0125] In this embodiment, reference Figure 6 , Figure 6 This is a unit map when there are obstacles on the initial travel path. The cells occupied by the obstacles in the unit map are marked. Since the division of cells in this embodiment is based on the size of the inspection robot, if a cell is completely or partially occupied by an obstacle, the inspection robot will be unable to pass.

[0126] Based on this, this embodiment expands at least two derived travel paths adjacent to the initial travel path based on the direction of the initial travel path, thereby reducing the cost of the inspection robot's movement and the workload of re-path planning.

[0127] Among them, Manhattan distance refers to the movement cost of the inspection robot on two adjacent cells when it moves from the initial travel distance to the derived travel path. Euclidean distance is used to characterize the distance cost of the straight line from the current position to the target position. The deviation between Manhattan distance and the Euclidean distance can be used to characterize the movement cost of the inspection robot switching from the initial travel path to the derived travel path. The derived travel path corresponding to the minimum value of the deviation between Manhattan distance and the Euclidean distance is used as the final target travel path. This method not only reduces the workload of re-path planning, but also reduces the path movement cost of the inspection robot.

[0128] Furthermore, the inspection information also includes: laser point cloud information;

[0129] The inspection control method of the inspection robot further includes:

[0130] When the environment category is an indoor environment, the laser point cloud information is subjected to dimensionality reduction processing to obtain a ground point cloud depth image;

[0131] Calculating a smoothing value of the ground point cloud depth image;

[0132] Segmenting the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images;

[0133] Feature extraction is performed on each depth sub-image of the ground point cloud to obtain multiple target point cloud sets;

[0134] Path planning is performed according to the target point cloud set and a preset map of the indoor environment to obtain a target travel path.

[0135] It should be noted that the indoor environment is different from the outdoor environment. There are a large number of electrical equipment installed in the indoor environment of the power plant, and the wiring between the equipment is complicated. In addition, the image information and radar information will be interfered by the equipment and electromagnetic fields, and the travel path cannot be accurately formulated. As a result, simple path planning algorithms are difficult to achieve path planning in indoor environments, affecting the inspection efficiency of the inspection robot.

[0136] It can be understood that laser point cloud information is a data set of spatial points obtained by scanning with a three-dimensional laser radar device. Each point cloud contains three-dimensional coordinates and laser reflection intensity, and the distance information of the target object is measured by the speed of light and the time information from laser emission to return.

[0137] Since laser point cloud information is three-dimensional information, when participating in path planning, point cloud information that is not on the ground will interfere with the path planning of the inspection robot. For example, obstacles located at high altitudes will not affect normal passage during the actual inspection process, but in the laser point cloud information, the obstacle will be displayed as overlapping with the inspection robot, increasing the difficulty of data processing in the path planning process.

[0138] Therefore, in this embodiment, the three-dimensional laser point cloud information is first subjected to dimensionality reduction processing, and only the ground point cloud depth image is retained. Specifically, the dimensionality reduction processing refers to segmenting the ground point cloud information and non-ground point cloud information according to the pitch angle information of each point cloud information, and retaining the ground point cloud information, thereby removing the coordinate information in the spatial dimension and obtaining the ground point cloud depth image.

[0139] The calculation formula of the smoothing value is:

[0140]

[0141] Among them, ri and rj are the Euclidean distances between any two points and the sensor of the inspection robot, and S is a set of point clouds with uniform height in the ground point cloud depth image.

[0142] By calculating the smoothness value between two point clouds, it is determined whether the two points belong to the same subject. If they do, it means that the area cannot be passed. Finally, the extracted point cloud set and the preset map of the indoor environment are combined for path planning to improve the robot's inspection efficiency in complex indoor environments.

[0143] Further, controlling the inspection robot to perform mobile inspection based on the target travel path includes:

[0144] When the environment category is an indoor environment, acquiring image information at consecutive time intervals;

[0145] Perform obstacle judgment on the image information according to the multiple target point cloud sets, and update the target travel path in real time according to the judgment result;

[0146] The inspection robot is controlled to perform mobile inspection according to the updated target travel path.

[0147] It should be noted that, taking into account the complexity and interference of the indoor environment, in this embodiment, when the inspection robot is in an indoor environment, during the inspection process of the robot, the robot can obtain image frames at continuous time intervals and determine whether there are sudden obstacles through adjacent image frames. If so, the robot returns to the step of obtaining inspection information of multiple dimensions of the inspection robot in the area where it is located and re-plans the path.

[0148] This embodiment performs dimensionality reduction processing on the laser point cloud information to obtain a ground point cloud depth image when the environment category is an indoor environment; calculates a smoothing value of the ground point cloud depth image; segments the ground point cloud depth image based on the smoothing value to obtain a plurality of ground point cloud depth sub-images; performs feature extraction on each ground point cloud depth sub-image to obtain a plurality of target point cloud sets; performs path planning according to the target point cloud sets and a preset map of the indoor environment to obtain a target travel path, and improves the inspection efficiency of the inspection robot in the indoor environment by performing path planning in advance and updating the path in real time during the inspection process.

[0149] This application also provides an information processing device for an inspection robot, please refer to Figure 7 , the information processing device of the inspection robot comprises:

[0150] The acquisition module 10 is used to acquire device images during the mobile inspection process of the inspection robot.

[0151] The segmentation module 20 is used to perform instrument detection on the device image through a target detection model, and segment the instrument area image in the device image according to the detection result.

[0152] The correction module 30 is used to correct the instrument area image to obtain a target instrument area image.

[0153] The recognition module 40 is used to recognize the equipment instrument information in the target instrument area image.

[0154] The self-check module 50 is used to perform a self-check on the device based on the device instrument information.

[0155] This embodiment collects equipment images during the mobile inspection of the inspection robot, and performs instrument detection based on the collected equipment images, thereby determining the instrument area image in the collected equipment image, reducing the impact of noise in the environment, and then correcting and identifying the instrument area image to extract the instrument information of each equipment image collected during the inspection process, thereby improving the information processing accuracy of the inspection robot during the inspection process, and finally performing equipment self-inspection based on the equipment instrument information, thereby improving the safe operation capability of the equipment in the power plant and avoiding the technical problem of low efficiency in inspecting equipment in the power plant in the prior art.

[0156] In one embodiment, the correction module 30 is also used to extract candidate feature pixel points in the instrument area image; match theoretical feature pixel points corresponding to the candidate feature pixel points in the theoretical device instrument image; perform angle mapping based on the candidate feature pixel points and the theoretical feature pixel points to obtain a target instrument area image after the instrument area image is corrected.

[0157] In one embodiment, the recognition module 40 is further used to perform feature extraction on the target instrument area image to obtain an appearance feature vector of the instrument area; determine the instrument type of the target instrument area based on the appearance feature vector; when the instrument type is an exponential instrument, match the appearance feature vector with a theoretical appearance feature vector corresponding to the instrument type; when the match is successful, determine the equipment instrument information in the target instrument area image based on the instrument information corresponding to the successfully matched theoretical appearance feature vector.

[0158] In one embodiment, the recognition module 40 is further used to identify the pointer and scale lines of the target instrument area based on the appearance feature vector when the instrument type is a pointer type instrument; calculate the angle between the pointer and each scale line; and determine the equipment instrument information based on the angle.

[0159] In one embodiment, the information processing device of the inspection robot also includes: a planning module 50, which is used to obtain inspection information of the inspection robot in multiple dimensions in the area where it is located; determine the environmental category of the area where the inspection robot is located; perform path planning based on each inspection information through a target planning model corresponding to the environmental category to obtain a target travel path; and control the inspection robot to perform mobile inspection based on the target travel path.

[0160] In one embodiment, the planning module 50 is also used to determine the current position of the inspection robot according to the image information and the radar positioning information when the environment category is an outdoor environment; obtain the inspection task list of the inspection robot to determine the equipment to be inspected and the target position of the equipment to be inspected; query the theoretical map through a preset database according to the current position and the target position; generate an initial travel path according to the current position, the target position and the theoretical map; when there are obstacles in the initial travel path, re-plan the path through the outdoor path planning model to obtain the target travel path.

[0161] The present application provides an information processing device for an inspection robot, the information processing device for the inspection robot comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the information processing method for the inspection robot in the above-mentioned embodiment one.

[0162] Reference below Figure 8 , which shows a schematic diagram of the structure of the information processing device of the inspection robot suitable for implementing the embodiment of the present application. The information processing device of the inspection robot in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The information processing device of the inspection robot shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0163] like Figure 8As shown, the information processing device of the inspection robot may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the information processing device of the inspection robot are also stored in RAM1004. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the information processing device of the inspection robot to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows the information processing device of the inspection robot with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0164] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0165] The information processing device of the inspection robot provided by the present application adopts the information processing method of the inspection robot in the above embodiment, which can solve the technical problem of information processing of the inspection robot. Compared with the prior art, the beneficial effects of the information processing device of the inspection robot provided by the present application are the same as the beneficial effects of the information processing method of the inspection robot provided by the above embodiment, and the other technical features of the information processing device of the inspection robot are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0166] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0168] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the information processing method of the inspection robot in the above-mentioned embodiment.

[0169] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0170] The computer-readable storage medium may be included in the information processing device of the inspection robot; or may exist independently without being assembled into the information processing device of the inspection robot.

[0171] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the information processing device of the inspection robot, the information processing device of the inspection robot performs information processing of the inspection robot.

[0172] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0175] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned information processing method of the inspection robot, and can solve the technical problems of information processing of the inspection robot. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the information processing method of the inspection robot provided by the above-mentioned embodiment, and will not be repeated here.

[0176] The present application also provides a computer program product, including a computer program, which implements the steps of the information processing method of the inspection robot as described above when executed by a processor.

[0177] The computer program product provided by this application can solve the technical problem of information processing of the inspection robot. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the information processing method of the inspection robot provided by the above embodiment, which will not be repeated here.

[0178] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An information processing method for an inspection robot, characterized in that: The information processing method of the inspection robot comprises: Collecting images of equipment during the mobile inspection process of the inspection robot; Performing instrument detection on the device image through a target detection model, and segmenting the instrument area image in the device image according to the detection result; Correcting the instrument area image to obtain a target instrument area image; Identifying equipment instrument information in the target instrument area image; Perform equipment self-check based on the equipment instrument information.

2. The information processing method of the inspection robot according to claim 1, characterized in that: The correcting the instrument area image to obtain a target instrument area image includes: Extracting the characteristic pixel points to be selected in the instrument area image; Matching theoretical feature pixel points corresponding to the feature pixel points to be selected in the theoretical equipment instrument image; Angle mapping is performed based on the candidate characteristic pixel points and the theoretical characteristic pixel points to obtain a target instrument area image after the instrument area image is corrected.

3. The information processing method of the inspection robot according to claim 1, characterized in that: The identifying the equipment instrument information in the target instrument area image includes: Extracting features from the target instrument area image to obtain an appearance feature vector of the instrument area; Determining the instrument type of the target instrument area according to the appearance feature vector; When the meter type is an exponential meter, matching is performed based on the appearance feature vector and a theoretical appearance feature vector corresponding to the meter type; When the match is successful, the device instrument information in the target instrument area image is determined according to the instrument information corresponding to the theoretical appearance feature vector that is successfully matched; When the matching fails, error repair information is generated and uploaded to the cloud server.

4. The information processing method of the inspection robot according to claim 3, characterized in that: After determining the instrument type of the target instrument area according to the appearance feature vector, the method further includes: When the instrument type is a pointer-type instrument, identifying a pointer and scale lines of the target instrument area based on the appearance feature vector; Calculating the angle between the pointer and each scale line; The equipment instrument information is determined according to the angle.

5. The information processing method of the inspection robot according to any one of claims 1 to 4, characterized in that: Before collecting the device image during the mobile inspection process of the inspection robot, the method further includes: Obtain inspection information of the inspection robot in multiple dimensions in the area where it is located; Determining the environmental category of the area where the inspection robot is located; Based on each inspection information, path planning is performed through a target planning model corresponding to the environment category to obtain a target travel path; The inspection robot is controlled to perform mobile inspection based on the target travel path.

6. The method according to claim 5, characterized in that The step of performing path planning according to the inspection information through a target planning model corresponding to the environment category to obtain a target travel path includes: When the environment category is an outdoor environment, determining the current position of the inspection robot according to the image information and the radar positioning information; Obtaining an inspection task list of the inspection robot to determine a device to be inspected and a target position of the device to be inspected; According to the current location and the target location, a theoretical map is searched through a preset database; Generate an initial travel path according to the current position, the target position and the theoretical map; When there are obstacles in the initial travel path, the path is replanned using an outdoor path planning model to obtain a target travel path.

7. An information processing device for an inspection robot, characterized in that: The information processing device of the inspection robot comprises: An acquisition module, used to acquire device images during the mobile inspection process of the inspection robot; A segmentation module, used to perform instrument detection on the device image through a target detection model, and segment the instrument area image in the device image according to the detection result; A correction module, used for correcting the instrument area image to obtain a target instrument area image; An identification module, used for identifying equipment instrument information in the target instrument area image; The self-check module is used to perform equipment self-check based on the equipment instrument information.

8. An information processing device for an inspection robot, characterized in that: The information processing device of the inspection robot includes: a memory, a processor, and an information processing program of the inspection robot stored in the memory and executable on the processor, wherein the information processing program of the inspection robot is configured to implement the steps of the information processing method of the inspection robot as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores an information processing program of the inspection robot, and when the information processing program of the inspection robot is executed by the processor, the steps of the information processing method of the inspection robot according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the information processing method of the inspection robot according to any one of claims 1 to 6 are implemented.

Images