Scheduling inspection method and system based on intelligent inspection robot and storage medium

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a scheduling inspection method and system based on an intelligent inspection robot and a storage medium, and the method comprises the steps that an inspection server obtains and records equipment information, associates an inspection point with equipment, marks an inspection area according to a map of the inspection point in a target area, and creates an inspection plan according to business needs; the inspection plan is submitted for examination and approval, after the examination and approval are passed, an inspection path is planned according to the inspection plan before inspection is started, after path planning is completed, an inspection robot is allocated to each sub-region, the inspection path is sent to the corresponding inspection robot, and an inspection task is executed on the sub-region according to the planned inspection path. The inspection robot shoots video data and image data of corresponding inspection points in real time and sends the video data and the image data to the inspection server when inspecting each device, and the inspection server analyzes the image data to obtain an inspection result after receiving the image data. According to the invention, an inspection task can be efficiently completed and an accurate inspection result can be obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a scheduling inspection method, system and storage medium based on an intelligent inspection robot. Background Art

[0002] With the rapid development of my country's high-speed railways and urban rail transit, the requirements for the operational reliability of electric traction power supply systems are becoming increasingly stringent. With factors such as short construction period, high quality requirements, low-price product bids, and changes in the natural environment, better guaranteeing the operational quality of traction power supply projects has become a major demand and a practical problem that needs to be solved in recent years. In the operation and maintenance of rail transit traction power supply projects, the quality of maintenance and inspection work, work efficiency, and operating costs all require more intelligent technical means to ensure. In the process of electric traction power supply operation and maintenance, subway distribution rooms and other places usually adopt a 7×24h dedicated duty method. Staff need to inspect the computer room infrastructure on time and check, verify and analyze the basic operating data. This time-consuming and labor-intensive method leads to low work efficiency, and the standardization of inspections is relatively difficult to ensure. Similar prior art is a Chinese invention patent with publication number CN103824233B, which provides a GIS-based UAV power line inspection and dispatching platform, including a dispatching server, a human-computer interaction module, an RFID reader, an ID card reader, and a short message transceiver module, and is equipped with a GIS-based UAV power line inspection and dispatching method, which considers factors such as inspection site status information, equipment information, and personnel information, and dispatches the inspection equipment, personnel, time, and line through the dispatching terminal. Similar prior art also has a Chinese invention patent with publication number CN104539053B, which discloses a power dispatching automatic inspection robot and inspection method based on crawler technology, wherein the inspection robot includes a remote interface service module, a main control program module, a task module, a script parsing module, an inspection upload data processing module, an abnormal management module, and an operation status monitoring module. The inspection robot is based on crawler technology, simulates HTTP requests through HTTP proxy, and uses HttpClient and HtmlParser open source components for URL connection and interface parsing technology to crawl the links and key data of the business system that needs to be monitored, and realize automatic inspection of the information system. However, the above two methods do not solve the problem of how to perform inspection tasks more efficiently and obtain more accurate inspection results. Therefore, the present invention provides a scheduling inspection method, system and storage medium based on an intelligent inspection robot. Summary of the invention

[0003] The present invention creates an inspection plan, plans a path for the inspection robot, uses the inspection robot to efficiently perform inspection tasks instead of manual labor, obtains image data and video data of the equipment in real time, analyzes the image data to obtain the status of the equipment in real time, improves analysis accuracy, and solves the problems of low inspection efficiency and low inspection standardization.

[0004] In order to achieve the above-mentioned purpose of the invention, the present invention provides the following scheduling inspection method based on the intelligent inspection robot, which is implemented by executing the following steps:

[0005] The inspection server obtains and records device information, including device ID, device type, and original image of the device, associates inspection points with devices, marks inspection areas based on inspection points on the map of the target area, sets identification codes for devices, creates inspection plans based on business needs, and then submits the inspection plans for approval;

[0006] After approval, the inspection route is planned according to the inspection plan before the inspection begins, the map data of the target area is obtained, all inspection points are obtained, and the path map between each inspection point is obtained based on the inspection points and map data. The total number of inspection robots is obtained, and the target area is divided into multiple sub-areas. The number of sub-areas is the total number of inspection robots. An inspection point in the sub-area is obtained as the starting position, and the inspection path is planned for the sub-area using the path planning algorithm. After the path planning is completed, an inspection robot is assigned to each sub-area, and the inspection robot is dispatched to the corresponding starting position, and the corresponding inspection path is sent to the corresponding inspection robot. The inspection robot performs the inspection task on the sub-area according to the planned inspection path;

[0007] The inspection robot is connected to the inspection server via a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, takes the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data and obtains the inspection results. It also saves the inspection records of each device, which include the inspection point, image data, video data, inspection results and inspection time.

[0008] The management terminal sends query conditions to the inspection server. The inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives the query conditions, sends the corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

[0009] As a preferred technical solution of the present invention, analyzing the image data to obtain the inspection results includes the following steps:

[0010] A detection threshold is set, the area where the device is located is captured from multiple original images to generate a feature image, and the average pixel values ​​of the pixels at the same position in the multiple feature images are calculated, each average value is set at the pixel at the corresponding position to generate a new image, the generated new image is used as the reference image, and multiple image data of the corresponding device taken during the inspection are obtained. The image with the greatest similarity to the reference image is obtained from the multiple image data as the device image, the area where the device is located is captured from the device image to generate a target image to be analyzed, the difference in feature values ​​at each pixel between the reference image and the target image is obtained, and abnormalities are identified on the target image according to the detection threshold and the feature value difference.

[0011] As a preferred technical solution of the present invention, setting the detection threshold includes the following steps:

[0012] Acquire multiple feature images and reference images, calculate the eigenvalue change amount corresponding to each pixel point between each feature image and the reference image, and obtain multiple eigenvalue change amounts for each pixel point. The standard deviation of the corresponding pixel point calculated based on the multiple eigenvalue change amounts is called the year-on-year standard deviation. Also calculate the eigenvalue change amount between each pixel point in the reference image and the adjacent pixel point centered on the pixel point. The multiple eigenvalue change amounts are calculated, and the regional standard deviation of each pixel point is calculated based on the multiple eigenvalue change amounts. According to the year-on-year standard deviation and the regional standard deviation, different formulas are used to calculate the detection threshold in different situations.

[0013] As a preferred technical solution of the present invention, different formulas are used to calculate the detection threshold in different situations, including the following steps:

[0014] When both the year-on-year standard deviation and the regional standard deviation are greater than the first threshold, the detection threshold is calculated using the first formula, which is: R=0.5*A+0.5*B, where R is the detection threshold, A is the year-on-year standard deviation, and B is the regional standard deviation. When the year-on-year standard deviation is greater than the first threshold and the regional standard deviation is less than or equal to the first threshold, the detection threshold is calculated using the second formula, which is R=0.7*A+0.3*B. When the year-on-year standard deviation is less than or equal to the first threshold and the regional standard deviation is greater than the first threshold, the detection threshold is calculated using the third formula, which is R=0.3*A+0.7*B. When both the year-on-year standard deviation and the regional standard deviation are less than or equal to the first threshold, the detection threshold is set to zero.

[0015] As a preferred technical solution of the present invention, the target image is identified as abnormal according to the detection threshold and the feature value difference, including the following steps:

[0016] Compare the detection threshold and eigenvalue difference of each pixel in the target image. If the eigenvalue difference is greater than the detection threshold, set the first marking value for the corresponding pixel of the target image. If the eigenvalue difference is less than or equal to the detection threshold, set the second marking value for the corresponding pixel of the target image. After marking all the pixels of the target image, generate a marked image, analyze the marked image, obtain a first area composed of pixels continuously marked with the first marking value, obtain the total number of pixels contained in the first area, and if the total number of pixels is greater than the second threshold, mark the first area as an abnormal area.

[0017] As a preferred technical solution of the present invention, calculating the characteristic value change amount corresponding to each pixel point between each characteristic image and the reference image includes the following steps:

[0018] Each feature image and the reference image are converted into HSL space to obtain the corresponding H value. The H value of each pixel point of each feature image is subtracted from the H value of the corresponding pixel point of the reference image to obtain multiple differences. The multiple differences are the changes in the feature values ​​corresponding to each pixel point between each feature image and the reference image.

[0019] As a preferred technical solution of the present invention, calculating the average pixel values ​​of pixels at the same position in multiple feature images includes the following steps:

[0020] Get the RGB value of each pixel of each feature image. The RGB value includes R value, G value and B value. Add the R value, G value and B value of the pixel points at the same position of each feature image respectively and then divide by three to get the average R value, average G value and average B value of the corresponding pixel points. The combination of the average R value, average G value and average B value is the average value of the pixel points of the corresponding reference image.

[0021] As a preferred technical solution of the present invention, after completing the inspection task, the inspection robot automatically returns to the designated position and waits for instructions for the next round of inspection tasks.

[0022] The present invention also provides a dispatching inspection system based on an intelligent inspection robot, comprising the following modules:

[0023] Inspection server, used to obtain and record equipment information, including equipment ID, equipment type and original image of equipment, associate inspection points with equipment, mark inspection areas according to inspection points on the map of target areas, set identification codes for equipment, create inspection plans according to business needs, and then submit inspection plans for approval;

[0024] The path planning module is used to plan the inspection path according to the inspection plan before the inspection starts after approval, obtain the map data of the target area, obtain all inspection points, obtain the path map between the inspection points based on the inspection points and the map data, obtain the total number of inspection robots, divide the target area into multiple sub-areas, the number of sub-areas is the total number of inspection robots, obtain a inspection point in the sub-area as the starting position, use the path planning algorithm to plan the inspection path for the sub-area, and after the path planning is completed, assign a patrol robot to each sub-area, dispatch the patrol robot to the corresponding starting position, and send the corresponding patrol path to the corresponding patrol robot, and the patrol robot performs the inspection task on the sub-area according to the planned patrol path;

[0025] The inspection robot is used to connect to the inspection server through a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, takes the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data to obtain the inspection result, and also saves the inspection record of each device, which includes the inspection point, image data, video data, inspection result and inspection time;

[0026] The management terminal is used to send query conditions to the inspection server. The inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives query conditions, sends corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

[0027] The present invention also provides a storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned scheduling and inspection methods based on the intelligent inspection robot.

[0028] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0029] In the present invention, an inspection plan is first created, and then the area is divided according to the target area of ​​the inspection, an inspection robot is assigned to each area, and the optimal path is planned for the inspection robot so that the inspection robot can efficiently complete the inspection. Video data and image data are captured in real time during the inspection, and the original image and image data are compared and analyzed to obtain abnormal conditions of the inspection equipment. The video data is also integrated and managed for subsequent viewing by staff. The present invention can obtain abnormal conditions of each inspection point by analyzing image data, accurately obtain inspection results, efficiently perform inspection tasks, and also make the inspection more standardized. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of the steps of the scheduling inspection method based on the intelligent inspection robot of the present invention;

[0031] Figure 2 This is a structural diagram of the scheduling and inspection system based on the intelligent inspection robot of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0034] The present invention provides Figure 1 The scheduling inspection method based on the intelligent inspection robot is implemented by executing the following steps:

[0035] Step S1, the inspection server obtains and records the equipment information, including the equipment ID, equipment type and original image of the equipment, associates the inspection point with the equipment, marks the inspection area according to the map of the inspection point in the target area, sets an identification code for the equipment, creates an inspection plan according to business needs, and then submits the inspection plan for approval.

[0036] Specifically, in order to create a reasonable inspection plan, the inspection server first obtains the equipment information that needs to be inspected and records the equipment information. The original image of the equipment refers to the image information when the equipment was initially installed, which is used to compare with the equipment image taken during subsequent inspections of the equipment, determine the abnormality of the equipment, and associate the inspection point with the equipment. The inspection point refers to the location information of the equipment installation, such as the distribution room, relay cabinet room, and computer room, etc. The inspection area is marked on the map of the target area according to the inspection point. The target area refers to the overall area where all equipment that needs to be inspected is located, such as a high-speed railway station. An identification code is set for the equipment to facilitate identification of the corresponding equipment during inspection. Then, an inspection plan is created according to business needs, and the inspection plan is submitted for approval.

[0037] Step S2, after approval, plan the inspection path according to the inspection plan before the inspection begins, obtain map data of the target area, obtain all inspection points, obtain the path map between each inspection point based on the inspection points and map data, obtain the total number of inspection robots, divide the target area into multiple sub-areas, the number of sub-areas is the total number of inspection robots, obtain a inspection point in the sub-area as the starting position, use the path planning algorithm to plan the inspection path for the sub-area, after the path planning is completed, assign a patrol robot to each sub-area, dispatch the patrol robot to the corresponding starting position, and send the corresponding patrol path to the corresponding patrol robot, the patrol robot performs the inspection task on the sub-area according to the planned patrol path.

[0038] Specifically, in order to conduct efficient inspections of the target area, after the inspection plan is approved, the inspection path is planned according to the inspection plan before the inspection begins. First, the path map between the inspection points is obtained, and then the total number of inspection robots is obtained. The target area is divided into multiple sub-areas, and the number of sub-areas is the total number of inspection robots, so that the inspection robots can achieve maximum utilization. Then, an inspection point in the sub-area is obtained as the starting position, and a path planning algorithm is used to plan an inspection path for the sub-area to make the inspection path as short as possible. The path planning algorithm can use existing technologies such as the Dijkstra algorithm. Then, a patrol robot is assigned to each sub-area, and the patrol robot is dispatched to the starting position of the corresponding sub-area. The planned path is sent to the corresponding patrol robot, and the patrol robot performs the inspection task on the sub-area according to the planned inspection path, so as to achieve the purpose of efficient inspection.

[0039] Step S3, the inspection robot is connected to the inspection server via a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, captures the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data to obtain the inspection result, and also saves the inspection record of each device. The inspection record includes the inspection point, image data, video data, inspection result and inspection time.

[0040] Specifically, in order to obtain the status of each device, the inspection robot can be connected to the inspection server through a wireless network. The inspection robot is equipped with a shooting device that can be used to record videos and take photos. When inspecting each device, the video data and image data of the inspection point are shot and sent to the inspection server. After receiving the image data, the inspection server analyzes the image data to obtain the inspection results, and also saves the inspection records of each device. The video data can be viewed through the management terminal. For example, if an abnormality is found in a certain inspection point through the inspection results, the video data of the inspection point at a certain time can be obtained for manual re-inspection to confirm the inspection results. The inspection time can be used as a query condition to obtain the video data of the corresponding time point.

[0041] Step S4, the management terminal sends query conditions to the inspection server, the inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives the query conditions, sends the corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

[0042] Specifically, after the inspection server receives the video data, it integrates and manages the video data and supports multiple query methods to obtain the video data. For example, it can query the video data of a certain inspection point at a certain time point. It also establishes a communication connection through the network and the management terminal. When the query conditions are received from the management terminal, the corresponding video data is sent to the management terminal. The staff can view the on-site conditions of each inspection point through the management terminal.

[0043] Through the above steps, an inspection plan is first created. Then, sub-areas are divided according to the target area of ​​the inspection, and an inspection robot is assigned to each sub-area. The optimal path is planned for the inspection robot so that the inspection robot can complete the inspection efficiently. During the inspection, video data and image data are captured in real time. The original image and image data are compared and analyzed to obtain abnormal conditions of the inspection equipment. The video data is also integrated and managed for subsequent viewing by staff.

[0044] Furthermore, the image data is analyzed to obtain the inspection results, including the following steps:

[0045] A detection threshold is set, the area where the device is located is captured from multiple original images to generate a feature image, and the average pixel values ​​of the pixels at the same position in the multiple feature images are calculated, each average value is set at the pixel at the corresponding position to generate a new image, the generated new image is used as the reference image, and multiple image data of the corresponding device taken during the inspection are obtained. The image with the greatest similarity to the reference image is obtained from the multiple image data as the device image, the area where the device is located is captured from the device image to generate a target image to be analyzed, the difference in feature values ​​at each pixel between the reference image and the target image is obtained, and abnormalities are identified on the target image according to the detection threshold and the feature value difference.

[0046] Specifically, in order to analyze the image data to determine whether each device has an abnormality, a detection threshold is set. The detection threshold is a criterion for determining whether the device is abnormal. The specific setting method of the detection threshold will be explained in detail later. The area where the device is located is captured from multiple original images to generate multiple feature images, and the average pixel values ​​of the pixel points at the same position in the multiple feature images are calculated. Each average value is used as the pixel value of the pixel point at the corresponding position, and the image with the new pixel value is used as the reference image. Multiple image data of the corresponding device taken by the inspection robot during the inspection are obtained, and the image with the greatest similarity to the reference image is obtained from the multiple image data as the device image. The area where the device is located is captured from the device image to obtain the target image to be analyzed, and the reference image and the target image are obtained in each The feature value difference at the pixel point is used to identify the abnormality of the target image according to the detection threshold and the feature value difference. The feature image is extracted from the original image. The original image is the image of the device taken when the device was initially installed, which represents a normal device image. Multiple original images can be taken at different times. Since the intensity of external light may be different at different times, different original images may have differences due to environmental influences. Therefore, the above method is used to first obtain the feature image according to the original image, and then obtain the reference image according to the feature image. The reference image is compared with the real-time device image to obtain a more accurate difference. The difference can be used to quickly and accurately determine whether the target image is abnormal. The specific judgment method will be explained in detail later.

[0047] Furthermore, setting the detection threshold includes the following steps:

[0048] Acquire multiple feature images and reference images, calculate the eigenvalue change amount corresponding to each pixel point between each feature image and the reference image, and obtain multiple eigenvalue change amounts for each pixel point. The standard deviation of the corresponding pixel point calculated based on the multiple eigenvalue change amounts is called the year-on-year standard deviation. Also calculate the eigenvalue change amount between each pixel point in the reference image and the adjacent pixels centered on the pixel point. The multiple eigenvalue change amounts are calculated, and the regional standard deviation of each pixel point is calculated based on the multiple eigenvalue change amounts. According to the year-on-year standard deviation and the regional standard deviation, different formulas are used to calculate the detection threshold in different situations.

[0049] Specifically, in order to set a reasonable detection threshold and make image recognition analysis more accurate, multiple feature images and reference images are obtained, and the change in the feature value of each pixel point between each feature image and the reference image is calculated to obtain multiple feature value changes corresponding to each pixel point. Then, the standard deviation of the multiple feature value changes is calculated based on the multiple feature value changes as the year-on-year standard deviation of the corresponding pixel point. The year-on-year standard deviation refers to the degree of change in the feature value of the pixel point between the images. The regional standard deviation is also obtained. The regional standard deviation refers to the change in the feature value between each pixel point in the reference image and the adjacent pixel points centered on each pixel point, which represents the degree of change in the feature value within the image. For example The eigenvalue change between the central pixel and the eight adjacent pixels around it is calculated, and then the regional standard deviation of the central pixel is obtained based on the calculated multiple eigenvalue changes. The method for calculating the standard deviation is a prior art and will not be explained in detail. The original image is an image of a device without abnormalities taken. Due to the different shooting time points, the external shooting environment may be different, which may lead to different eigenvalue changes within and between images. Therefore, setting the same detection threshold for all pixels may lead to inaccurate judgment. Therefore, based on the year-on-year standard deviation and the regional standard deviation, multiple different detection thresholds are calculated in different situations to make the image analysis more reasonable and accurate.

[0050] Furthermore, different formulas are used to calculate the detection threshold in different situations, including the following steps:

[0051] When both the year-on-year standard deviation and the regional standard deviation are greater than the first threshold, the detection threshold is calculated using the first formula, which is: R=0.5*A+0.5*B, where R is the detection threshold, A is the year-on-year standard deviation, and B is the regional standard deviation. When the year-on-year standard deviation is greater than the first threshold and the regional standard deviation is less than or equal to the first threshold, the detection threshold is calculated using the second formula, which is R=0.7*A+0.3*B. When the year-on-year standard deviation is less than or equal to the first threshold and the regional standard deviation is greater than the first threshold, the detection threshold is calculated using the third formula, which is R=0.3*A+0.7*B. When both the year-on-year standard deviation and the regional standard deviation are less than or equal to the first threshold, the detection threshold is set to zero.

[0052] Specifically, when the year-on-year standard deviation and the regional standard deviation are both greater than the first threshold, it means that the variation range of the characteristic values ​​of the pixels between different feature images is relatively large, and the variation range of the characteristic values ​​of the pixels within the image is also relatively large, indicating that the possibility that this pixel may change due to external environmental factors is relatively large, so the corresponding detection threshold is set larger to make it more accurate in judging abnormalities. Therefore, the detection threshold is calculated using the first formula. When the year-on-year standard deviation is greater than the first threshold but the regional standard deviation is less than or equal to the first threshold, it means that the variation range of the characteristic values ​​of the pixels between different feature images is relatively large, but the variation range of the characteristic values ​​of the pixels within the image is relatively small. Therefore, the second formula is used to set the detection threshold. When the year-on-year standard deviation is less than or equal to the first threshold, the regional standard deviation is less than or equal to the first threshold. When the regional standard deviation is greater than the first threshold, it means that the variation range of the characteristic values ​​of the pixels between different characteristic images is relatively small, but the variation range of the characteristic values ​​of the pixels within the image is relatively large, so the detection threshold is calculated using the third formula. When the year-on-year standard deviation and the regional standard deviation are both less than or equal to the first threshold, it means that the pixel is relatively stable and not easily affected by the external environment, and the variation of the characteristic values ​​within the image and the surrounding adjacent images is not large, indicating that the characteristic value of the pixel within the image is not easy to change and is relatively stable, so the detection threshold is set to zero at this time. Different detection thresholds are set for different situations through the above method, so that the setting of the detection threshold is more accurate, and the subsequent analysis of whether the image is abnormal through the detection threshold is also more accurate.

[0053] Furthermore, the target image is subjected to abnormality recognition according to the detection threshold and the feature value difference, including the following steps:

[0054] Compare the detection threshold and eigenvalue difference of each pixel in the target image. If the eigenvalue difference is greater than the detection threshold, set a first marking value for the corresponding pixel of the target image. If the eigenvalue difference is less than or equal to the detection threshold, set a second marking value for the corresponding pixel of the target image. After marking all pixels of the target image, generate a marked image, analyze the marked image, obtain a first area composed of pixels continuously marked with the first marking value, obtain the total number of pixels contained in the first area, and if the total number of pixels is greater than the second threshold, mark the first area as an abnormal area.

[0055] Specifically, in order to analyze whether the target image is abnormal, the target image is judged according to the detection threshold and the eigenvalue difference. Since the eigenvalue difference and the detection threshold are calculated in units of pixels, it is first determined whether the eigenvalue difference of each pixel is greater than the detection threshold. If it is, the pixel is set to a first marking value, otherwise the second marking value is set. After marking all the pixels of the target image, a marked image is generated, and then the marked image is analyzed to obtain a first area composed of pixels that are continuously marked with the first marking value. There may be multiple first areas. If only one or a small number of pixels in an image are marked with the first marking value, it may be due to errors caused by environmental factors or other factors. In this case, the corresponding first area is not marked as an abnormal area. Therefore, it is necessary to judge the total number of pixels in the first area. If the total number of pixels in the first area is greater than the second threshold, it means that the first area is relatively large. At this time, the first area is marked as an abnormal area.

[0056] Furthermore, calculating the characteristic value variation of each pixel between each characteristic image and the reference image includes the following steps:

[0057] Each feature image and the reference image are converted into HSL mode to obtain the corresponding H value, and the H value of each pixel point of each feature image is subtracted from the H value of the corresponding pixel point of the reference image to obtain multiple differences. The multiple differences are the changes in the feature values ​​corresponding to each pixel point between the feature image and the reference image.

[0058] Specifically, the HSL mode is a method of representing points in the RGB color model in a cylindrical coordinate system, with three values, namely H value, S value and L value. The present invention uses the H value. The specific method for calculating the eigenvalue change is, for example, there are eight feature images, taking the calculation of a difference as an example, obtaining the H value of a pixel point on a feature image, obtaining the H value of the pixel point at the corresponding position on the reference image, subtracting the two H values ​​to obtain the corresponding difference, which is the eigenvalue change of the current pixel point, and using the same method to calculate the eigenvalue change of other pixel points on the feature image. The eigenvalue change of each pixel point is calculated by the above method, which serves as a basis for subsequent calculations based on the eigenvalue change.

[0059] Furthermore, calculating the average pixel values ​​of pixels at the same position in multiple feature images includes the following steps:

[0060] Get the RGB value of each pixel in the feature image. The RGB value includes R value, G value and B value. Add the R value, G value and B value of the pixel at the same position in the feature image respectively and then divide by three to get the average R value, average G value and average B value of the corresponding pixel. The combination of the average R value, average G value and average B value is the average value of the pixel of the corresponding reference image.

[0061] Specifically, since there are multiple feature images, the average pixel value of each pixel in each feature image is calculated by the above method to obtain a reference image. When subsequently judging whether the captured device images are different, the reference image is used as a comparison object to obtain a more accurate judgment result.

[0062] Furthermore, after completing the inspection task, the inspection robot automatically returns to the designated position and waits for instructions for the next round of inspection tasks.

[0063] According to another aspect of the embodiment of the present invention, Figure 2 As shown, a scheduling and inspection system based on an intelligent inspection robot is also provided, including an inspection server, a path planning module, an inspection robot and a management terminal, which are used to implement the scheduling and inspection method based on the intelligent inspection robot as described above. The specific functions of each module are as follows:

[0064] Inspection server, used to obtain and record equipment information, including equipment ID, equipment type and original image of equipment, associate inspection points with equipment, mark inspection areas according to inspection points on the map of target areas, set identification codes for equipment, create inspection plans according to business needs, and then submit inspection plans for approval;

[0065] The path planning module is used to plan the inspection path according to the inspection plan before the inspection starts after approval, obtain the map data of the target area, obtain all inspection points, obtain the path map between the inspection points based on the inspection points and the map data, obtain the total number of inspection robots, divide the target area into multiple sub-areas, the number of sub-areas is the total number of inspection robots, obtain a inspection point in the sub-area as the starting position, use the path planning algorithm to plan the inspection path for the sub-area, and after the path planning is completed, assign a patrol robot to each sub-area, dispatch the patrol robot to the corresponding starting position, and send the corresponding patrol path to the corresponding patrol robot, and the patrol robot performs the inspection task on the sub-area according to the planned patrol path;

[0066] The inspection robot is used to connect to the inspection server through a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, takes the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data to obtain the inspection result, and also saves the inspection record of each device, which includes the inspection point, image data, video data, inspection result and inspection time;

[0067] The management terminal is used to send query conditions to the inspection server. The inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives query conditions, sends corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

[0068] According to another aspect of an embodiment of the present invention, a storage medium is further provided, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned scheduling and inspection methods based on the intelligent inspection robot.

[0069] In summary, the scheduling inspection method, system and storage medium based on the intelligent inspection robot of the present invention include the inspection server acquiring and recording equipment information, associating inspection points with equipment, marking inspection areas according to the inspection points on the map of the target area, creating inspection plans according to business needs, and then submitting the inspection plans for approval. After approval, the inspection paths are planned before the inspection begins according to the inspection plan. After the path planning is completed, a inspection robot is assigned to each sub-area, and the inspection paths are sent to the corresponding inspection robots. Inspection tasks are performed on the sub-areas according to the planned inspection paths. When the inspection robots inspect each device, they shoot the video data and image data of the corresponding inspection points in real time and send them to the inspection server. After the inspection server receives the image data, it analyzes the image data to obtain the inspection results. The present invention can efficiently complete the inspection tasks and obtain accurate inspection results.

[0070] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0071] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0072] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0074] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dispatching inspection method based on an intelligent inspection robot, the method is applied to a dispatching inspection system based on an intelligent inspection robot, the system includes an inspection server, a path planning module, an inspection robot and a management terminal, characterized in that: The steps include: The inspection server obtains and records device information, including device ID, device type, and original image of the device, associates inspection points with devices, marks inspection areas based on inspection points on the map of the target area, sets identification codes for devices, creates inspection plans based on business needs, and then submits the inspection plans for approval; After approval, the inspection route is planned according to the inspection plan before the inspection begins, the map data of the target area is obtained, all inspection points are obtained, and the path map between each inspection point is obtained based on the inspection points and map data. The total number of inspection robots is obtained, and the target area is divided into multiple sub-areas. The number of sub-areas is the total number of inspection robots. An inspection point in the sub-area is obtained as the starting position, and the inspection path is planned for the sub-area using the path planning algorithm. After the path planning is completed, an inspection robot is assigned to each sub-area, and the inspection robot is dispatched to the corresponding starting position, and the corresponding inspection path is sent to the corresponding inspection robot. The inspection robot performs the inspection task on the sub-area according to the planned inspection path; The inspection robot is connected to the inspection server via a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, takes the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data and obtains the inspection results. It also saves the inspection records of each device, which include the inspection point, image data, video data, inspection results and inspection time. The management terminal sends query conditions to the inspection server. The inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives the query conditions, sends the corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

2. The method according to claim 1, characterized in that Analyzing the image data to obtain the inspection results includes the following steps: Set a detection threshold, capture the area where the device is located from multiple original images to generate a feature image, calculate the average pixel values ​​of pixels at the same position in the multiple feature images, set each average value at the pixel at the corresponding position to generate a new image, use the generated new image as the reference image, obtain multiple image data of the corresponding device taken during the inspection, obtain the image with the greatest similarity to the reference image from the multiple image data as the device image, capture the area where the device is located from the device image to generate a target image to be analyzed, obtain the difference in feature values ​​of each pixel between the reference image and the target image, and identify abnormalities in the target image according to the detection threshold and the feature value difference.

3. The method according to claim 2, characterized in that Setting the detection threshold includes the following steps: Acquire multiple feature images and reference images, calculate the eigenvalue change amount corresponding to each pixel point between each feature image and the reference image, and obtain multiple eigenvalue change amounts for each pixel point. The standard deviation of the corresponding pixel point calculated based on the multiple eigenvalue change amounts is called the year-on-year standard deviation. Also calculate the eigenvalue change amount between each pixel point in the reference image and the adjacent pixel point centered on the pixel point. The multiple eigenvalue change amounts are calculated, and the regional standard deviation of each pixel point is calculated based on the multiple eigenvalue change amounts. According to the year-on-year standard deviation and the regional standard deviation, different formulas are used to calculate the detection threshold in different situations.

4. The method according to claim 3, characterized in that Different formulas are used to calculate the detection threshold in different situations, including the following steps: When both the year-on-year standard deviation and the regional standard deviation are greater than the first threshold, the detection threshold is calculated using the first formula, which is: R=0.5*A+0.5*B, where R is the detection threshold, A is the year-on-year standard deviation, and B is the regional standard deviation. When the year-on-year standard deviation is greater than the first threshold and the regional standard deviation is less than or equal to the first threshold, the detection threshold is calculated using the second formula, which is R=0.7*A+0.3*B. When the year-on-year standard deviation is less than or equal to the first threshold and the regional standard deviation is greater than the first threshold, the detection threshold is calculated using the third formula, which is R=0.3*A+0.7*B. When both the year-on-year standard deviation and the regional standard deviation are less than or equal to the first threshold, the detection threshold is set to zero.

5. The method according to claim 4, characterized in that The abnormality recognition of the target image is performed based on the detection threshold and the feature value difference, including the following steps: Compare the detection threshold and eigenvalue difference of each pixel in the target image. If the eigenvalue difference is greater than the detection threshold, set the first marking value for the corresponding pixel of the target image. If the eigenvalue difference is less than or equal to the detection threshold, set the second marking value for the corresponding pixel of the target image. After marking all the pixels of the target image, generate a marked image, analyze the marked image, obtain a first area composed of pixels continuously marked with the first marking value, obtain the total number of pixels contained in the first area, and if the total number of pixels is greater than the second threshold, mark the first area as an abnormal area.

6. The method according to claim 3, characterized in that Calculating the characteristic value change of each pixel point between each characteristic image and the reference image includes the following steps: Each feature image and the reference image are converted into HSL space to obtain the corresponding H value. The H value of each pixel point of each feature image is subtracted from the H value of the corresponding pixel point of the reference image to obtain multiple differences. The multiple differences are the changes in the feature values ​​corresponding to each pixel point between each feature image and the reference image.

7. The method according to claim 2, characterized in that: Calculating the average pixel value of pixels at the same position in multiple feature images includes the following steps: Get the RGB value of each pixel of each feature image. The RGB value includes R value, G value and B value. Add the R value, G value and B value of the pixel points at the same position of each feature image respectively and then divide by three to get the average R value, average G value and average B value of the corresponding pixel points. The combination of the average R value, average G value and average B value is the average value of the pixel points of the corresponding reference image.

8. The method according to claim 1, characterized in that After completing the inspection task, the inspection robot automatically returns to the designated location and waits for instructions for the next round of inspection tasks.

9. A dispatching inspection system based on an intelligent inspection robot, used to implement the method according to any one of claims 1 to 8, characterized in that: Includes the following modules: Inspection server, used to obtain and record equipment information, including equipment ID, equipment type and original image of equipment, associate inspection points with equipment, mark inspection areas according to inspection points on the map of target areas, set identification codes for equipment, create inspection plans according to business needs, and then submit inspection plans for approval; The path planning module is used to plan the inspection path according to the inspection plan before the inspection starts after approval, obtain the map data of the target area, obtain all inspection points, obtain the path map between the inspection points based on the inspection points and the map data, obtain the total number of inspection robots, divide the target area into multiple sub-areas, the number of sub-areas is the total number of inspection robots, obtain a inspection point in the sub-area as the starting position, use the path planning algorithm to plan the inspection path for the sub-area, and after the path planning is completed, assign a patrol robot to each sub-area, dispatch the patrol robot to the corresponding starting position, and send the corresponding patrol path to the corresponding patrol robot, and the patrol robot performs the inspection task on the sub-area according to the planned patrol path; The inspection robot is used to connect to the inspection server through a wireless network. The inspection robot has the function of recording and taking pictures. When inspecting each device, it identifies the identification code of the device, takes the video data and image data of the corresponding inspection point in real time, and sends the video data and image data to the inspection server. After receiving the image data, the inspection server analyzes the image data to obtain the inspection result, and also saves the inspection record of each device, which includes the inspection point, image data, video data, inspection result and inspection time; The management terminal is used to send query conditions to the inspection server. The inspection server integrates and saves the captured video data, establishes a communication connection with the management terminal through the network, receives query conditions, sends corresponding video data to the management terminal according to the query conditions, and checks the on-site conditions of each inspection area through the management terminal.

10. A storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the scheduling and inspection method based on the intelligent inspection robot according to any one of claims 1 to 8.

Images