A Robotic Inspection System Based on Comparison Detection

Abstract

This invention discloses a robot inspection system based on comparison detection, comprising: a general video monitoring module, a comparison detection module, an anomaly recognition module, and an alarm notification module; wherein, the general video monitoring module includes: a server module for task control and device configuration, a client module for human-computer interaction and video display, and a video perception module for video streaming and preprocessing; the comparison detection module includes: setting an image of the device to be inspected in a non-abnormal state during the initial setup phase, comparing the acquired image with the image set in the initial phase during the real-time operation phase, identifying anomalies, and locating the anomalies; the anomaly recognition module is used to identify anomalies; and the alarm notification module is used to record anomalies and send alarm notifications.

Description

Technical Field

[0001] This invention relates to a robotic inspection system, and more particularly to a robotic inspection system based on comparative detection. Background Technology

[0002] Currently, the monitoring and maintenance of many factories, warehouses, and infrastructure in China still relies primarily on manual inspections. Examples include substations in the power industry and warehouses in the logistics industry. Under high-voltage, ultra-high-voltage, and severe weather conditions, manual operations not only pose significant personal risks but also compromise the safe operation of facilities. Intelligent inspection systems utilize inspection robots to automatically inspect equipment along predetermined navigation routes, employing their built-in visible light and thermal imaging cameras to achieve unattended or minimally-attended facility monitoring and maintenance. Intelligent inspection systems offer the following advantages: reducing equipment losses due to human negligence and missed inspections, thus improving facility operational quality; reducing personnel input and lowering labor costs; and mitigating the chronic health hazards of adverse environmental conditions.

[0003] Current inspection robots automate the location and identification of equipment anomalies, generally based on general or feature-specific detection deep models on specific datasets. These models require large training datasets and manual annotation, resulting in huge costs. If new equipment or new requirements emerge, data annotation and retraining are necessary. Furthermore, for deep models to achieve good results, models with a large number of parameters are often required, posing a significant challenge to the computational costs of the robot.

[0004] However, an inspection system based on a change comparison detection method can overcome the aforementioned limitations while achieving low cost and high accuracy. However, this method requires specific procedures and methods, necessitating adjustments and modifications to the existing inspection system. This adjustment process involves a lot of repetitive work, resulting in low efficiency. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a robot inspection system based on comparative detection, which addresses the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses a robot inspection system based on comparative detection, comprising: a general video monitoring module, a comparative detection module, an anomaly recognition module, and an alarm notification module;

[0007] The general video surveillance module includes: a server module, a client module, and a video sensing module. The server module is used for task control and device configuration, the client module is used for human-computer interaction and video display, and the video sensing module is used for video streaming and preprocessing. The server module and the client module communicate using a remote procedure call protocol, and the client module and the video sensing module communicate using a real-time streaming protocol.

[0008] The comparison detection module includes an initial setting stage and a real-time operation stage. In the initial setting stage, an image of the device to be detected in a non-abnormal state is set. In the real-time operation stage, the collected image is compared with the image set in the initial stage to determine anomalies and locate the anomalies.

[0009] The anomaly identification module compares the located anomalies with the detection results of the comparison detection module to determine the anomaly category or whether it is a false detection. If it is a false detection, it will not be displayed in the general video monitoring module.

[0010] The alarm notification module is used to record the anomaly and issue an alarm notification when an anomaly is detected.

[0011] Beneficial effects:

[0012] Using the aforementioned system, the performance of the YOLO detection model can be achieved without a large amount of labeled data. The improvements in the system and process of this invention not only avoid the shortcomings of traditional methods in terms of low generalization, but also overcome the problem that deep models require a large amount of labeled data and computational resources. It can avoid the time-consuming and laborious manual inspection and safety issues under limited resources, and provide real-time monitoring and timely alarm feedback. Attached Figure Description

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0014] Figure 1 This is a schematic diagram of the process of the present invention.

[0015] Figure 2 This is a schematic diagram of the front-end interface in the general video surveillance module of the system of the present invention. Detailed Implementation

[0016] Example:

[0017] This invention discloses a robot inspection system based on contrastive detection, applicable to scenarios requiring a low-cost, convenient, and efficient inspection system. Existing inspection systems struggle to function properly or achieve expected results when there is insufficient training data and limited computing resources. This system employs a two-stage process: setting a baseline map in the initial stage and then performing contrastive detection during runtime. This significantly reduces the need for labeled data, thereby reducing the time and financial costs associated with manual labeling. Because the method employed has low computational resource requirements, this system can be easily deployed to edge devices, particularly robotic platforms, greatly improving its applicability and portability. While maintaining a certain level of detection accuracy, it facilitates system deployment, reduces application costs, and enhances inspection efficiency and safety.

[0018] This invention discloses a robot inspection system based on contrast detection. The use of the system includes two stages, such as... Figure 1 As shown: the initial stage and the running stage.

[0019] The initial stage includes robot path planning, robot location setting, time-segmented and seasonal baseline map creation, and baseline map storage. Robot path planning involves setting the robot's inspection route at locations such as substations. Robot location setting involves assigning robot docking positions and attitude adjustment markers to equipment requiring attention. Time-segmented and seasonal baseline map creation involves taking baseline images during periods of significant difference in lighting conditions and according to the season. Baseline map storage involves saving baseline images for each location for retrieval during the operational phase.

[0020] The operational phase includes robot inspection, robot docking, robot posture adjustment, comparative detection, anomaly identification, and anomaly handling. Robot inspection involves the robot following a planned path. Robot docking refers to the robot stopping at marked locations. Robot posture adjustment involves the robot adjusting the camera's orientation according to crosshair markings. Comparative detection involves comparing a baseline image with the currently captured image to detect changes in the surrounding area. The anomaly identification module determines whether detected changes are genuine anomalies or false positives. Anomaly handling involves issuing alarms for anomalies and notifying relevant personnel for processing.

[0021] This invention discloses a robot inspection system based on comparison detection. The system includes: a general video monitoring module, a comparison detection module, a recognition module, and an alarm notification module.

[0022] The general video surveillance module includes a server module, a client module, and a video sensing module. The server module and the client module communicate using a remote procedure call protocol, and the client module and the video sensing module communicate using a real-time streaming protocol.

[0023] The comparison and detection module includes an initial setup phase and a real-time operation phase. The initial setup phase displays a normal device status image, and the real-time operation phase compares this image with the initial setup to locate any anomalies.

[0024] The identification module compares the located anomalies with the results of the comparison module, and gives the anomaly category or marks it as a false detection, which is not displayed in the general video surveillance module.

[0025] The alarm notification module is used to record the anomaly and notify relevant personnel for further processing when an anomaly occurs.

[0026] Image validity detection, also known as validity detection, is to confirm whether there are screen distortions or frame drops caused by the camera device, as well as occlusions or motion blurs that occur during use.

[0027] Image calibration transformation, also known as image calibration transformation, refers to the process of adjusting images to the same paradigm because different camera devices use different optical components, resulting in different distortions.

[0028] The image comparison detection module locates the difference region by comparing two images. Due to slight differences in the camera's pose, the two images will have different perspectives. The comparison algorithm needs to exclude the differences caused by the perspective and only retain the parts of the target that have changed.

[0029] Furthermore, this invention also discloses a robot inspection method based on contrast detection, the steps of which include...

[0030] Step 101, initial setup, including: robot path planning, robot location setting, setting benchmark maps by time period and season, and importing benchmark maps into the database.

[0031] Specifically, for robot path planning, the robot's route can be manually planned within the point cloud scene generated by robot SLAM, and the points where it needs to automatically stop can be set. When the robot stops at a point, the optical lens captures posture cue images, such as visual markers like crosshairs, and the camera posture is adjusted to capture a baseline image. Baseline images are acquired through the camera, and for each point, images need to be collected at various times of day and in different seasons with significantly varying lighting conditions. This improves the accuracy of the comparison and detection phase. The baseline images need to be stored in a database for the system to access during operation.

[0032] Step 102: Train the model, which includes an anomaly detection model;

[0033] Specifically, in this step, we need to determine whether an image is abnormal, and if so, what kind of abnormality it is. However, the training data only contains labeled abnormal data, and general methods cannot train a model that can distinguish between abnormal and non-abnormal data using this data. Since anomalies are destructive sample data—not existing, but artificially introduced to create data containing both abnormal and non-abnormal elements—this step converts the image into a feature vector. The method then determines whether the image belongs to an abnormal category based on whether its distance from the feature vector of the abnormal category meets a threshold. In this step, training the feature vector is the main part of training the recognition model.

[0034] Step 201: The system front-end connects to the edge device (here, the camera on the robot platform) via a real-time streaming protocol to transmit video streams in real time. Once the robot reaches the designated location and adjusts the camera's orientation, the comparison detection function is activated, and the front-end transmits the acquired real-time images and location information to the comparison detection module.

[0035] Step 202: The real-time image is received for comparison and detection, and the corresponding baseline image is retrieved from the database based on the location information for comparison and detection.

[0036] Specifically, this comparison detection module, due to its low computational complexity, can be deployed on both edge robot platforms and cloud servers. The comparison detection method in this step first calculates the key points in the real-time and baseline images, along with the feature descriptors for each key point. For each feature descriptor in the real-time image, the closest candidate feature descriptor in the baseline image is calculated. For each matched pair of feature descriptors, prior filtering methods are first used to filter out obviously mismatched pairs, and then a vector field consistency test is used to more accurately filter out incorrectly paired pairs. Then, for the remaining key point pairs, homography transformation matrices are applied to solve for them, grouping points with consistent transformation matrices into a single group. Grid sampling is performed on the real-time image, using the points with the calculated transformation matrices as guidelines for the transformation matrices to be applied to the grid points; here, the nearest neighbor voting method is used. The transformation matrices are applied to the grid points in the real-time image. For points still falling within the baseline image, feature descriptors are calculated. If the feature descriptor differs significantly from the corresponding key point's feature descriptor in the real-time image, it is considered a changed point. After calculating all the change points, for adjacent change points, a rectangular bounding box is calculated to accommodate these change points, which is considered a change region. The real-time graph and the corresponding change region information are returned.

[0037] Step 301: The front end receives the real-time image and the corresponding change area information, crops the change area from the real-time image, and sends the image to the anomaly recognition module.

[0038] Step 302: The anomaly detection module calculates the feature vector of the image and compares it with the feature vector of the registered anomalies. It outputs whether the image is an anomaly, and if it is, it outputs the type of anomaly.

[0039] Specifically, for the determination of anomalies, an appropriate threshold can be set according to the required recall and precision. If it is rejected by all anomaly categories, it is considered normal; otherwise, the most similar anomaly vector is selected as the anomaly category.

[0040] Step 401: The system front-end receives the return information from the anomaly detection module. If an anomaly is detected, it is recorded in the log, a handling plan is queried, and the relevant personnel are notified for further processing. If the anomaly is detected, the subsequent inspection targets continue.

[0041] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention content of the robot inspection system based on comparative detection provided by this invention, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0042] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0043] This invention provides a concept and method for a robot inspection system based on comparative detection. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A robot inspection system based on contrast detection, characterized in that, include: The system includes a general video surveillance module, a comparison and detection module, an anomaly identification module, and an alarm notification module. The general video surveillance module includes: a server module, a client module, and a video sensing module. The server module is used for task control and device configuration, the client module is used for human-computer interaction and video display, and the video sensing module is used for video streaming and preprocessing. The server module and the client module communicate using a remote procedure call protocol, and the client module and the video sensing module communicate using a real-time streaming protocol. The comparison detection module includes an initial setting stage and a real-time operation stage. In the initial setting stage, an image of the device to be detected in a non-abnormal state is set. In the real-time operation stage, the collected image is compared with the image set in the initial stage to determine anomalies and locate the anomalies. The anomaly identification module compares the located anomalies with the detection results of the comparison detection module to determine the anomaly category or whether it is a false detection. If it is a false detection, it will not be displayed in the general video monitoring module. The alarm notification module is used to record the anomaly and issue an alarm notification when an anomaly is detected. The system employs a robot inspection method based on contrast detection, comprising the following steps: Step 1, initial setup, namely the robot path planning, robot point setting, time-based and seasonal setting of the baseline map and its storage; Step 2: Train the anomaly detection model in the anomaly detection module; Step 3: The robot collects real-time video according to the initial settings and transmits the real-time images and location information in the real-time video to the comparison and detection module; Step 4: The comparison and detection module performs comparison and detection to obtain the changed area image, and then transmits the changed area image to the anomaly recognition module. Step 5: The anomaly detection module uses the trained anomaly detection model to judge the image of the changed area and sends the judgment result, i.e., the return information, to the robot. Step 6: The robot receives the return information from the anomaly detection module and performs subsequent processing; The training of the anomaly detection model described in step 2 specifically includes: The anomaly recognition model is trained using only labeled abnormal data, that is, images containing anomalies are converted into feature vectors, and whether they are anomalies is determined based on whether the distance between the feature vectors and the anomaly category feature vectors meets a threshold. The comparison detection module described in step 4 performs comparison detection, and the specific methods include: Calculate key points in the real-time image and the baseline image, as well as feature descriptor operators for the key points. For the feature descriptor operators of the real-time image, calculate and match the closest feature descriptor operator in the baseline image to obtain feature descriptor operator pairs. For all the matched feature descriptor pairs, a priori filtering method is used to filter out mismatched feature descriptor pairs. Then, the vector field consistency test method is used to filter out incorrectly paired feature descriptor pairs. Finally, for the remaining feature descriptor pairs, the homography transformation matrix is ​​solved, and the feature descriptor pairs with consistent transformation matrices, i.e., keypoint pairs, are grouped into a reference group. Grid sampling is performed on real-time images. A reference group is used as a guide for the transformation matrix of the grid points. The nearest neighbor voting method is used to apply the transformation matrix to the grid points of the real-time images. For points that still fall in the reference image, feature descriptors are calculated. If the difference between the feature descriptor and the corresponding key point in the real-time image exceeds a threshold, it is considered a changed point. After calculating all the change points, for similar change points, a rectangle is calculated to accommodate these change points, and the rectangle is a change area; The real-time image is cropped based on the changed area to obtain the changed area image.

2. The robot inspection system based on comparison detection according to claim 1, characterized in that, The system operation includes two phases: the initial phase and the operation phase; The initial stage includes: robot path planning, robot location setting, setting and storing benchmark maps by time period and season; robot path planning involves setting the robot's inspection route in the area to be inspected; robot location setting involves setting visual markers for robot stopping positions and posture adjustments for locations requiring attention; setting benchmark maps by time period and season involves capturing benchmark maps using the video perception module in a general video monitoring module during periods and seasons with significant differences in lighting conditions; storing benchmark maps in the database involves saving benchmark maps for each location to be inspected using the server module in a general video monitoring module for retrieval during the operational phase. The operational phase includes: robot inspection, robot docking, robot posture adjustment, comparative detection, anomaly identification, and anomaly handling. Robot inspection refers to the robot inspecting along a planned path. Robot docking refers to the robot stopping at the positions set during the initial robot positioning phase. Robot posture adjustment refers to the robot adjusting the posture of its camera according to visual markers. Comparative detection involves using a comparative detection module to compare and detect changes in a reference image and the image currently captured by the robot. Anomaly identification involves using an anomaly identification module to judge detected changes and determine whether they are anomalies or false detections. Anomaly handling involves using an alarm notification module to issue alarms and notifications for identified anomalies.

3. The robot inspection system based on comparison detection according to claim 2, characterized in that, The comparison detection module performs comparison detection, including: Real-time video and image validity detection, that is, detecting and confirming whether there are screen distortion and frame drops caused by the camera device on the robot, as well as occlusion and motion blur that occur during the inspection process; Image calibration transformation is the process of adjusting the optical distortion of images captured by the camera device to the same paradigm. Image comparison detection is a method of locating the difference region by comparing two images; the image comparison detection method excludes differences caused by perspective and only retains the part of the target that has changed.

4. The robot inspection system based on comparison detection according to claim 3, characterized in that, The subsequent processing described in step 6 specifically includes: If the returned information is determined to be abnormal, it will be recorded in the log, and a handling plan will be queried and reported. If it is determined to be normal, the inspection will continue.

5. The robot inspection system based on comparison detection according to claim 4, characterized in that, The comparison detection module is deployed on the robot's end or in a cloud server at the back end.

6. The robot inspection system based on comparison detection according to claim 5, characterized in that, The robot is a mobile platform, and it has the ability to plan paths and move.

7. The robot inspection system based on comparison detection according to claim 6, characterized in that, The robot inspection system is configured on the computing device carried by the robot or deployed in the cloud.

Images