Visual correction method and system for four-footed inspection robot

Abstract

In order to solve the problems of low correction efficiency and short correction validity period caused by complex correction conditions and large calculation amount in the prior art, the invention provides the visual correction method and system for the four-footed inspection robot, and the correction method and system for the four-footed inspection robot carry out staged correction by taking pictures with different multiplying powers and respectively judging the pictures, so that the correction efficiency is greatly improved. Compared with the prior art, a large search range is not needed any more, complex calculation does not need to be carried out every time, after pose correction and cradle head correction are completed, correction does not need to be carried out again within a short time for the same target, and therefore the method is simple in correction condition, small in calculation amount, high in correction efficiency and long in correction validity period.

Description

Technical Field

[0001] The invention relates to the field of data processing, and in particular to a method and system for visual deviation correction of a four-legged inspection robot. Background Art

[0002] At present, intelligent inspection robots often encounter problems such as the offset of shooting target objects during autonomous operation. The main reason for this is the cumulative motion deviation of the inspection robot. For example, the inspection robot encounters a large number of preset position offsets due to positioning and odometer deviations, and the gimbal camera gradually experiences a large number of offsets due to wear and tear of the gimbal and internal mechanical parts of the camera.

[0003] For the problems such as the offset of the shooting target object that often occurs in the autonomous operation of intelligent inspection robots, the current solutions or recovery solutions mainly rely on manual / automatic recalibration and calibration solutions. For example, the patent with publication number CN111414012A discloses a method for regional retrieval and gimbal correction of inspection robots, including the following steps: S1, the gimbal arrives at the specified position and calculates whether the environment matches the template image according to the matching algorithm; S2, if so, take a picture and record the gimbal posture; if not, expand the search range until the environment matches the template image, then take a picture and record the gimbal posture; S3, bind the specified position and the gimbal posture. However, this solution has complex conditions and requires a large search range, otherwise it is difficult to complete the deviation correction.

[0004] For example, the publication (announcement) number CN109976344A proposes a posture correction method for an inspection robot, and the specific steps are: complete the camera calibration according to the transformation relationship between the image coordinate system and the camera coordinate system, and the camera coordinate system and the world coordinate system, and obtain the camera internal parameters; perform feature matching on the images taken by the inspection robot based on the feature extraction and matching algorithm of ORB, and optimize the matching results; establish the posture relationship between the camera and the robot, perform relative posture solution based on the obtained matching point pairs, and obtain the current rotation angle of the robot, which includes the odometer rotation angle and the visual rotation angle; fuse the odometer rotation angle and the visual rotation angle to obtain the current angle of the robot to be corrected, and complete the posture correction. However, this solution has a large amount of calculation and is mainly used for angle correction. If position information is added, the additional amount of calculation will significantly affect the inspection efficiency and is difficult to implement.

[0005] Therefore, although the existing technology utilizes visual correction technology, it has great limitations and cannot be fully applied to various inspection scenarios. Moreover, as time goes by, the accumulated errors or the wear of the motion mechanism increase, the probability and degree of deviation will increase. Only repetitive correction will result in a large workload and low efficiency. Summary of the invention

[0006] In view of the problems in the prior art that the correction conditions are complex and the amount of calculation is large, resulting in low correction efficiency and a short correction validity period, the present invention provides a method and system for visual correction of a four-legged inspection robot. By taking pictures of different magnifications and judging them separately, correction is performed in stages. A larger search range is no longer required, and complex calculations do not need to be performed each time. After the posture correction and gimbal correction are completed, the same target does not need to be corrected again in a short period of time. Therefore, the correction conditions of the present invention are simple, the amount of calculation is small, the correction efficiency is high and the correction validity period is long.

[0007] The following is the technical solution of the present invention.

[0008] A method for visual deviation correction of a quadruped inspection robot comprises the following steps: Main process: after judging that the inspection robot has reached the preset point, the target position is photographed based on the first magnification to obtain a first picture, and the first correction judgment is performed on the first picture. If the first correction judgment passes, the target position is photographed based on the second magnification to obtain a second picture, and the second correction judgment is performed on the second picture. If the second correction judgment still passes, no correction is performed; Posture correction: If the first correction judgment fails, the posture correction is performed according to the offset value of the target position in the first image until the first correction judgment is passed, and the posture parameters and preset point parameters of the inspection robot are updated, and the second correction judgment is continued; Gimbal correction: If the second correction judgment fails, the target position is photographed based on the third magnification to obtain a third picture, and the gimbal parameters are adjusted according to the offset value of the target position in the third picture until the third picture passes the third correction judgment. The gimbal parameters are updated and the second correction judgment is re-executed.

[0009] Preferably, the first magnification is less than the required magnification for patrol inspection photography, and the second magnification is equal to the required magnification for patrol inspection photography.

[0010] Preferably, the third magnification is greater than the required magnification for inspection photography, or is between the first magnification and the second magnification.

[0011] Preferably, performing a first correction judgment on the first image includes: using feature point matching or a deep learning algorithm, matching the first image with a preset first template, calculating a matching score, and passing if the matching score is greater than a first threshold, otherwise it is deemed to have failed.

[0012] Preferably, performing a second correction judgment on the second image includes: using feature point matching or a deep learning algorithm, matching the second image with a preset second template, calculating a matching score, and passing if the matching score is greater than a second threshold, otherwise it is deemed to have failed.

[0013] Preferably, the third correction judgment includes: using feature point matching or deep learning algorithm, using the third image to match with a preset third template, calculating the matching score, if the matching score is greater than a third threshold, it is passed, otherwise it is deemed to have failed.

[0014] Preferably, the posture correction is performed according to the offset value of the target position in the first image, including: comparing the coordinates of feature points of the first image and the first template, calculating the adjustment parameters of the inspection robot through inverse kinematics according to the coordinate offset values ​​of different feature points, and updating the posture parameters and preset point parameters based on the adjustment parameters.

[0015] Preferably, the target position is photographed based on a third magnification to obtain a third picture, and the gimbal parameters are adjusted according to the offset value of the target position in the third picture, including: comparing the coordinates of feature points of the third picture with those of a third template, calculating the adjustment parameters of the gimbal through inverse kinematics according to the coordinate offset values ​​of different feature points, and updating the gimbal parameters based on the adjustment parameters.

[0016] A system for visual deviation correction of a four-legged inspection robot comprises an inspection robot body and a deviation correction control module. When the deviation correction control module is in operation, the above-mentioned method for visual deviation correction of a four-legged inspection robot is executed.

[0017] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps of the above-mentioned method for visual deviation correction of a four-legged inspection robot are implemented.

[0018] The present invention also provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the steps of the above-mentioned method for visual deviation correction of a four-legged inspection robot are implemented.

[0019] The substantial effects of the present invention include: By taking pictures of different magnifications in stages and making correction judgments, the correction conditions are effectively simplified and the amount of calculation is reduced. In the main process, first take pictures at a lower magnification and make a preliminary correction judgment. If it fails, the posture correction is performed; then take pictures at a normal magnification for a secondary judgment. If it still fails, use a higher or moderate magnification to shoot and perform pan-tilt correction. Use feature point matching or deep learning algorithms to match pictures with templates, and judge whether the correction is passed based on the matching score. At the same time, adjust the posture and pan-tilt parameters based on the offset value. Not only does it improve the correction efficiency, but it also extends the correction validity period, so that the inspection robot does not need to repeat the correction of the same target in a short time. In general, the visual correction method and system of the quadruped inspection robot of the present invention achieves efficient and accurate visual correction by optimizing the correction process, improves the working performance and stability of the inspection robot, and has significant technical advantages and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution will be clearly and completely described below in combination with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0023] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0024] It should be understood that in the present invention, "plurality" refers to two or more than two. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "Contains A, B and C", "Contains A, B, C" means that A, B, and C are all included, "Contains A, B or C" means that one of A, B, and C is included, and "Contains A, B and / or C" means that any one, any two, or any three of A, B, and C are included.

[0025] The technical solution of the present invention is described in detail with specific embodiments below. The embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0026] Example: A system for visual deviation correction of a four-legged inspection robot comprises an inspection robot body and a deviation correction control module. When the deviation correction control module is in operation, the above-mentioned method for visual deviation correction of a four-legged inspection robot is executed.

[0027] In this embodiment, the inspection robot body adopts a quadruped robot, which has autonomous navigation, environmental perception and image acquisition capabilities. It is also equipped with a pan-tilt head for shooting images of inspection points, and the pan-tilt head supports PTZ (Pan-Tilt-Zoom) function.

[0028] like Figure 1 As shown, a method for visual deviation correction of a quadruped inspection robot includes the following steps: Main process: after judging that the inspection robot has reached the preset point, the target position is photographed based on the first magnification to obtain a first picture, and the first correction judgment is performed on the first picture. If the first correction judgment passes, the target position is photographed based on the second magnification to obtain a second picture, and the second correction judgment is performed on the second picture. If the second correction judgment still passes, no correction is performed; The first magnification is less than the required magnification of the inspection photography, the second magnification is equal to the required magnification of the inspection photography, that is, the normal magnification, and the third magnification can be greater than the required magnification of the inspection photography, or between the first magnification and the second magnification.

[0029] Performing a first correction judgment on the first image includes: using feature point matching or deep learning algorithm, using the first image to match with a preset first template, calculating the matching score, and passing if the matching score is greater than a first threshold, otherwise it is considered as not passing. For example, if the matching score is 85%, which is greater than the first threshold of 80%, proceed to the next step.

[0030] Performing a second correction judgment on the second image includes: using feature point matching or deep learning algorithm, using the second image to match with the preset second template, calculating the matching score, if the matching score is greater than the second threshold, it is passed, otherwise it is considered as failed. For example, if the matching score is 90%, which is greater than the second threshold of 85%, the process ends normally.

[0031] The third correction judgment includes: using feature point matching or deep learning algorithm, using the third image to match with a preset third template, calculating the matching score, if the matching score is greater than a third threshold, it passes, otherwise it is deemed to have failed.

[0032] In this embodiment, at each inspection point, high-quality pictures need to be taken in advance as templates according to actual needs. For example, in the inspection of a substation, the transformer nameplate is selected as the shooting object, and pictures of small magnification (template A, as the first template), normal recognition magnification (template B, as the second template) and other recognition magnifications (template B-1, as the third template) are taken and stored in the template picture library.

[0033] For example, the first image has the same small magnification (the magnification of template A). Based on the gradient angle of the header, the similar area is found through fast Fourier transform, and the correctness of the template matching result is verified by the fast Fourier transform result. If the magnification is different, the extracted header is scaled according to the ratio of the current magnification to the small magnification (assuming that the original small magnification is 30 and the current magnification is 90, the extracted template header is enlarged by 90 / 30=3 times, and if the current magnification is 15, the header is reduced to 1 / 2 of the original), and the scaled header is used as the template for template matching with the current image. Considering the deviation between optical zoom and actual pixel zoom (that is, after the magnification is 30->90, the pixel size occupied by the header is not necessarily enlarged by 3 times), the above module matching is searched according to a certain scale (the search scale is 1:1 when the magnifications before and after are consistent, and can be set to (0.7~1.1):1) when they are inconsistent.

[0034] Posture correction: If the first correction judgment fails, the posture correction is performed according to the offset value of the target position in the first image until the first correction judgment is passed, and the posture parameters and preset point parameters of the inspection robot are updated, and the second correction judgment is continued, including: The coordinates of the feature points of the first image and the first template are compared, and the adjustment parameters of the inspection robot are calculated through inverse kinematics according to the coordinate offset values ​​of different feature points, and the posture parameters and preset point parameters are updated based on the adjustment parameters.

[0035] PTZ correction: If the second correction judgment fails, the target position is photographed based on the third magnification to obtain a third picture, and the PTZ parameters are adjusted according to the offset value of the target position in the third picture until the third picture passes the third correction judgment, the PTZ parameters are updated, and the second correction judgment is re-executed, including: Compare the coordinates of the feature points of the third image and the third template, calculate the adjustment parameters of the gimbal through inverse kinematics according to the coordinate offset values ​​of different feature points, and update the gimbal parameters based on the adjustment parameters, such as adjusting the gimbal angle and zoom.

[0036] After each successful match, this embodiment updates the current robot's posture parameters and PTZ parameters to the system database as a reference for subsequent operations. After multiple iterations, the system is gradually optimized to form the best posture and PTZ configuration solution for a specific inspection environment.

[0037] In this embodiment, the correction of the offset is achieved through the "zoom center + zoom factor" model. By using similarity transformation, the pixel coordinates of the object after zooming can be predicted (linear relationship, which can be represented by a matrix). Therefore, by converting the pixel deviation of moving the object to the center of the image (at this time the object is "straightened") into the pan / tilt angle, it can be ensured that the object is imaged at the center of the image. Since the actual zoom process does not necessarily satisfy such a linear relationship, and the zoom center may change significantly, the use of a single linear model will cause a large deviation, so the zoom center and zoom factor between adjacent camera magnifications are actually recorded here through calibration. For zooms with a larger span, the pixel coordinates of different magnifications are calculated in a step-by-step manner to obtain the final predicted coordinates. For example, assuming the magnification of the small image is z1, the magnification of the large image is z2, and the point P in the three-dimensional coordinates is z1. i The projection points in the small and large images are When the camera changes from small magnification to large magnification, the center of the image is The corresponding scaling system is

[0038] The corresponding equation is: Transformed into a matrix: After multiple image matching, the accurate shooting posture and gimbal parameters are obtained, and correction based on visual algorithms is achieved.

[0039] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps of the above-mentioned method for visual deviation correction of a four-legged inspection robot are implemented.

[0040] This embodiment also provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the steps of the above-mentioned method for visual deviation correction of a four-legged inspection robot are implemented.

[0041] The substantial effects of this embodiment include: By taking pictures of different magnifications in stages and making correction judgments, the correction conditions are effectively simplified and the amount of calculation is reduced. In the main process, first shoot at a lower magnification and make a preliminary correction judgment. If it fails, perform posture correction; then shoot at a normal magnification for a secondary judgment. If it still fails, use a higher or moderate magnification to shoot and perform pan-tilt correction. Use feature point matching or deep learning algorithms to match the picture with the template, and judge whether the correction is passed based on the matching score. At the same time, adjust the posture and pan-tilt parameters based on the offset value. Not only does it improve the correction efficiency, but it also extends the correction validity period, so that the inspection robot does not need to repeat the correction of the same target in a short time. In general, the visual correction method and system of the quadruped inspection robot of this embodiment achieves efficient and accurate visual correction by optimizing the correction process, improves the working performance and stability of the inspection robot, and has significant technical advantages and application value.

[0042] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the specific device can be divided into different functional modules to complete all or part of the functions described above.

[0043] In the embodiments provided in the present application, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the embodiments of the structure described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another structure, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0044] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0045] In addition, each functional unit in the embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0046] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0047] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for visual deviation correction of a quadruped inspection robot, characterized in that: The following steps are involved: Main process: after judging that the inspection robot has reached the preset point, the target position is photographed based on the first magnification to obtain a first picture, and the first correction judgment is performed on the first picture. If the first correction judgment passes, the target position is photographed based on the second magnification to obtain a second picture, and the second correction judgment is performed on the second picture. If the second correction judgment still passes, no correction is performed; Posture correction: If the first correction judgment fails, the posture correction is performed according to the offset value of the target position in the first image until the first correction judgment is passed, and the posture parameters and preset point parameters of the inspection robot are updated, and the second correction judgment is continued; Gimbal correction: If the second correction judgment fails, the target position is photographed based on the third magnification to obtain a third picture, and the gimbal parameters are adjusted according to the offset value of the target position in the third picture until the third picture passes the third correction judgment. The gimbal parameters are updated and the second correction judgment is re-executed.

2. The method for visual deviation correction of a quadruped inspection robot according to claim 1, characterized in that: The first magnification is less than the required magnification of the patrol inspection and shooting, and the second magnification is equal to the required magnification of the patrol inspection and shooting.

3. The method for visual deviation correction of a quadruped inspection robot according to claim 1, characterized in that: The third magnification is greater than the required magnification for inspection photography, or is between the first magnification and the second magnification.

4. The method for visual deviation correction of a quadruped inspection robot according to claim 1, characterized in that: The performing of the first correction judgment on the first image includes: using feature point matching or a deep learning algorithm, using the first image to match a preset first template, calculating a matching score, and passing if the matching score is greater than a first threshold, otherwise it is considered to have failed.

5. The method for visual deviation correction of a quadruped inspection robot according to claim 1, characterized in that: The performing of the second correction judgment on the second image includes: using feature point matching or a deep learning algorithm, using the second image to match a preset second template, calculating a matching score, and passing if the matching score is greater than a second threshold, otherwise it is considered to have failed.

6. The method for visual deviation correction of a quadruped inspection robot according to claim 1, characterized in that: The third correction judgment includes: using feature point matching or deep learning algorithm, using the third image to match with a preset third template, calculating the matching score, if the matching score is greater than a third threshold, it is passed, otherwise it is considered to have failed.

7. The method for visual deviation correction of a quadruped inspection robot according to claim 4, characterized in that: The posture correction according to the offset value of the target position in the first image includes: comparing the coordinates of the feature points of the first image and the first template, calculating the adjustment parameters of the inspection robot through inverse kinematics according to the coordinate offset values ​​of different feature points, and updating the posture parameters and preset point parameters based on the adjustment parameters.

8. The method for visual deviation correction of a quadruped inspection robot according to claim 6, characterized in that: The method comprises: photographing the target position based on the third magnification to obtain a third picture, and adjusting the gimbal parameters according to the offset value of the target position in the third picture, including: comparing the coordinates of feature points of the third picture with those of the third template, calculating the adjustment parameters of the gimbal through inverse kinematics according to the coordinate offset values ​​of different feature points, and updating the gimbal parameters based on the adjustment parameters.

9. A visual deviation correction system for a quadruped inspection robot, comprising an inspection robot body and a deviation correction control module, characterized in that: When the deviation correction control module is running, the method for visual deviation correction of a quadruped inspection robot as described in any one of claims 1 to 8 is executed.

Images