[0058] The present invention will be further limited in combination with the drawings and specific embodiments.
[0059] A visual servo-based substation inspection robot pan-tilt control method. First, the configuration of the inspection robot includes the need to save the equipment images of each preset position of the substation taken by the robot to the template library, and the equipment area to be inspected It is presented in the center of the image with an appropriate size, and the equipment area to be detected is manually calibrated in the template image. The equipment image in the template library corresponds to the equipment position of the substation. When assigning inspection tasks to the robot, it is necessary to clearly point out the various parameters of the template image taken at each preset parking space (such as pan-tilt angle, camera focal length, etc.), and the inspection robot travels to this preset parking space After that, the posture adjustment is performed according to the collection method of the device template image, the real-time visible light image or infrared image of the device is taken and the image is transmitted back to the upper pattern recognition server. The server retrieves the specified template image from the template library of the device, and then uses the SIFT (Scale-invariant feature transform) algorithm to match the real-time inspection image with the template image, find the common feature points of the two images, and calculate The pixel displacement deviation of each feature point pair after the template image and the collected image are screened is calculated and the average value is calculated to verify whether the collected image contains the complete area of the equipment to be tested. If the device area is complete, the captured image can be directly used for device status recognition. Otherwise, if the device area is out of the image, calculate the PTZ rotation angle offset represented by the image pixel offset according to the tangent ratio of the image distance to the focal length , And convert the angle offset into the PTZ rotation control parameters, and call the PTZ rotation to compensate the previous angle error. This process is called the "visual servo" of the PTZ. Due to the accuracy of the small-angle motion control of the gimbal itself, the gimbal needs to be fine-tuned at a small angle (horizontal or vertical rotation is less than 2 degrees). The gimbal first rotates in the reverse direction to a certain large angle, and then moves forward with the same angle value plus error The sum of angle values, this method is mostly used for "secondary servo", and through verification, the defect of inaccurate fine-tuning of small angles can be solved. If the robot needs to observe the details of equipment such as meters from a long distance (over 15 meters from the target), the camera should be zoomed in and the magnification should be increased (for example, the focal length is 25 times or more). The image is prone to large deviation errors, resulting in the failure of the registration with the template image due to the lack of the same image information, which makes it impossible to calculate the image offset for PTZ correction control. In such cases, it is necessary to enable "large focus servo" The strategy is to collect the low-magnification focal length image and the high-magnification focal length image separately when configuring the preset template image of the robot. When the robot performs inspection tasks on the preset position, first call the low-magnification template image shooting parameters to perform the image Acquisition and PTZ calibration control. After completing a visual servoing, keep the PTZ angle unchanged and only call the focal length parameters of high-magnification template image shooting. After the captured image matches the template, perform "secondary servo" to fine-tune the PTZ angle. When performing inspection tasks at night or under poor visibility conditions, the robot can use the infrared thermal imager it carries to collect the thermal image of the device, calculate the pixel deviation of the infrared image, calculate the pan-tilt error, and complete the angle compensation, so that the robot can be in 24 hours Accurately collect visible light images and infrared thermal images of electrical equipment under all weather conditions. At this point, after performing the corresponding "visual servoing" at any preset position, the error between the picture presented in the robot camera and the template image is already very small, and there is no need to make adjustments. Collect real-time equipment inspection images and template images to match the equipment area Calibration and identification of equipment working status.
[0060] figure 1 As shown, the flow chart of the "visual servo-based substation inspection robot pan-tilt control method" of the present invention:
[0061] The robot pan/tilt control method based on visual servoing is divided into the following specific steps:
[0062] Step 1. Configure the inspection robot. The background operator controls the robot to drive to the inspection preset position of each device, set the appropriate PTZ rotation angle and camera focal length and other information, and display the area of the device to be inspected in an appropriate size After the center of the image, it is taken as the template image of the device, and the device area is calibrated in the image, and the image and parameters are saved to the template image database. If it is a device that needs to zoom in for observation, first configure the primary visual servo template image with the standard focal length (preset value), and then zoom the camera focus closer to configure the secondary servo template image with visible details required for pattern recognition.
[0063] Step 2: After receiving the inspection task, the robot stops at the preset position in sequence according to the equipment in the inspection route, and calls the parameters such as the angle of the pan/tilt and the focal length of the camera according to the acquisition method of the equipment template image.
[0064] Step 3. Collect device images, take visible light images or infrared images, and upload them to the background service.
[0065] Step 4, call the template image of the device, use the SIFT (Scale-invariant feature transform) algorithm to match the real-time inspection image and the template image, and register the common feature points of the two images.
[0066] Step 5. Calculate the pixel displacement deviation corresponding to each pair of feature points in the above registration, count the average value of the pixel displacement deviation corresponding to all the feature points, and use the calibration device area position information in the template image to subtract the same features obtained from the image registration Point the offset pixel offset(pix) in the collected image to obtain the position information of the device area in the collected image.
[0067] Step 6, verify whether the collected image contains the complete area of the equipment to be detected.
[0068] Step 7. If the verification result of step 6 is that the device area does not deviate from the image, it is determined whether the focal length needs to be increased for "secondary servo" based on the device template parameters. If you need to pull the focus, keep the pan/tilt position, and perform step 3 after adjusting only the focus; if you do not need to pull the focus, perform step 13 to identify the working status of the device.
[0069] Step 8. If the verification result of step 6 is that the device area deviates from the image, according to the tangent ratio of the image distance to the focal length, calculate the pan/tilt rotation angle offset represented by the image pixel offset, and convert the angle offset to cloud PTZ rotation control parameters are called to compensate the previous angle error.
[0070] Step 9. Determine whether the angle error calculated in step 8 is a "small angle", that is, the angle value is less than the set threshold.
[0071] Step 10. If the result of step 9 is "small angle", the pan/tilt adopts the "small angle" compensation motion control strategy. The pan/tilt first rotates in reverse to a certain large angle (preset value of the control strategy), and then moves forward The sum of the same angle value plus the error angle value, in order to overcome the error caused by the short movement distance, the pan/tilt motor may not reach the set speed, and thus the inertia movement cannot be achieved due to the low speed. Then, go to step 12.
[0072] Step 11. If the result of step 9 is not "small angle", then the pan/tilt compensation movement normally follows the error direction.
[0073] Step 12: From step 3 to step 11, a complete "visual servo" control pan-tilt compensation is completed, and the number of "visual servo" is counted in this step. If the number of times reaches a certain preset value, it means that the work of collecting images at this preset position has failed. According to the sequence of step 2, the robot travels to the next preset position; if not, step 3 is performed.
[0074] Step 13: Perform pattern recognition processing on the device image collected by the robot at the preset position for the last time, and output the real-time working status of the device. According to the sequence of step 2, the robot travels to the next preset position.
[0075] The method of pattern recognition on the device image is:
[0076] (1) Through the feature point registration of the collected image and the device template image, the coordinate position of the device area in the real-time collected image is determined, and the sub-image of the device area to be recognized is cut out in the collected image;
[0077] (2) According to the equipment type corresponding to the preset position in the equipment template library, such as instrument, switch, switch, etc., call the corresponding image processing and pattern recognition algorithm to identify the equipment working status on the equipment area sub-map. The equipment recognition algorithm for example refers to the paper, [Fang Hua, Ming Zhiqiang, etc., an instrument recognition algorithm suitable for substation inspection robots [J], "Automation and Instrumentation" 2013, Issue 5].
[0078] Among them, the image SIFT registration algorithm in step 4 is as follows:
[0079] (1) Use a set of continuous Gaussian convolution kernels to convolve with the original image to generate an image in the scale space, subtract the images of adjacent scales, search for local extreme points, and determine the location of key points through the detection of scale space extremes And scale
[0080] (2) Use the gradient direction distribution characteristics of the pixels in the neighborhood of the key point to specify the direction parameter for each key point, so that the key point has rotation invariance;
[0081] (3) Take the key point as the center, calculate the neighborhood gradient histogram, draw the cumulative value of each gradient direction, and form the SIFT feature vector for a key point;
[0082] (4) Calculate the matching relationship between the feature points of the two images: After generating the SIFT feature vector of the image, use the Euclidean distance of the key point feature vector in the two images as the similarity judgment measure, and take a key point in the source image , And find the first two key points closest to the Euclidean distance in the image to be matched. In these two key points, if the closest distance divided by the next closest distance is less than a certain ratio threshold, then this pair of matching is accepted Characteristic points;
[0083] (5) After obtaining the matching relationship between the feature points of the two images, the two images have such a transformation process:
[0084]
[0085] [x',y',1] T ,[x,y,1] T They are respectively the image point coordinates of a certain set of feature matching points on the source image and the image to be matched. Calculate the projection transformation matrix H, and realize the accurate positioning of the device in the image to be detected according to the device position in the template image.
[0086] Among them, the formula for calculating the pixel displacement deviation between the template image and the characteristic points of the collected image in step 5 is:
[0087] Calculate the pixel displacement deviation of each feature point pair after the template image and the collected image are screened, and calculate the average value.
[0088]
[0089] offset (pix): pixel offset, horizontal (H) and vertical (V) direction; C temp : The position of the feature point in the template image; C cap : The position of the feature point in the collected image.
[0090] Among them, the calculation method of converting the image pixel deviation in step 8 into the PTZ rotation compensation angle value is as follows:
[0091] (1) PTZ offset angle calculation
[0092] According to the tangent ratio of the image distance to the focal length, calculate the PTZ rotation angle offset represented by the image pixel offset, such as figure 2 Shown:
[0093] Offset is the physical offset distance of the image pixel offset projected on the (CCD or IR) camera imaging device, calculated using the following formula:
[0094]
[0095] offset: the physical offset of the pixel offset on the imaging device, in the horizontal (H) and vertical (V) directions;
[0096] solution: The actual distance occupied by each pixel on the imaging device.
[0097] in figure 2 In, since the distance between the target position T and the lens is far greater than the focal length of the lens, the corresponding T'point in the template image and the acquired image of the T point can be approximated as a positional translation, and t and t' The points are the projections of points T and T'on the imaging device, and the displacement of t'relative to t represents the pixel offset between the template image calculated by formula (1) and the captured image. Then the point T'is generated relative to the point T The lens rotation angle is:
[0098]
[0099] offset (Ang): the deflection angle between the template image and the acquired image, in the horizontal (H) and vertical (V) directions.
[0100] f is the image shooting focal length of the camera.
[0101] (2) PTZ rotation control amount conversion
[0102] When the offset of the acquired image corresponding to the template image is compensated by the rotation of the pan/tilt, the corresponding compensation value of the movement control of the pan/tilt is calculated as follows:
[0103]
[0104] ptzCtrlOffset is the PTZ control compensation, ptzSolution is the control sampling value of the PTZ rotation unit angle, both in the horizontal (H) and vertical (V) directions.
[0105] (3) PTZ offset compensation rotation control
[0106] The target position of the gimbal movement is the current coordinate of the gimbal plus gimbal movement compensation, as follows:
[0107]
[0108] Among them, ptz is the control parameter value after the angle rotation compensation of the PTZ, ptzCur is the current position parameter value of the PTZ, ptzCtrlOffset is the parameter value of the PTZ control compensation, and the image offset is the offset of the collected image relative to the template image ( That is, the captured image-template image); (-V) is because when the pan-tilt moves up excessively, the position of the feature point in the captured image is lower than the template image, the image offset is "+", and the compensation value should be It is "—"; the compensation in the horizontal (H) direction is consistent with the sign of the compensation amount.
[0109] After the experimental test and the work record inspection of the robot running on the substation site, the "visual servo-based substation inspection robot pan-tilt control method" of the present invention greatly improves the accuracy of the substation inspection robot collecting equipment images. The accuracy of the collected images after the servo can be over 99%, and the robot can observe the details of the operation status of the power equipment at a long distance and a large focal length, achieving the goal of the inspection robot to observe the full coverage of the substation equipment. The invention indicates that the inspection robot can autonomously adjust and control the PTZ attitude through the visible light or infrared images taken in real time, realizes the automatic control of the "head" and "neck" parts of the robot, frees the background operator, and saves labor. Labor resources. The robot can carry visible light and infrared cameras for visual servoing. After the implementation of the present invention, the robot can perform a 24-hour round-the-clock power equipment inspection task in the substation to ensure the safe operation of the substation equipment.
[0110] Although the specific embodiments of the present invention are described above with reference to the accompanying drawings, they do not limit the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solutions of the present invention, those skilled in the art do not need to pay creative work Various modifications or variations that can be made are still within the protection scope of the present invention.
