Inspection process of an inspection robot

Abstract

The present invention relates to the field of image acquisition technology, and discloses a method for high-quality automatic fixed-point image acquisition of an inspection robot, comprising the following steps: S1: adjusting the focal length f of the camera; S2: turning on the camera's automatic focus mode and waiting for the focusing to end, at which point the camera's image distance is v; S3: capturing a target object in the image clearly, recording the target object's image distance as vt, where vt=v; observing whether the background and foreground in the image are within the same depth of field, recording the non-target object's equivalent image distance as vnt, where vnt=v; if not within the same depth of field, switching the camera to manual focus mode, manually adjusting the camera's equivalent image distance to v, making the background or foreground clear, and recording the non-target object's equivalent image distance as vnt, where vnt=v. In the present invention, the image distance can be adaptively adjusted to make the captured target clear, and can effectively solve the problem of blurred shooting caused by the fixed image distance when the camera's actual focal length changes in the existing solution one.

Description

Technical Field

[0001] The present invention relates to the technical field of image acquisition, and in particular to an inspection process of an inspection robot. Background Art

[0002] With the continuous advancement of the intelligent power grid in China, intelligent substation inspection robots are playing an indispensable role in maintaining the normal and stable operation of equipment. Equipped with variable-magnification visible light cameras, thermal imaging cameras, and other sensors, these robots conduct regular inspections of substation equipment. Monitoring of substation equipment appearance, meter readings, and switch opening and closing status primarily relies on visible light cameras to capture images for identification and analysis. Existing intelligent inspection robots employ visible light target inspection solutions, primarily the following: Solution 1: Using a teach-in-like method, the target position and visible light camera parameters are pre-calibrated for a clear target. During the inspection, the camera is re-positioned to the calibrated preset position and an image is captured. Solution 2: Using the camera's autofocus function to capture images, the target position and visible light camera parameters are pre-calibrated for a clear target. During the inspection, the camera is positioned at the calibrated preset position, autofocus is enabled, and the image is captured after focusing is complete. This existing technical solution, Solution 1, assumes that the camera's focal length remains unchanged during the calibration and inspection process. The principle formula for camera imaging is as follows:

[0003]

[0004] Where f, v, and u represent the focal length, image distance, and object distance, respectively. The prior art assumes that the focal length f of the camera does not change during calibration and inspection, and the distance u between the camera and the target is also fixed. Therefore, the movement parameters can be pre-calibrated, and during inspection, the movement is directed to a pre-set position to complete image acquisition.

[0005] However, zoom cameras generally consist of components such as a lens group, an imaging unit, and an aperture. The camera movement uses a stepper motor and a lead screw to change the relative positions of the lenses in the lens group, thereby changing the overall focal length of the camera and achieving optical zoom. Therefore, the combination of the lens group determines the camera's true focal length f. When the camera's focal length f changes, continuing to use the image distance v used during calibration for imaging will cause the captured image to appear out of focus and blurry. The combination of the lenses is affected by factors such as temperature, wear between the actuators, and execution errors of the actuator motor. These uncontrollable factors will cause the camera's true focal length f to change during inspections. When there is a large deviation between the true focal length and the focal length used during calibration, the image distance v used during calibration cannot guarantee clear imaging of the target, resulting in a decrease in the quality of visible light image acquisition and affecting the analysis of inspection results.

[0006] Existing technical solution 2 effectively avoids the problem of out-of-focus images captured with fixed camera parameters due to changes in the camera's actual focal length f. However, in substation inspection environments, where the foreground or background may be present, the camera's automatic focus may focus on these non-target objects, resulting in blurred images of the target object. Therefore, we propose a patrol robot inspection process. Summary of the Invention

[0007] The present invention mainly solves the technical problems existing in the above-mentioned prior art and provides an inspection process of an inspection robot.

[0008] In order to achieve the above objectives, the present invention adopts the following technical solution, a patrol robot inspection process, including the following steps:

[0009] S1: Controls the camera to execute the preset position during calibration and turns on the autofocus mode;

[0010] S2: After the autofocus is completed, collect a picture image0 and record the equivalent image distance of the camera at this time as v c ;

[0011] S3: According to the current v c Non-target objects v during calibration nt Calculate the actual focal length f of the current camera c , the calculation formula is as follows:

[0012] f c =f+(v c -v nt )*gd (2)

[0013] In the above formula, f is the focal length of the camera when calibrating the inspection point, and gd is the gradient of the focal length changing with the equivalent image distance, which is obtained by calibrating the camera;

[0014] S4: Calculate the compensation coefficient of the equivalent image distance difference

[0015]

[0016] In the above formula, f is the focal length of the camera when calibrating the inspection point, f c is the actual focal length value calculated in S3;

[0017] S5: Calculate the equivalent image distance compensation value d v

[0018] d v =(v t -v nt )*α (4)

[0019] In the above formula, v t 、vnt are the image distances of the camera with clear target and clear non-target objects when calibrating the inspection point, respectively, and α is the equivalent image distance difference compensation coefficient calculated in S4;

[0020] S6: Calculate the corrected equivalent image distance v m

[0021] v m =v t +d v (5)

[0022] S7: Set the camera equivalent image distance to v m , collect a picture image1;

[0023] S8: Use the deep learning model to identify images0 and image1 and obtain the region ROI where the target in the image is located;

[0024] S9: Grayscale the ROIs of the two images and calculate the clarity coefficients δ0 and δ1 of the target areas of the two images. The larger the clarity coefficient, the clearer the image.

[0025] S10: Save the image with the largest resolution as the final image; t 、v nt Obtain it through the following methods:

[0026] A1: Adjust the camera focal length f;

[0027] A2: Turn on the camera's autofocus mode and wait for the focus to complete. The camera image distance is now v.

[0028] A3: If the target object is clear in the picture, record the target image distance as v t , v t =v; Observe whether the background and foreground are within the same depth of field. If they are, record the equivalent image distance of the non-target object as v nt , v nt =v; If they are not in the same depth of field, switch the camera to manual focus mode and manually adjust the camera equivalent image distance to v1 to make the non-target object clear, and record the equivalent image distance value of the non-target object as v nt , v nt =v1;

[0029] If the target in the picture is not clear, record the non-target object distance as v nt , v nt =v; switch the camera to manual focus mode, manually adjust the camera equivalent image distance to v2, so that the target object is clear, and record the target object equivalent image distance as v t , v t =v2.

[0030] As a further limitation of the above solution, in A1, the photographed object is placed in the center of the camera viewfinder and has an appropriate proportion.

[0031] As a further limitation of the above solution, the formula in S3 is based on the equivalent image distance value v c 、v nt Calculate the current real focal length f c , let the target object be A, the non-target object be B, the focal length during calibration be f, and the object distance be u A 、u B , image distance is v A 、v B , the focal length during inspection is f′, and the image distance is v′ A 、v′ B According to the imaging principle, we can get

[0032]

[0033] From formula (6), we can get the image distance differences between A and B during calibration and inspection as Δv and Δv respectively. ′ As shown in formula (7)

[0034]

[0035] In formula (8), represents the ratio of the image distance difference between A and B during calibration and inspection. Its size is only related to the change of focal length. Therefore, the formula of inspection process S4 calculates the compensation coefficient of the camera equivalent image distance difference according to the change of focal length. Combined with the equivalent image distance difference during calibration, the equivalent image distance value of the current target can be calculated by the formulas in inspection processes S5 and S6.

[0036] As a further limitation of the above scheme, image0 and image1 in S8, where image0 is obtained by automatic focusing, when automatically focusing on the target, image0 is the clearest picture of the target. If it automatically focuses on a non-target object, the camera equivalent image distance for a clear target object in the inspection is calculated according to the inspection process S3-S6, and then a picture, i.e., image1, is collected using the fixed focus mode. Then, image1 is a clear picture of the target. At this point, it is determined that one of the two pictures collected in one inspection must have a clear target. The target detection algorithm improved based on YOLO4 can quickly identify the target in the two pictures, and calculate the clarity of the target area after obtaining the target area.

[0037] As a further limitation of the above scheme, the selection of clarity judgment methods in the inspection process S9 includes image gradient-based methods, image edge detection algorithms, wavelet transform-based methods and statistical clarity judgment methods, among which image gradient-based methods include first-order derivatives Sobel, Robert, Prewitt and second-order derivatives Laplacian, LoG, and image edge detection algorithms include Canny and SUSAN.

[0038] Beneficial effects

[0039] The present invention provides a patrol robot inspection process with the following beneficial effects:

[0040] (1) The inspection process of the inspection robot proposed in the patent of this invention can adaptively adjust the image distance to make the shooting target clear, and can effectively solve the problem of blurred shooting caused by the fixed image distance when the actual focal length of the camera changes in the existing solution one. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely illustrative, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0042] Figure 1 This is the inspection flow chart of the present invention;

[0043] Figure 2 This is the relationship curve between focal length and image distance under fixed object distance of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Example: A patrol robot inspection process, such as Figure 1-Figure 2 As shown, the following steps are included:

[0046] S1: Controls the camera to execute the preset position during calibration and turns on the autofocus mode;

[0047] S2: After the autofocus is completed, collect a picture image0 and record the equivalent image distance of the camera at this time as vc ;

[0048] S3: According to the current v c Non-target objects v during calibration nt Calculate the actual focal length f of the current camera c , the calculation formula is as follows:

[0049] f c =f+(v c -v nt )*gd (2)

[0050] In the above formula, f is the focal length of the camera when calibrating the inspection point, and gd is the gradient of the focal length changing with the equivalent image distance, which is obtained by calibrating the camera;

[0051] S4: Calculate the compensation coefficient of the equivalent image distance difference

[0052]

[0053] In the above formula, f is the focal length of the camera when calibrating the inspection point, f c is the actual focal length value calculated in S3;

[0054] S5: Calculate the equivalent image distance compensation value d v

[0055] d v =(v t -v nt )*α (4)

[0056] In the above formula, v t 、v nt are the image distances of the camera with clear target and clear non-target objects when calibrating the inspection point, respectively, and α is the equivalent image distance difference compensation coefficient calculated in S4;

[0057] S6: Calculate the corrected equivalent image distance v m

[0058] v m =v t +d v (5)

[0059] S7: Set the camera equivalent image distance to v m , collect a picture image1;

[0060] S8: Use the deep learning model to identify images0 and image1 and obtain the region ROI where the target in the image is located;

[0061] S9: Grayscale the ROIs of the two images and calculate the clarity coefficients δ0 and δ1 of the target areas of the two images. The larger the clarity coefficient, the clearer the image.

[0062] S10: Save the image with the largest resolution as the final image; t 、v nt Obtain it through the following methods:

[0063] A1: Adjust the camera focal length f;

[0064] A2: Turn on the camera's autofocus mode and wait for the focus to complete. The camera image distance is now v.

[0065] A3: If the target object is clear in the picture, record the target image distance as v t , v t =v; Observe whether the background and foreground are within the same depth of field. If they are, record the equivalent image distance of the non-target object as v nt , v nt =v; If they are not in the same depth of field, switch the camera to manual focus mode and manually adjust the camera equivalent image distance to v1 to make the non-target object clear, and record the equivalent image distance value of the non-target object as v nt , v nt =v1;

[0066] If the target in the picture is not clear, record the non-target object distance as v nt , v nt =v; switch the camera to manual focus mode, manually adjust the camera equivalent image distance to v2, so that the target object is clear, and record the target object equivalent image distance as v t , v t =v2.

[0067] In A1, the subject is placed in the center of the camera frame and has an appropriate proportion.

[0068] The formula in S3 is based on the equivalent image distance value v c 、v nt Calculate the current real focal length f c , let the target object be A, the non-target object be B, the focal length during calibration be f, and the object distance be u A 、u B , image distance is v A 、v B , the focal length during inspection is f′, and the image distance is v′ A 、v′ B According to the imaging principle, we can get

[0069]

[0070] From formula (6), we can get the image distance differences between A and B during calibration and inspection as Δv and Δv respectively. ′ As shown in formula (7)

[0071]

[0072] In formula (8), represents the ratio of the image distance difference between A and B during calibration and inspection. Its size is only related to the change of focal length. Therefore, the formula of inspection process S4 calculates the compensation coefficient of the camera equivalent image distance difference according to the change of focal length. Combined with the equivalent image distance difference during calibration, the equivalent image distance value of the current target can be calculated by the formulas in inspection processes S5 and S6.

[0073] In S8, image0 and image1 are obtained by automatic focus. When the automatic focus is on the target, image0 is the clearest picture of the target. If the automatic focus is on a non-target object, the equivalent image distance of the camera for a clear target object is calculated according to the inspection process S3-S6, and then a picture, image1, is collected using the fixed focus mode. Image1 is a clear target picture. At this point, it is determined that one of the two pictures collected in one inspection must have a clear target. The target detection algorithm improved based on YOLO4 can quickly identify the target in the two pictures, obtain the target area, and calculate the clarity of the target area.

[0074] The selection of clarity judgment methods in the inspection process S9 includes image gradient-based methods, image edge detection algorithms, wavelet transform-based methods, and statistical clarity judgment methods. Among them, the image gradient-based methods include first-order derivatives Sobel, Robert, Prewitt and second-order derivatives Laplacian, LoG, and the image edge detection algorithms include Canny and SUSAN.

[0075] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

Claims

1. A patrol robot inspection process, characterized in that: The following steps are involved: S1: Controls the camera to execute the preset position during calibration and turns on the autofocus mode; S2: After the autofocus is completed, collect a picture image0 and record the equivalent image distance of the camera at this time as v c ; S3: According to the current v c Non-target objects v during calibration nt Calculate the actual focal length f of the current camera c , the calculation formula is as follows: f c =f+(v c -v nt )*gd(2) In the above formula, f is the focal length of the camera when calibrating the inspection point, and gd is the gradient of the focal length changing with the equivalent image distance, which is obtained by calibrating the camera; S4: Calculate the compensation coefficient of the equivalent image distance difference In the above formula, f is the focal length of the camera when calibrating the inspection point, f c is the actual focal length value calculated in S3; S5: Calculate the equivalent image distance compensation value d v d v =(v t -v nt )*α (4) In the above formula, v t 、v nt are the image distances of the camera with clear target and clear non-target objects when calibrating the inspection point, respectively, and α is the equivalent image distance difference compensation coefficient calculated in S4; S6: Calculate the corrected equivalent image distance v m v m =v t +d v (5) S7: Set the camera equivalent image distance to v m , collect a picture image1; S8: Use the deep learning model to identify images0 and image1 and obtain the region ROI where the target in the image is located; S9: Grayscale the ROIs of the two images and calculate the clarity coefficients δ0 and δ1 of the target areas of the two images. The larger the clarity coefficient, the clearer the image. S10: Save the image with the largest resolution as the final image; t 、v nt Obtain it through the following methods: A1: Adjust the camera focal length f; A2: Turn on the camera's autofocus mode and wait for the focus to complete. The camera image distance is now v. A3: If the target object is clear in the picture, record the target image distance as v t , v t =v; Observe whether the background and foreground are within the same depth of field. If they are, record the equivalent image distance of the non-target object as v nt , v nt =v; If the non-target object is not in the same depth of field, the camera is switched to manual focus mode and the equivalent image distance of the camera is manually adjusted to v1 to make the non-target object clear. The equivalent image distance value of the non-target object is recorded as v nt , v nt =v1; If the target in the picture is not clear, record the non-target object distance as v nt , v nt =v; switch the camera to manual focus mode, manually adjust the camera equivalent image distance to v2, so that the target object is clear, and record the target object equivalent image distance as v t , v t =v2.

2. The inspection process of the inspection robot according to claim 1, characterized in that: In A1, the subject is placed in the center of the camera frame and has an appropriate proportion.

3. The inspection process of the inspection robot according to claim 1, characterized in that: The formula in S3 is based on the equivalent image distance value v c 、v nt Calculate the current real focal length f c , let the target object be A, the non-target object be B, the focal length during calibration be f, and the object distance be u A 、u B , image distance is v A 、v B , the focal length during inspection is f′, and the image distance is v′ A 、v′ B According to the imaging principle, we can get From formula (6), the image distance differences between A and B during calibration and inspection are Δv and Δv′ respectively, as shown in formula (7): In formula (8), represents the ratio of the image distance difference between A and B during calibration and inspection. Its size is only related to the change of focal length. Therefore, the formula of inspection process S4 calculates the compensation coefficient of the camera equivalent image distance difference according to the change of focal length. Combined with the equivalent image distance difference during calibration, the equivalent image distance value of the current target can be calculated by the formulas in inspection processes S5 and S6.

4. The inspection process of the inspection robot according to claim 1, characterized in that: In S8, image0 and image1 are obtained by automatic focusing. When the target is automatically focused, image0 is the clearest picture of the target. If the target is automatically focused on a non-target object, the equivalent image distance of the camera for a clear target object is calculated according to the inspection process S3-S6, and then a picture, i.e., image1, is collected using the fixed focus mode. Then, image1 is a clear picture of the target. At this point, it is determined that one of the two pictures collected in one inspection must have a clear target. The target detection algorithm improved based on YOLO4 can quickly identify the target in the two pictures, and calculate the clarity of the target area after obtaining the target area.

5. The inspection process of the inspection robot according to claim 1, characterized in that: The selection of clarity judgment methods in the inspection process S9 includes image gradient-based methods, image edge detection algorithms, wavelet transform-based methods and statistical clarity judgment methods, among which the image gradient-based methods include first-order derivatives Sobel, Robert, Prewitt and second-order derivatives Laplacian, LoG, and the image edge detection algorithms include Canny and SUSAN.

Images