Target positioning method and device based on image matching

Abstract

The application discloses a target positioning method and device based on image matching, which comprises the following steps: acquiring a patrol image, reading information of a template region in a template image, performing region matching positioning on the patrol image to determine a potential target region in the patrol image, using a shape matching-based target positioning algorithm, using an HSV color space-based brightness normalization algorithm to normalize the brightness of the potential target region by using the brightness of the template region in the template image, and using a template matching positioning algorithm to perform matching positioning on the potential target region after brightness normalization to determine the accurate position of the target in the patrol image. The application is suitable for outdoor robot patrol scenes in environments with a large number of electric poles, iron towers and transformers, and can solve the problem of target positioning in the case of target detail loss and target local change.

Description

Technical Field

[0001] This invention belongs to the field of positioning technology, specifically relating to a target positioning method and apparatus based on image matching. Background Technology

[0002] Currently, outdoor inspection robots that use gimbals as actuators rely on visible light image matching technology to locate inspection targets, guide the gimbal to adjust to the target's angle, and then capture clear images of the target for identification and detection purposes. This inspection process requires the algorithm to quickly and accurately locate the target in the captured image.

[0003] Current methods for target localization primarily employ visible light target matching. This involves first acquiring a target template, recording the corresponding gimbal angle, camera focal length, and magnification. During inspection, the gimbal moves to the corresponding angle, and an image is taken. The image captured during inspection is then compared to the template image, and image registration and matching methods are used to locate the target. In practice, the robot's task duration is not fixed. If it's in low-light conditions such as rainy days or evening, the resulting task image (inspection image) will be unclear, making it difficult to determine the knife switch target on the task image based on the target outlined in the template image.

[0004] Existing target localization technologies mainly employ image matching methods, including template matching, feature matching, and deep learning-based target detection matching methods. However, none of these methods can simultaneously meet the requirements of resistance to illumination interference, low computational cost, and accurate matching results. Summary of the Invention

[0005] Technical objective: To address the aforementioned technical problems, this invention proposes a target localization method and apparatus based on image matching, which can solve the target localization problem under conditions of target detail loss and target local changes.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A target localization method based on image matching, characterized by comprising the following steps:

[0008] Coarsely locate the target range: acquire the inspection image, use a target localization algorithm based on shape matching, read the information of the template region in the template image, perform region matching localization on the inspection image, and determine the potential target region in the inspection image; the template image is an image containing the target that was acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image;

[0009] Normalized template region brightness: A brightness normalization algorithm based on the HSV color space is used to normalize the brightness of the target potential region using the brightness of the template region in the template image;

[0010] Precise target location: The template matching localization algorithm is used to match and locate the target potential area after brightness normalization, determine the target position in the potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image.

[0011] Preferably, the step of determining the potential region of a target in the inspection image includes:

[0012] The linemod-2D algorithm is used to extract gradient features from the template region in the template image.

[0013] In the inspection images, the sliding window method is used to find regions similar to the template region as potential target regions.

[0014] Preferably, the method includes the following steps:

[0015] A rectangular region marking the target location is used to extract linmod-2D template features;

[0016] In the upper layer of the image pyramid, the linemod-2d algorithm based on quantization direction as the matching feature is used for coarse template matching;

[0017] Subpixel correction was performed on edge points in the linemod-2d template features using orthogonal Fourier-Marin moments.

[0018] Preferably, the step of performing brightness normalization processing on the potential target region includes:

[0019] Convert the template image and the inspection image to the HSV color space, extract the mean value of the V component of the template region, i.e. the first mean value v1, extract the mean value of the V component of the target potential region in the inspection image, i.e. the second mean value v2, and calculate the ratio of the first mean value and the second mean value s = v1 / v2.

[0020] The V component is calculated pixel by pixel in the potential target region and multiplied by the scale s to normalize the brightness of the potential target region to be consistent with that in the template region.

[0021] Preferably, precise target positioning includes the following steps:

[0022] In the target potential region after normalized brightness processing, the normalized correlation coefficient matching method is used to perform target template matching to obtain a rectangular box for precise target localization.

[0023] Take the center point of the rectangle as the target location;

[0024] The target location is mapped onto the inspection image to obtain the target's precise location on the entire inspection image.

[0025] Preferably, the target is an outdoor electrical disconnect switch.

[0026] A target localization device based on image matching, characterized in that it comprises:

[0027] The coarse target range positioning module is used to acquire inspection images. It adopts a target positioning algorithm based on shape matching, reads the information of the template region in the template image, performs region matching positioning on the inspection image, and determines the potential target region in the inspection image. The template image is an image containing the target that is acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image.

[0028] The normalized template region brightness module is used to process the brightness of the target potential region by using a brightness normalization algorithm based on the HSV color space and normalizing the brightness of the template region in the template image.

[0029] The precise target positioning module is used to match and locate the target potential area after brightness normalization using a template matching positioning algorithm, determine the target position in the target potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image.

[0030] Preferably, it further includes: a template image acquisition module, used to acquire a template image of the target, wherein the template image includes a template region.

[0031] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0032] The method of this invention is applicable to outdoor robot inspection scenarios in environments with a large number of utility poles, towers and transformers. A typical application scenario is equipment status inspection in substations. It can significantly improve the accuracy of matching and positioning of outdoor electrical disconnectors under complex lighting conditions, while taking into account the characteristics of lighting robustness, low computing power consumption and accurate target positioning results. Attached Figure Description

[0033] Figure 1 This is a flowchart of the target localization method based on image matching of the present invention;

[0034] Figure 2 Template images and inspection images of electrical disconnect switches under changing lighting conditions;

[0035] Figure 3 To determine based on shape matching algorithm Figure 1 A schematic diagram of the potential target area in the mid-level inspection image;

[0036] Figure 4 for Figure 2 A comparison of the brightness of the potential target region before and after normalization processing;

[0037] Figure 5 This is a schematic diagram of the target location results in the final inspection image;

[0038] Figure 6 This is a schematic diagram of the structure of an ideal two-dimensional image edge model. Detailed Implementation

[0039] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0040] The purpose of this invention is to improve a shape-matching-based target localization algorithm, and then integrate this algorithm with a brightness normalization algorithm and a template matching algorithm to achieve target localization of electrical disconnectors under conditions of large illumination variations. Specifically, the improved shape-matching-based target localization algorithm is used to coarsely locate the potential area of ​​the disconnector target in an inspection image. This method, based on shape matching, can accurately locate the potential target area under conditions of strong light, weak light, backlight, and when the target is in shadow. Then, a brightness normalization algorithm based on the HSV color space is used to adjust the brightness of the potential target area in the inspection image to match the brightness of the template area in the template image. Finally, the template matching localization method is used to obtain the precise location of the target.

[0041] Example 1

[0042] This embodiment provides a target localization method based on image matching, including the following steps:

[0043] Coarsely locate the target range: acquire the inspection image, use a target localization algorithm based on shape matching, read the information of the template region in the template image, perform region matching localization on the inspection image, and determine the potential target region in the inspection image; the template image is an image containing the target that was acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image;

[0044] Normalized template region brightness: A brightness normalization algorithm based on the HSV color space is used to normalize the brightness of the target potential region using the brightness of the template region in the template image;

[0045] Precise target location: The template matching localization algorithm is used to match and locate the target potential area after brightness normalization, determine the target position in the potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image.

[0046] Specifically, the steps for identifying potential regions of a target in an inspection image include:

[0047] The linemod-2D algorithm is used to extract gradient features from the template region in the template image.

[0048] Then, the sliding window method is used in the inspection image to find regions similar to the template region as potential template regions for the target.

[0049] The method of the present invention will be described in detail below using an electrical disconnect switch for outdoor high-altitude operations as an example.

[0050] This refers to a target positioning method for electrical disconnect switches based on shape matching, implemented according to the following steps:

[0051] Step 1: First, select a fixed-size region containing the target from the pre-acquired template image as the template region. Use the linemod-2D algorithm to extract the gradient features of the entire image of the template region. Here, the feature extraction method is modified so that the distribution of the image gradient information extracted by the linemod-2D algorithm is more inclined to the target edge. Then, use the sliding window method to find regions similar to the template image on the inspection image to complete the target potential region localization task.

[0052] A template image can be considered a pre-taken reference image. The portion outlined above the target in the template image is the target that needs to be located on the inspection map. Generally, template images are acquired under well-lit conditions, and the target equipment must be clearly visible in the template image. Figure 2 The target device in the example diagram on the left-hand side is a disconnect switch.

[0053] Step 2: The target brightness normalization module converts the template image and the inspection image to the HSV color space. It extracts the mean V component value v1 from the template region and the mean V component value v2 from the potential template region in Step 1, calculating their ratio s = v1 / v2. Then, it calculates the v component for each pixel in the potential target region, multiplies it by s, and normalizes the brightness of the potential target region to match that of the template region. The HSV color space model is a descriptive model for images, where H represents hue, S represents saturation, and V represents brightness. V describes the brightness of the image's colors. Normalizing brightness within the potential target region effectively reduces computational load.

[0054] Step 3: Precise positioning module for disconnect switch target. After the above two steps, the uniqueness of the disconnect switch target in the potential target area can be basically guaranteed. The target position can be accurately located by using template matching.

[0055] Specifically, step 1 is implemented according to the following steps:

[0056] Step 1.1: Extract Linemod-2D template features from the rectangular region marked with the target location on the template image. Linemod-2D is an image matching algorithm that uses the gradient direction of the target's edge points as matching features. In this embodiment, a Linemod-2D algorithm based on quantization direction as matching features is used for coarse template matching in the upper layer of the image pyramid. Then, orthogonal Fourier-Marin moments are used to perform sub-pixel correction on the edge points in the Linemod-2D template. This ensures both the matching speed and the matching accuracy of the image edges. An image pyramid is a multi-scale representation of an image. An image pyramid is actually a collection of images at different scales; the bottom of the pyramid is a high-resolution image, while the top is a low-resolution image.

[0057] Orthogonal Fourier-Marin Moment (OFMM):

[0058] Figure 6 This is an ideal two-dimensional image edge model:

[0059] Where h1 and h2 represent the grayscale background value and target grayscale value of hi, respectively, l is the normalized distance from the actual edge point to the origin, and θ is the angle between the edge normal and the x-axis.

[0060] The OFMM definition of the order of a second-order continuous function is:

[0061]

[0062] In the formula, Let f denote the moment, p denote the order, q denote the number of iterations, the function f is defined in the x,y plane, and α denote a normalized constant.

[0063] Formula for calculating edge model parameters:

[0064]

[0065] In the formula, k is the contrast between the target grayscale and the background grayscale, Im represents the imaginary part of the complex number, and Re represents the real part of the complex number.

[0066] For the content of second-order continuous functions and edge model parameters, please refer to reference [1], Dang Hongshe, Hu Zunfeng, Fang Xin. Improved sub-pixel edge detection algorithm based on orthogonal Fourier-Mellin moments [J]. Electronic Technology Application, 2009, 35(02): 125-127+130.

[0067] Step 1.2: After coarse positioning in Step 1.1, the potential area of ​​the knife switch target on the inspection image can be obtained. The potential area and the template area of ​​the template image are extracted and converted to the HSV color space.

[0068] Specifically, step 2 is carried out according to the following steps:

[0069] Step 2: Calculate the mean V component value v1 of the template region and the mean V component value v2 of the potential template region in Step 1.2, calculate the ratio between the two s = v1 / v2, then calculate the v component for each pixel of the target potential region, multiply it by s, and normalize the brightness of the target potential region to be consistent with that of the template region.

[0070] Specifically, step 3 is carried out according to the following steps:

[0071] Step 3.1: After normalizing the brightness in step 2, the target potential area is matched with the knife switch target template using the normalized correlation coefficient matching method to obtain a rectangular box for precise target positioning.

[0072] Step 3.2: Take the center point of the rectangle from Step 3.1 as the target position.

[0073] Step 3.3: Map the target location onto the inspection image to obtain the target's precise location on the entire inspection image.

[0074] This embodiment first designs an improved image matching and localization method based on the linemod-2d algorithm. This method can roughly locate the target position even when the target details are lost under uneven lighting and insufficient lighting conditions. Then, an image brightness normalization algorithm based on the HSV color space is used to normalize the brightness of the template region roughly located in the previous step, so that it is consistent with the brightness of the template region of the template image. Finally, an image template matching algorithm is applied within the template region to accurately locate the target position.

[0075] Example 2

[0076] This embodiment provides a target localization device based on image matching, including:

[0077] The coarse target range positioning module is used to acquire inspection images. It adopts a target positioning algorithm based on shape matching, reads the information of the template region in the template image, performs region matching positioning on the inspection image, and determines the potential target region in the inspection image. The template image is an image containing the target that is acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image.

[0078] The normalized template region brightness module is used to process the brightness of the target potential region by using a brightness normalization algorithm based on the HSV color space and normalizing the brightness of the template region in the template image.

[0079] The precise target positioning module is used to match and locate the target potential area after brightness normalization using a template matching positioning algorithm, determine the target position in the target potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image.

[0080] Preferably, it further includes: a template image acquisition module, used to acquire a template image of the target, wherein the template image includes a template region.

[0081] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A target localization method based on image matching, characterized in that, Including the following steps: Coarsely locate the target range: acquire the inspection image, use a target localization algorithm based on shape matching, read the information of the template region in the template image, perform region matching localization on the inspection image, and determine the potential target region in the inspection image; the template image is an image containing the target that was acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image; Normalized template region brightness: A brightness normalization algorithm based on the HSV color space is used to normalize the brightness of the target potential region using the brightness of the template region in the template image; Precise target location: The template matching localization algorithm is used to match and locate the target potential area after brightness normalization, determine the target position in the potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image. The steps for brightness normalization of the potential target region include: Convert the template image and the inspection image to the HSV color space, extract the mean V component of the template region, i.e. the first mean v1, extract the mean V component of the target potential region in the inspection image, i.e. the second mean v2, and calculate the ratio of the first mean and the second mean s=v1 / v2. The V component is calculated pixel by pixel in the potential target region and multiplied by the scale s to normalize the brightness of the potential target region to be consistent with that in the template region.

2. The target localization method based on image matching according to claim 1, characterized in that, The steps for identifying potential regions of targets in inspection images include: The linemod-2D algorithm is used to extract gradient features from the template region in the template image. In the inspection images, the sliding window method is used to find regions similar to the template region as potential target regions.

3. The target localization method based on image matching according to claim 2, characterized in that, Including the following steps: A rectangular region marking the target location is used to extract linmod-2D template features; In the upper layer of the image pyramid, the linemod-2d algorithm based on quantization direction as the matching feature is used for coarse template matching; Subpixel correction was performed on edge points in the linemod-2d template features using orthogonal Fourier-Marin moments.

4. The target localization method based on image matching according to claim 1, characterized in that, Precise target location includes the following steps: In the target potential region after normalized brightness processing, the normalized correlation coefficient matching method is used to perform target template matching to obtain a rectangular box for precise target localization. Take the center point of the rectangle as the target location; The target location is mapped onto the inspection image to obtain the target's precise location on the entire inspection image.

5. The target localization method based on image matching according to claim 1, characterized in that, The target is an outdoor electrical disconnect switch.

6. A target localization device based on image matching, performing the target localization method of claim 1, characterized in that, The device includes: The coarse target range positioning module is used to acquire inspection images. It adopts a target positioning algorithm based on shape matching, reads the information of the template region in the template image, performs region matching positioning on the inspection image, and determines the potential target region in the inspection image. The template image is an image containing the target that is acquired in advance under sufficient lighting conditions, and the template region is the region containing the target determined on the template image. The normalized template region brightness module is used to process the brightness of the target potential region by using a brightness normalization algorithm based on the HSV color space and normalizing the brightness of the template region in the template image. The precise target positioning module is used to match and locate the target potential area after brightness normalization using a template matching positioning algorithm, determine the target position in the target potential area, and map the target position to the inspection image to obtain the precise position of the target in the inspection image.

7. A target localization device based on image matching according to claim 6, characterized in that, Also includes: The template image acquisition module is used to acquire the template image of the target, which includes the template region.

Images