A target detection method based on template matching algorithm
Through Kinect depth camera acquisition and OpenGL rendering image data, combined with sliding window sampling and local feature matching algorithm, the problems of low recognition rate and poor noise immunity in complex environments are solved, and the problem of traditional three-dimensional object detection is not high in complex environments is realized. Highly accurate object detection.
Patent Information
- Application Number
- CN202311157663.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-08
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-09-08
AI Technical Summary
In complex environments, traditional three-dimensional object detection methods based on template matching have problems such as low recognition rate and poor noise immunity, especially the noise and hollow problems of point cloud data collected by depth cameras.
Kinect depth cameras are used to collect RGB channel information and depth information in real time, pre-process them through combined bilateral filtering and radius filters, generate rendered image data of the three-dimensional model, and train and learn the position estimation algorithm of the target object in OpenGL, combined with Sliding window sampling and local feature matching algorithm improve detection accuracy.
The accuracy and stability of three-dimensional object detection is significantly improved in complex environments, and can effectively identify weak texture objects. It is suitable for robots, automation equipment and smart homes.
Smart Images

Figure CN117197501B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision, and in particular relates to a target detection method based on a template matching algorithm (Linemod). Background Art
[0002] In the field of 3D object detection, traditional methods often require object feature extraction and classifiers to identify objects, but in complex environments, the detection accuracy will be greatly reduced. Therefore, studying object detection methods based on template matching has become an important research direction. However, due to the existence of noise and holes in the point cloud data collected by the depth camera, the traditional object detection method based on template matching still has problems such as low recognition rate and poor noise resistance. Summary of the invention
[0003] The purpose of the present invention is to provide an object detection method based on template matching, which is suitable for target detection of three-dimensional objects with weak textures in complex environments. The present invention uses a Kinect depth camera to collect RGB channel information and depth information in real time, aligns them, and pre-processes the point cloud information by means of joint bilateral filtering, radius filter, etc., to remove empty point cloud points and reduce the number of point clouds. The three-dimensional model of the target object is imported into OpenGL (Open Graphics Library), and a large amount of image data for training and learning of pose estimation algorithms is automatically generated and rendered.
[0004] Specifically, the present invention is achieved through the following technical solutions.
[0005] The target detection method based on the template matching algorithm described in the present invention comprises the following steps:
[0006] (1) Collect point cloud data and color gradient information of the object and extract feature points;
[0007] (2) Generate rendered images of the object at different viewing angles in OpenGL, and calculate the matching threshold of each local feature by traversing the local feature database of feature points to reduce the interference of noise, and finally calculate the matching threshold;
[0008] (3) Based on the template matching algorithm, collect images in the scene, increase local sampling of objects, and extract RGB features of the regional image from the whole through sliding window sampling. Use the template matching algorithm to detect the collected object data, and compare the similarity of the RGB template experimental results at different angles in the database with the experimental matching to judge the accuracy of the detection results;
[0009] (4) The effectiveness of this method is verified by detecting objects in a public database and occluding and rotating the template, and then compared with the traditional object detection method based on template matching.
[0010] The implementation of the present invention can be implemented based on the Kinect V2 depth camera in the Ubuntu 16.04 system, combined with the ROS system and the real-time visualization plug-in Rviz.
[0011] Specifically, the target detection method based on the template matching algorithm described in the present invention comprises the following steps:
[0012] Step 1. Collect the image data of the target, generate rendered images of the target at different viewing angles in OpenGL, extract the gradient features of the RGB image and the normal features of the depth image, retain the direction of the maximum feature, and generate a template library to be matched.
[0013] (1) For a given template library M, extract the view image I and corresponding pose parameters R of each template.
[0014] (2) For each perspective image I, extract d c The local image patch of size p j .
[0015] (3) Traverse all local image blocks and add them to the local image set P.
[0016] (4) For each local image block p in P k , extract its feature vector f k , and add it to the local feature set B.
[0017] Through such an algorithm flow, the local feature set of each template in the template library M can be obtained for use in subsequent target detection tasks.
[0018] Step 2. Use the Kinect camera to collect the RGB image and depth information of the target in the scene, extract the gradient features of the color image, and extract the normal features of the depth image. Use the sliding window matching method to calculate the similarity with the template features in the template library, and determine whether it is the target object based on the maximum value of the similarity calculation.
[0019] (1) First, the target scene is segmented into image blocks to obtain a set of image blocks (p i , T i ), where p i represents the image block, T i Indicates the corresponding label.
[0020] (2) For each image block p i , extract its feature vector fi , use the Encoder function to encode the image block p i Converted into its eigenvector f i .
[0021] (3) For each eigenvector f i , traverse k feature vectors f m , if f i and f m If the similarity between them is greater than the preset threshold s, they are considered to represent the same target.
[0022] (4) For each image patch p that is considered to be the same target i and p m , label them as T i and T m As a data set, use the classification algorithm to calculate its similarity V r .
[0023] (5) For all local image blocks, threshold judgment is performed. If the similarity V of a local image block is r Greater than the preset threshold t s , then its feature vector h r Add it to the set of matching results H{}. Finally, calculate the matching result based on the feature vectors in the set H{}.
[0024] The present invention uses a template matching-based algorithm to process the information collected by the sensor and compare it with the data in the database. It uses a local sampling and partition voting mechanism to improve the detection algorithm, improve stability and recognition accuracy in an obstructed environment. It has a wide range of application prospects and can be applied not only to robots, automation equipment and other fields, but also to smart homes, smart phones and other fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is the overall working flow chart of the present invention.
[0026] Figure 2 Some template images are collected for the present invention.
[0027] Figure 3 This is a diagram of the target detection experiment results of the present invention.
[0028] Figure 4 The target workpieces used in the template matching comparison experiment. Among them, (a) is the target workpiece of template one, (b) is the target workpiece of template two, and (c) is the target workpiece of template three. DETAILED DESCRIPTION
[0029] The present invention will be further described through the following embodiments with reference to the accompanying drawings.
[0030] Example 1.
[0031] This embodiment uses an experimental platform based on a Kinect v2 depth camera and an Ubuntu 16.04 operating system. On a host equipped with an Intel i7 eight-core processor, combined with the ROS system and the real-time visualization plug-in Rviz, the effectiveness of the algorithm is verified. The experiment collects point cloud data and RGB images of the target object on the workbench, and uses the template matching algorithm proposed in the present invention for target detection. The target of the experiment is a flange commonly used in industrial production. The collected data is compared with the RGB and point cloud templates at different angles in the database.
[0032] The specific steps are as follows:
[0033] (1) This embodiment uses CAD software to build a three-dimensional model of the flange, and samples the object by setting the viewpoint of the virtual camera on the spherical surface, and collects templates at different viewing angles. In order to ensure the uniform distribution of the sampling points on the spherical surface, the sampling angle interval of each change is set to 20°, and the RGB image data of the flange three-dimensional model at different viewing angles is collected. OpenGL is used to generate the rendered images of the flange at different viewing angles. The collected partial images are shown in Figure 1. Figure 2 shown.
[0034] (2) 563 template images of flanges at different viewing angles were collected, and the images of each viewing angle were divided into 64 image blocks of the same size. For each local image block, the gradient was calculated by using the Sobel operator, and then the gradient direction and amplitude were solved at each pixel point. The corresponding threshold data was set and the feature points exceeding the threshold were added to the local feature set. In this way, the local feature set of each template in the flange template library was obtained, which was used in the subsequent flange target detection task.
[0035] (3) Place the flange on the experimental table, use the Kinect V2 depth camera to collect the RGB image of the flange in the experimental scene, segment the scene image into 8×8, obtain a set of 64 images, and record the corresponding labels. Then use the Sobel operator to calculate the gradient of each image block, solve the gradient direction and amplitude at each pixel point, and retain the feature points that exceed the threshold and the amplitude corresponding to the feature points.
[0036] (4) Traverse the local feature set of each template in the template library and search for matches in the scene image block using a sliding window.
[0037] For each matching result, its similarity score is calculated. If the score is higher than the threshold, the matching result is considered to be part of the flange image. When the matching image block exceeds 90% of the template image, it is determined that the target flange exists in the scene.
[0038] The similarity of experimental matching is as follows Figure 3 The experimental results show that the object detection method based on template matching of the present invention can be effectively applied to the field of three-dimensional object detection and has good practicality and feasibility.
[0039] Example 2.
[0040] Comparison of the effects of template sampling methods based on template matching and adding local matching.
[0041] In order to more intuitively compare the effects of the template sampling method based on template matching and the template sampling method with local matching, this embodiment conducted three groups of experiments and recorded the relevant matching thresholds. Figure 4 As shown in the figure, the experiment selected flange, tee pipe and transmission gear as the recognition targets. In each group of experiments, we selected different templates and conducted template matching and template matching experiments with increased local sampling. At the same time, in order to simulate the situation where local sampling information is missing, the template image is blocked. The experimental platform of the Ubuntu 16.04 operating system uses the template matching algorithm and the template with increased local matching to calculate the similarity of the images before and after the occlusion. Table 1 shows the algorithm flow of target detection based on the template matching algorithm. Table 2 shows the algorithm flow of target detection with increased local sampling.
[0042] Table 1 Algorithm flow of target detection based on template matching algorithm
[0043]
[0044] Table 2 Adding local sampling target detection algorithm flow
[0045]
[0046] The thresholds of the experimental results are shown in Table 3. When the image of the target object is partially occluded, the matching similarity of the improved algorithm of the present invention is higher than that of the algorithm before the improvement. This shows that the template matching algorithm with local sampling performs better in the target detection experiment and has certain effectiveness.
[0047] Table 3 Matching thresholds of targets under different sampling methods
[0048]
Claims
1. A target detection method based on a template matching algorithm, characterized in that The following steps are involved: Step 1. Collect the image data of the target, generate rendered images of the target at different viewing angles in OpenGL, extract the gradient features of the RGB image and the normal features of the depth image, retain the direction of the maximum feature, and generate a template library to be matched; (1) For a given template library M, extract the view image I and corresponding pose parameters R of each template; (2) For each perspective image I, extract d c The local image patch of size p j ; (3) Traverse all local image blocks and add them to the local image set P; (4) For each local image block p in P k , extract its feature vector f k , and add it to the local feature set B; Obtain the local feature set of each template in the template library M; Step 2. Use the Kinect camera to collect the RGB image and depth information of the target in the scene, extract the gradient features of the color image, and extract the normal features of the depth image; use the sliding window matching method to calculate the similarity with the template features in the template library, and determine whether it is the target object based on the maximum value of the similarity calculation: (1) First, the target scene is segmented into image blocks to obtain a set of image blocks (p i , T i ), where p i represents the image block, T i Indicates the corresponding label; (2) For each image block p i , extract its feature vector f i , use the Encoder function to encode the image block p i Converted into its eigenvector f i ; (3) For each eigenvector f i , traverse k feature vectors f m , if f i and f m If the similarity between them is greater than the preset threshold s, they are considered to represent the same target; (4) For each image patch p that is considered to be the same target i and p m , label them as T i and T m As a data set, use the classification algorithm to calculate its similarity V r ; (5) For all local image blocks, threshold judgment is performed. If the similarity V of a local image block r Greater than the preset threshold t s , then its feature vector h r Add it to the set H{} of matching results; finally, calculate the matching result based on the feature vectors in the set H{}.
