A visual positioning method based on triangular shape feature descriptors for visual cameras to overcome illumination invariance

Abstract

The present invention discloses a visual positioning method for a triangular shape feature descriptor for a visual camera to overcome illumination invariance. In order to overcome the huge challenge posed by complex and changeable environmental lighting to the image information obtained by the camera, which leads to poor accuracy of feature information extracted by the camera and subsequent camera positioning error accumulation problems. The method first performs histogram equalization on the input image data so that the image can highlight the texture features of local edges. Feature points are then extracted from the input current frame and the feature vectors of the feature points are calculated, and the strong edge feature area and the weak edge feature area are divided according to the weights of the feature vectors. Finally, the three optimal feature points of the strong edge feature area are used to construct a triangular shape feature descriptor. The constructed triangular shape feature descriptor belongs to the geometric attributes of the image, consists of length and angle, is insensitive to illumination changes, and can overcome the influence of the time-varying illumination environment.

Description

Technical Field

[0001] The present invention relates to the technical field of simultaneous localization and mapping (SLAM) of visual sensors, and in particular to a visual localization method of a triangular shape feature descriptor for a visual camera to overcome illumination invariance. Background Art

[0002] With the continuous advancement of modern science and technology, SLAM technology based on visual sensors is one of the key technologies for achieving intelligent and autonomous unmanned systems, and is particularly widely used in the field of robotics. Over the past few decades of SLAM technology development, due to its ability to acquire rich texture information and its unlimited potential for exploration, it has been applied to unmanned systems as a solution to positioning problems. This has not only achieved extensive theoretical breakthroughs but also promoted the rapid development of many application areas. By using visual sensors to acquire unknown environmental information, it is possible to perceive environmental maps and determine the camera's own position and posture information in the environment. However, visual camera sensors are also susceptible to time-varying ambient lighting. Therefore, it is extremely important to develop an environmental information acquisition method that can overcome the effects of lighting changes.

[0003] Visual cameras estimate their own motion and pose by acquiring real-time 3D information about their environment. Currently, mainstream visual SLAM methods can be categorized as direct methods and feature point methods. Direct methods do not require the camera to acquire and match texture features of the environment. Instead, they directly compare the grayscale differences between pixels in two adjacent image frames and use a grayscale error function to estimate and optimize the camera pose. Because direct methods do not require the two stages of feature extraction and matching, they acquire environmental information more quickly. However, due to their assumption of grayscale consistency, they can easily lose the camera's position in time-varying lighting environments. Compared to direct methods, feature point-based methods are the front-end of visual SLAM. These methods acquire 3D information of the environment through a camera and then convert it into a 2D image, extracting texture-rich feature points from the image. Data association is performed through the relationship between two adjacent frames, and the camera sensor pose is calculated. To overcome the effects of varying illumination, triangular feature descriptors are constructed from the acquired feature points. Descriptors constructed from feature points in regions with strong edge features exhibit strong illumination invariance.

[0004] The present invention proposes a triangular shape feature extraction and description method for a visual camera to overcome illumination invariance. In order to overcome the huge challenges brought by complex and changeable environmental lighting to the image information obtained by the camera, which leads to poor accuracy of the feature information extracted by the camera and the accumulation of positioning errors in subsequent cameras. The method first performs histogram equalization on the input image data so that the image can highlight the texture features of the local edges. Then, the feature points of the input current frame are extracted and the feature vectors of the feature points are calculated, and the strong edge feature area and the weak edge feature area are divided according to the weight of the feature vector. Finally, the three optimal feature points of the strong edge feature area are used to construct a triangular shape feature descriptor. The constructed triangular shape feature descriptor belongs to the geometric attributes of the image, consists of length and angle, is insensitive to illumination changes, and can overcome the influence of the time-varying illumination environment. Summary of the Invention

[0005] This paper discloses a triangular feature extraction and description method for visual cameras to overcome illumination invariance. The method aims to extract the optimal three feature points from a region with strong edge features, thereby constructing a triangular feature descriptor. The triangular feature descriptor consists of geometric information connecting two feature points, and is highly resistant to illumination changes.

[0006] The technical solution of the present invention is:

[0007] A method for extracting and describing triangular shape features for a visual camera to overcome illumination invariance, characterized by comprising the following steps:

[0008] Step 1: Input a frame of RGB image, perform histogram equalization on the input RGB image of the current frame, and highlight the local edge texture features of the image.

[0009] Step 2: According to the distribution of local edge texture of the image, feature points are extracted from the area with obvious local edge texture information.

[0010] Step 3: In the current frame, set the total number of extracted feature points to P sum , the feature vector weight W of each feature point i Calculate the eigenvector of each feature point and determine whether the weight of the eigenvector of each feature point is greater than the mean weight Among them, the mean weight If the weight of the eigenvector is greater than the mean weight And the adjacent feature points must be located within a circle with a radius of 5 pixels, then the area is called a strong edge feature area. Otherwise, the other areas are weak edge feature areas.

[0011] Step 4: Considering that the feature points of the strong edge feature area of the image are more stable and less sensitive to changes in illumination compared to the feature points of the weak edge feature area, the feature vectors of the feature points of the strong edge feature area are calculated and the position and size of the feature vectors are added to the feature vector list. In the equation, add the direction angle of the corresponding eigenvector to list.

[0012] Step 5: Calculate the feature vector list of the current strong edge feature area The distance between two adjacent feature points in , and arranged from small to large. At the same time, calculate The direction angle between two adjacent feature points in the , and arranged from small to large.

[0013] Step 6: Obtain the optimal three feature points P in the current strong edge feature area through the median method i 、P j 、P m , Similarly, repeat the above steps 4 and 5 to obtain the optimal three feature points of other strong edge feature areas.

[0014] Step 7: Use the three feature points in the strong edge feature area to construct a triangular shape feature descriptor, and obtain the feature point pixels P respectively. i With P m The gradient direction is and Feature point pixel P j With P m The gradient direction and Feature point pixel P i With P j The gradient direction is θ i and θ j .

[0015] Step 8: Set the feature point pixel P i 、P j 、P m The two points are connected and combined to form a triangular shape descriptor. The constructed triangular shape feature descriptor is shown in the following formula (1). Finally, the feature points of the constructed triangular shape descriptor are used to solve the camera pose change through the rotation matrix and translation vector, avoiding the problem of using feature point information in areas that are easily affected by lighting changes to solve the camera pose, which would increase the positioning error.

[0016]

[0017] Preferably, in step 1, the local edge texture features refer to regions in the image where pixel grayscale differences vary significantly. These regions are rich in texture information and are insensitive to illumination changes.

[0018] Preferably, in step 2, the feature point extraction process is a corner point detection, and if an algorithm determines that a pixel has a large difference from adjacent pixels and satisfies the algorithm setting parameters, the pixel is determined to be a feature point.

[0019] Beneficial effects of the present invention: By generating strong edge feature areas, the present invention can make the feature points extracted by the algorithm more representative and avoid wasting a lot of time on useless weak edge feature areas; compared with the direct method of obtaining image information, the improved method of triangular shape feature descriptors based on feature points has superior anti-light performance. At the same time, the algorithm can quickly and stably adapt to complex environments with unstable lighting indoors or outdoors. The present invention can also be used in the fields of robot navigation, 3D modeling, unmanned autonomous driving, augmented reality / virtual reality, smart home, epidemic prevention and disinfection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly present the technical solutions in the embodiments of the present invention, the following briefly and in detail introduces the drawings required for describing the embodiments.

[0021] Figure 1 This is an overall block diagram of a triangular shape feature extraction and description method for visual cameras to overcome illumination invariance;

[0022] Figure 2 Flowchart for enhancing the input RGB image;

[0023] Figure 3 Schematic diagram of feature point extraction;

[0024] Figure 4 Schematic diagram of the strong edge feature area;

[0025] Figure 5 Schematic diagram for constructing a triangular shape feature descriptor;

[0026] Figure 6 .Schematic diagram of a wheeled mobile robot. DETAILED DESCRIPTION

[0027] In order to provide a further understanding and recognition of the method of the present invention, the technical solutions in the embodiments of the present invention will be briefly and detailedly described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0028] Example 1

[0029] See also Figure 1 As shown, the present invention provides a visual positioning method for triangular shape feature descriptors used by a visual camera to overcome illumination invariance, comprising the following steps:

[0030] See also Figure 2 As shown, in step 1, a frame of RGB image is input, and histogram equalization is performed on the input RGB image of the current frame to highlight the local edge texture features of the image.

[0031] Specifically, the size of the input RGB image is 752×480, the RGB image is grayscaled, and then histogram equalization is performed to achieve the purpose of a clear image.

[0032] See also Figure 3 As shown, in step 2, according to the distribution of local edge texture of the image, feature points are extracted from the area with obvious local edge texture information of the image.

[0033] Specifically, in areas with rich image textures, more feature points are extracted, and the texture feature information in this area is insensitive to illumination changes.

[0034] See also Figure 4 As shown, step 3, in the current frame, set the total number of extracted feature points to P sum , the feature vector weight W of each feature point i Calculate the eigenvector of each feature point and determine whether the weight of the eigenvector of each feature point is greater than the mean weight Among them, the mean weight If the weight of the eigenvector is greater than the mean weight And the adjacent feature points must be located within a circle with a radius of 5 pixels, then the area is called a strong edge feature area. Otherwise, the other areas are weak edge feature areas.

[0035] Specifically, the feature vector of a feature point is composed of the magnitude of the vector and the angle of the vector. Different feature points have different feature vectors, and their assigned feature weights are also different.

[0036] Step 4: Considering that the feature points of the strong edge feature area of the image are more stable and less sensitive to changes in illumination compared to the feature points of the weak edge feature area, the feature vectors of the feature points of the strong edge feature area are calculated and the position and size of the feature vectors are added to the feature vector list. In the equation, add the direction angle of the corresponding eigenvector to list.

[0037] Step 5: Calculate the feature vector list of the current strong edge feature area The distance between two adjacent feature points in , and arranged from small to large. At the same time, calculate The direction angle between two adjacent feature points in the , and arranged from small to large.

[0038] Step 6: Obtain the optimal three feature points P in the current strong edge feature area through the median method i 、P j 、P m , Similarly, repeat the above steps 4 and 5 to obtain the optimal three feature points of other strong edge feature areas.

[0039] See also Figure 5 As shown, step seven, using the three feature points in the strong edge feature area to construct a triangular shape feature descriptor, respectively obtain the feature point pixel P i With P m The gradient direction is and Feature point pixel P j With P m The gradient direction and Feature point pixel P i With P j The gradient direction is θ i and θ j .

[0040] Specifically, a triangular shape feature descriptor is constructed by using the three optimal feature points in the strong edge feature area. The descriptor is composed of the angles and sides of the three different feature points and has strong resistance to illumination changes.

[0041] Step 8: Set the feature point pixel P i 、P j 、P m The two points are connected and combined to form a triangular shape descriptor. The constructed triangular shape feature descriptor is shown in the following formula (1). Finally, the feature points of the constructed triangular shape descriptor are used to solve the camera pose change through the rotation matrix and translation vector, avoiding the problem of using feature point information in areas that are easily affected by lighting changes to solve the camera pose, which would increase the positioning error.

[0042]

[0043] Example 2: Application Case

[0044] This paper proposes a triangular shape feature extraction and description method for visual cameras that overcomes illumination invariance, which has applications in robotics. In this field, for robots to perform intelligent and autonomous movements in unknown environments, they must obtain information about the surrounding scene and determine their own position within it. Time-varying illumination also poses a significant challenge. Therefore, robots can use visual sensors installed or carried by the robot to map its surroundings and estimate its own position.

[0045] First, a wheeled mobile robot is designed and the visual camera sensor is installed on the top of the wheeled mobile robot. Figure 6 As shown in Figure 2, the mobile robot collects information about the environment in real time by carrying a camera sensor, and transmits the collected RGB image information to the computer for processing.

[0046] The wheeled mobile robot is primarily composed of a chassis, left and right differential wheels mounted on the chassis, an encoder, a control board, and a main control board. Furthermore, a microcomputer is installed within the chassis. When a camera acquires real-time information about the external environment, the data is transmitted to the microcomputer for processing.

[0047] The microcomputer then preprocesses the acquired RGB image data, primarily by grayscaling the RGB image and performing histogram equalization, effectively reducing the impact of ambient noise. An angle detection function is used to extract feature point information and construct strong edge feature regions. Furthermore, the optimal three feature points for each strong edge feature region in the current frame are extracted, and a triangular shape feature descriptor is constructed using a descriptor constructor. This enables accurate environmental feature extraction in time-varying lighting environments, laying the foundation for subsequent mapping and navigation tasks for the mobile robot.

[0048] Finally, the present invention is compared through test experiments, and the results of the comparison are shown in Table 1. In the experimental test, the image data information obtained by the visual camera is first transmitted to the computer processing center, and then different algorithms are called and tested through modular functions, and finally the different data obtained by the different comparison algorithms are printed. In Table 1, the experiment sets up the ORB and Shi-Tomasi mainstream methods based on point features for comparison to demonstrate the effectiveness of the proposed factors. As can be seen from Table 1, APE is the absolute trajectory error, which is used to measure the error size of the visual camera positioning. The errors of the visual camera in the x-axis, y-axis, and z-axis directions are recorded respectively. It can be seen that in the x-, y-, and z-axis directions, the APE errors of the method of the present invention are smaller than those of the other two methods. This is because the visual camera extracts the information of the feature points in the strong edge feature area for posture estimation, which is conducive to overcoming the influence of variable lighting.

[0049] Table 1 APE comparison of different algorithms (m)

[0050] x y z ORB 0.513 0.297 0.726 Shi-Tomasi 0.652 0.338 0.753 The present invention 0.409 0.254 0.681

Claims

1. A visual positioning method for triangular shape feature descriptors used for visual cameras to overcome illumination invariance, characterized in that: The steps include: Step 1: Input a frame of RGB image, perform histogram equalization on the input RGB image of the current frame, and highlight the local edge texture features of the image; Step 2: Extract feature points from the region with obvious local edge texture information based on the distribution of local edge texture of the image; Step 3: In the current frame, set the total number of extracted feature points to , the feature vector weight of each feature point ; Calculate the eigenvector of each feature point and determine whether the weight of the eigenvector of each feature point is greater than the mean weight ; Among them, the mean weight ; If the weight of the feature vector is greater than the mean weight , and the adjacent feature points must be located within a circle with a radius of 5 pixels, then the area is called a strong edge feature area; otherwise, the other areas are weak edge feature areas; Step 4: Calculate the eigenvectors of the feature points in the strong edge feature area and add the position and size of the eigenvectors to the eigenvector list In the list, add the direction angle of the corresponding eigenvector to the list middle; Step 5: Calculate the feature vector list of the current strong edge feature area The distance between two adjacent feature points in the array is arranged from small to large. At the same time, the calculation The direction angle between two adjacent feature points in the image is arranged from small to large; Step 6: Obtain the best three feature points in the current strong edge feature area through the median method , Similarly, repeat the above steps 4 and 5 to obtain the optimal three feature points of other strong edge feature areas; Step 7: Use the three feature points in the strong edge feature area to construct a triangular shape feature descriptor and obtain the feature point pixels respectively. and The gradient direction is and , feature point pixels and The gradient direction and , feature point pixels and The gradient direction is and ; Step 8: Pixel the feature points The two are connected and combined to form a triangular shape descriptor. The constructed triangular shape feature descriptor is shown in formula (1); ， in, Pixel To pixel distance, Pixel To pixel distance, Pixel To pixel distance; The feature points of the triangular shape descriptor are used to solve the camera's pose change through the rotation matrix and translation vector to avoid the problem of increased positioning error caused by using the feature point information in the area affected by lighting changes to solve the camera's pose.

2. The method according to claim 1, wherein In step three, the feature vector includes the gradient magnitude and gradient direction of the feature point.

3. A visual positioning device for triangular shape feature descriptors used in visual cameras to overcome illumination invariance, comprising an imaging device and a data processor, wherein the imaging device provides image data; The data processor includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program. The computer program code implements the method according to claim 1 or 2, and the obtained result is used for visual positioning.

4. The device according to claim 3, characterized in that The data processor is physically connected to the camera device via a wired connection or wirelessly.

5. Application of the method according to claim 1 or 2 or the device according to claim 3 or 4 in robot navigation, 3D modeling, unmanned autonomous driving, augmented reality, virtual reality, smart home control, or visual positioning for epidemic prevention and disinfection.

Images