A feature extraction method, device and terminal suitable for infrared SLAM

Abstract

The present invention discloses a feature extraction method, device and terminal suitable for infrared SLAM, the method comprising: obtaining an image to be processed, extracting line features of the image to be processed; determining feature points of the image to be processed according to the line features of the image to be processed; and determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed. The present invention first extracts the line features of the image to be processed, and then determines the feature points and the descriptors of the feature points according to the line features. The line features are edge information in the image. In infrared images, the edge information of objects at different distances can be well preserved in the infrared images. Using line features for edge extraction can filter out the interference of nearby textures and reduce the false matching rate of infrared SLAM.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a feature extraction method, device and terminal suitable for infrared SLAM. Background Art

[0002] The SLAM (simultaneous localization and mapping) system that integrates visible light images and infrared images refers to the use of visible light RGB-D cameras and infrared cameras to perform real-time positioning and mapping of the hardware running the system (robots, virtual reality glasses, sweeping robots, etc.) in unknown environments. With the development of computer vision and machine learning, the research on visible light SLAM has achieved good results. Visible light SLAM works better when there is a stable light source, but when the light source is unstable and the light is dim, the accuracy is low and the robustness is poor; it cannot complete the positioning and mapping functions when there is no light source.

[0003] One solution to the illumination problem in infrared is to replace the visible light camera with an infrared camera. The infrared camera is an active infrared camera, which forms images by actively emitting infrared light and receiving infrared light through a filter camera. It is less affected by visible light and can also form images in dark environments at night. It has good performance in military night operations, night monitoring, cave exploration and other fields. Using infrared cameras as sensors in SLAM systems can effectively solve the problem of poor robustness of visible light SLAM in dark and dim environments. However, at the same time, infrared imaging has the disadvantages of not rich plane textures, low contrast, and low signal-to-noise ratio. In addition, infrared imaging is affected by distance, resulting in rich texture details in the near distance but not in the far distance. In general feature point extraction, in order to ensure the extraction and matching speed, the number of points extracted will be limited. This results in most feature points of an image being extracted on the near texture. And the near texture usually contains repeated textures such as the floor, which leads to an increase in the mismatch rate.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, the present invention provides a feature extraction method, device and terminal suitable for infrared SLAM, aiming to solve the problem that the feature points of infrared image feature extraction in the prior art are concentrated on nearby textures, resulting in a high mismatch rate of infrared SLAM.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A first aspect of the present invention provides a feature extraction method suitable for infrared SLAM, the method comprising:

[0008] Acquire an image to be processed, and extract line features of the image to be processed;

[0009] Determining feature points of the image to be processed according to line features of the image to be processed;

[0010] The descriptor corresponding to each feature point is determined according to the surrounding pixels of each feature point of the image to be processed.

[0011] The feature extraction method applicable to infrared SLAM, wherein before extracting the line features of the image to be processed, comprises:

[0012] Perform grayscale transformation processing on the image to be processed.

[0013] The feature extraction method applicable to infrared SLAM, wherein the grayscale transformation processing of the image to be processed includes:

[0014] A gamma transformation method is used to perform grayscale transformation processing on the image to be processed.

[0015] The feature extraction method applicable to infrared SLAM, wherein the extracting of line features of the image to be processed comprises:

[0016] Constructing a scale pyramid according to the image to be processed, wherein the scale pyramid includes multiple layers of images, each layer of images is obtained by scaling the next layer of images by a preset ratio, and the bottom layer of the scale pyramid is the image to be processed;

[0017] Line features are extracted from each layer of the image in the scale pyramid.

[0018] The feature extraction method applicable to infrared SLAM, wherein the step of constructing a scale pyramid according to the image to be processed comprises:

[0019] Scaling the image of the nth layer in the scale pyramid by a preset ratio to obtain an intermediate image;

[0020] The intermediate image is smoothed to obtain the image of the n+1th layer in the scale pyramid.

[0021] The feature extraction method applicable to infrared SLAM, wherein the extracting of line features in each layer of the image in the scale pyramid comprises:

[0022] The LSD algorithm is used to extract line features in each layer of the image in the scale pyramid.

[0023] The feature extraction method applicable to infrared SLAM, wherein determining the feature points of the image to be processed according to the line features of the image to be processed comprises:

[0024] The start and end points of each line feature are determined as feature points.

[0025] The feature extraction method suitable for infrared SLAM, wherein the step of determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed comprises:

[0026] Determining a target direction of the target feature point according to a gradient direction of the target feature point;

[0027] Determine at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point;

[0028] The descriptor corresponding to the target feature point is determined according to the pixel value size relationship between each pixel point in the at least one target pixel point and the target feature point.

[0029] The feature extraction method suitable for infrared SLAM, wherein the step of determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point, comprises:

[0030] The target direction of the target feature point is rotated clockwise or counterclockwise, a point selection direction is determined at every preset angle, and at least one pixel point is selected in each point selection direction as the target pixel point.

[0031] The feature extraction method applicable to infrared SLAM, wherein the direction of the target feature point is determined according to the line feature where the target feature point is located, comprises:

[0032] When the target feature point is the starting point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a first preset angle in a first preset clockwise direction;

[0033] When the target feature point is the end point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a second preset angle in a second preset clockwise direction;

[0034] The first preset clockwise direction is opposite to the second preset clockwise direction, and the sum of the first preset angle and the second preset angle is not equal to 360°.

[0035] A second aspect of the present invention provides a feature extraction device suitable for infrared SLAM, comprising:

[0036] A line feature extraction module, the line feature extraction module is used to obtain an image to be processed and extract line features of the image to be processed;

[0037] A feature point determination module, the feature point determination module is used to determine the feature points of the image to be processed according to the line features of the image to be processed;

[0038] A feature point description module is used to determine a descriptor corresponding to each feature point according to surrounding pixels of each feature point of the image to be processed.

[0039] According to a third aspect of the present invention, a terminal is provided, comprising a processor and a computer-readable storage medium communicatively connected to the processor, wherein the computer-readable storage medium is suitable for storing a plurality of instructions, and the processor is suitable for calling the instructions in the computer-readable storage medium to execute the steps of implementing any of the above-mentioned feature extraction methods applicable to infrared SLAM.

[0040] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the above-mentioned feature extraction methods applicable to infrared SLAM.

[0041] Compared with the prior art, the present invention provides a feature extraction method, device and terminal suitable for infrared SLAM. The feature extraction method suitable for infrared SLAM first extracts the line features of the image to be processed, and then determines the feature points and the descriptors of the feature points according to the line features. The line features are the edge information in the image. In the infrared image, the edge information of objects at different distances can be well preserved in the infrared image. The edge extraction using line features can filter out the interference of nearby textures and reduce the false matching rate of infrared SLAM. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of an embodiment of a feature extraction method applicable to infrared SLAM provided by the present invention;

[0043] Figure 2 Comparison of visible light image and infrared image Figure 1 ;

[0044] Figure 3 Comparison of visible light image and infrared image Figure 2 ;

[0045] Figure 4A schematic diagram of an image to be processed in an embodiment of a feature extraction method applicable to infrared SLAM provided by the present invention;

[0046] Figure 5 Effect of grayscale transformation processing on the image to be processed in the embodiment of the feature extraction method suitable for infrared SLAM provided by the present invention Figure 1 ;

[0047] Figure 6 Effect of grayscale transformation processing on the image to be processed in the embodiment of the feature extraction method suitable for infrared SLAM provided by the present invention Figure 2 ;

[0048] Figure 7 A structural principle diagram of an embodiment of a feature extraction device applicable to infrared SLAM provided by the present invention;

[0049] Figure 8 A schematic diagram of the principles of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] The feature extraction method applicable to infrared SLAM provided by the present invention can be applied to a terminal with computing capability. The terminal can execute the feature extraction method applicable to infrared SLAM provided by the present invention to extract features from an image to be processed. The terminal can be, but is not limited to, various computers, mobile terminals, smart home appliances, wearable devices, etc.

[0052] Embodiment 1

[0053] like Figure 1 As shown, in one embodiment of the feature extraction method applicable to infrared SLAM, the steps are as follows:

[0054] S100: Acquire an image to be processed, and extract line features of the image to be processed.

[0055] In SLAM, for each frame in the video captured by the camera, feature extraction is required and matching with the next frame is performed to determine the change in the camera's position. The image to be processed can be a frame of the video captured by the camera. Infrared SLAM is captured using an infrared camera under low light conditions. In this embodiment, the image to be processed is an infrared image.

[0056] like Figure 2 and Figure 3 As shown, compared with the visible light image ( Figure 2 ), infrared image( Figure 3 ) has the disadvantages of not rich plane texture, low contrast, and low signal-to-noise ratio. In addition, infrared imaging is affected by distance, and plane texture features will decrease from near to far due to the influence of distance, that is, the texture details are rich near but not far. However, in contrast to plane texture features, the edge features of objects can be well preserved in infrared images. In visible light images, the edge can only define the junction between one color block and another, and cannot distinguish the difference in distance between the two objects. Thanks to the infrared imaging method, in infrared images, the closer the object is to the infrared sensor, the higher the pixel grayscale value. Therefore, the grayscale on both sides of the object edge is high near and low far. There are many ways to extract edges, such as using Canny operator, Sobel operator, and Gabor filter to filter out non-edge information in the image and only retain edge information. In this embodiment, the method of extracting line features is adopted.

[0057] Since the original near-infrared image often has disadvantages such as low contrast and many noise points, it cannot meet the conditions required for feature extraction. Before extracting the line features of the image to be processed, the image to be processed is preprocessed. Before extracting the line features of the image to be processed, the steps include:

[0058] Perform grayscale transformation processing on the image to be processed.

[0059] For low-contrast images, the grayscale transformation methods that can be used are histogram equalization and gamma transformation, and their formulas are as follows:

[0060] Histogram equalization method:

[0061] Gamma Transform Method:

[0062] Among them, I x,y ′ represents the gray value of the point with coordinates (x, y) in the transformed image, I x,y Represents the pixel value of the point with coordinates (x, y) in the image before transformation, I max is the maximum pixel value in the image before transformation, I min is the minimum pixel value in the image before transformation, and α is a constant.

[0063] It is not difficult to see from the formula that after grayscale transformation using the histogram equalization method, the grayscale value of the pixel depends on the pixel value of the entire image, while the gamma change only depends on the set constant α. The imaging principle of the near-infrared camera is that the infrared light emits infrared rays to hit the object, and the infrared light is received by diffuse reflection to form an image. Therefore, when there is a mirror or other infrared light source in the environment (such as Figure 4At the same time, the distance from the object will also affect the gray value of the image. When it is close to an object, the sensor will receive more reflected infrared rays, and the gray value in the image will also become higher. Therefore, if the histogram equalization method is used for gray conversion, the gray value of the image will remain unchanged before and after the conversion (such as Figure 5 As shown in the figure, the grayscale value of the same pixel position in two frames of images differs too much, which affects the results of feature extraction and description. Therefore, in this embodiment, a relatively stable gamma transformation method is used to map the pixel value to the corresponding value, which ensures that the image contrast is improved and that even if an additional infrared light source is added at the same position, the pixel value of the same pixel position in different images does not differ too much (as shown in the figure). Figure 6 shown).

[0064] That is, the grayscale transformation processing of the image to be processed specifically includes the steps of:

[0065] A gamma transformation method is used to perform grayscale transformation processing on the image to be processed.

[0066] Furthermore, in this embodiment, in order to improve the accuracy of feature matching, a scale pyramid is used to perform multi-layer feature extraction. Specifically, the extraction of line features of the image to be processed includes:

[0067] S100, constructing a scale pyramid according to the image to be processed, wherein the scale pyramid includes multiple layers of images, each layer of images is obtained by scaling the next layer of images by a preset ratio, and the bottom layer of the scale pyramid is the image to be processed;

[0068] S200: extracting line features from each layer of the image in the scale pyramid.

[0069] Specifically, constructing a scale pyramid according to the image to be processed includes:

[0070] Scaling the image of the nth layer in the scale pyramid by a preset ratio to obtain an intermediate image;

[0071] The intermediate image is smoothed to obtain the image of the n+1th layer in the scale pyramid.

[0072] The bottom layer of the scale pyramid, i.e., the image of the first layer, is the image to be processed. The image to be processed is scaled by a preset ratio, and the scaled image is smoothed to obtain the image of the second layer of the scale pyramid. The image of the second layer is then scaled by a preset ratio, and the scaled image is smoothed to obtain the image of the third layer of the scale pyramid. This is repeated until the number of layers of the scale pyramid reaches a preset number. The scale pyramid is constructed, and the scaling ratio each time can be equal. The smoothing process can be implemented using a low-pass filter.

[0073] In a possible implementation, the LSD algorithm is used to extract line features, that is, the extraction of line features in each layer of the image in the scale pyramid includes the following steps:

[0074] The LSD algorithm is used to extract line features in each layer of the image in the scale pyramid.

[0075] Specifically, the pseudo code of the process of extracting line features by the LSD algorithm can be shown in the following table:

[0076]

[0077]

[0078] It can be understood that those skilled in the art can modify the values ​​in the above pseudo code according to actual conditions, such as the scale of Gaussian kernel sampling, the range of linked list container, etc.

[0079] Of course, those skilled in the art can understand that in the art, there are other line feature extraction methods, such as EDLines algorithm, etc., and the present invention is not limited to the LSD algorithm.

[0080] Please refer again Figure 1 The feature extraction method for infrared SLAM provided in this embodiment also includes the steps of:

[0081] S200: Determine feature points of the image to be processed according to line features of the image to be processed.

[0082] Each line feature includes a starting point, an end point, and an angle. After obtaining the line feature, it is refined into a point feature through the line feature, and the descriptor is obtained by describing the local information around the feature point. The descriptor of the feature point can be used for feature point matching between different images. Specifically, the descriptor is a descriptor with a dimension of n generated by sequentially comparing the pixel value size relationship between the feature point and the surrounding specific n points. Between two images, the best matching feature point can be found by calculating the Hamming distance between the descriptor of a feature point and the descriptors of all feature points in the other image.

[0083] In the process of extracting line features through the LSD algorithm, the extraction is performed based on the gradient direction and gradient value of the point. That is to say, the size relationship between the pixel values ​​on both sides of the line feature is fixed, and the grayscale value of the pixel on one side of the line segment is mostly greater than the grayscale value on the other side. If the point feature is extracted using the point in the middle of the line feature, the Hamming distance between the descriptors will be too close due to the monotony of the local information, resulting in an increase in the mismatch rate of the feature points. The starting point and the terminal of the line feature are the starting point and the terminal of the pixel value with the same gradient value and gradient direction, which means that the points on both sides of the starting point and the terminal have a gradient direction exceeding the threshold. The starting point and the terminal of the line feature in the image are the junction or the bend of the edge of the image, so the local information there is richer than the middle of the line segment. Using the starting point and the terminal of the line feature as the feature point can effectively reduce the mismatch rate. Therefore, in this embodiment, the feature points of the image to be processed are determined according to the line features of the image to be processed, including:

[0084] The start and end points of each line feature are determined as feature points.

[0085] S300: Determine a descriptor corresponding to each feature point according to surrounding pixels of each feature point of the image to be processed.

[0086] Specifically, determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed includes:

[0087] S310, determining a target direction of the target feature point according to a gradient direction of the target feature point;

[0088] S320, determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point;

[0089] S330. Determine a descriptor corresponding to the target feature point according to a pixel value size relationship between each pixel in the at least one target pixel and the target feature point.

[0090] Specifically, determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point includes:

[0091] The target direction of the target feature point is rotated clockwise or counterclockwise, a point selection direction is determined at every preset angle, and at least one pixel point is selected in each point selection direction as the target pixel point.

[0092] Taking the preset angle of 90° as an example, when determining the descriptor of the target feature point, the target direction is taken as the 0° direction, and at least one pixel point is selected as the target pixel point in the 0° direction, 90° direction, 180° direction, and 270° direction. The target pixel point can be a pixel point whose distance from the target feature point in the selected direction is within a preset range (for example, one pixel range, two pixel ranges).

[0093] The camera may rotate when shooting. In order to improve the accuracy of feature point matching, it is necessary to make the descriptors corresponding to the feature points consistent when the image is rotated. The pixel change of the target feature point in the image changes most dramatically along the gradient direction of the target feature point, and the physical characteristics such as the shape of the object in the image will not change. In other words, the gradient direction of the target feature point should be consistent in different images. In this embodiment, the gradient direction of the target feature point is used to determine the target direction of the target feature point, so that the point selection direction remains consistent in the rotated image.

[0094] Specifically, determining the direction of the target feature point according to the line feature where the target feature point is located includes:

[0095] When the target feature point is the starting point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a first preset angle in a first preset clockwise direction;

[0096] When the target feature point is the end point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a second preset angle in a second preset clockwise direction;

[0097] The first preset clockwise direction is opposite to the second preset clockwise direction, and the sum of the first preset angle and the second preset angle is not equal to 360°.

[0098] Since the gradient direction of the feature point is consistent in different images, the target direction determined based on the gradient direction can ensure that the point selection direction is consistent when generating the descriptor of the feature point, and ensure the rotation invariance of the feature. In actual applications, the starting point of a line feature and the end point of another line feature may be very close. In order to prevent the starting point of a line feature and the end point of another line feature from mismatching, in this embodiment, the starting point and the end point of the line feature use different rotation directions to determine the target direction, and the sum of the first preset angle and the second preset angle is not equal to 360°, which effectively prevents the starting points and end points of different line features from mismatching.

[0099] The effect of the present invention is verified by applying the feature extraction method for infrared SLAM provided by the present invention to the SLAM system. The CoRBS dataset of the German Artificial Intelligence Research Center is used in the verification. The reason for not using mainstream datasets such as KITTI and EuRoC is that they only contain visible light image information, but no near-infrared images. CoRBS is a rare dataset that contains visible light image, near-infrared image and depth image information. It is generated using Kinetic V2, and the depth information and near-infrared images are collected using a near-infrared camera. At the same time, the visible light image is collected using a visible light camera and aligned with the depth image.

[0100] At present, SLAM research mainly uses two evaluation indicators: relative pose error (RPE) and absolute trajectory error (ATE) to evaluate the accuracy of SLAM. In order to more directly evaluate the accuracy of the algorithm and the global consistency of the trajectory, this study uses ATE to evaluate the algorithm.

[0101] ATE is the direct difference between the estimated pose and the true pose, which can very intuitively reflect the algorithm accuracy and global consistency of the trajectory. It should be noted that the estimated pose and the groundtruth are usually not in the same coordinate system, so they need to be aligned first. For binocular SLAM and RGB-D SLAM, the scale is unified, so it is necessary to calculate a transformation matrix S∈SE(3) from the estimated pose to the true pose by the least squares method; for monocular cameras, there is scale uncertainty, and it is necessary to calculate a similar transformation matrix S∈Sim(3) from the estimated pose to the true pose. Therefore, the ATE of the i-th frame is defined as follows:

[0102]

[0103] When the total number n and the interval Δ are known, we can get m = n - Δ ATEs, and then we can use the root mean square error RMSE to calculate the error and get an overall value:

[0104]

[0105] Where trans(Ei) represents the translation part of the relative pose error. In order to comprehensively measure the performance of the algorithm, the average RMSE of all Δ can be calculated:

[0106]

[0107] The smaller the calculated RMSE is, the smaller the error between the trajectory and the actual trajectory of the system is, and the more accurate the positioning is.

[0108] After the SLAM system runs for a period of time, because of the accumulation of noise and system errors, the accuracy of positioning and mapping will also gradually decline due to the accumulation of errors. For the offset caused by the cumulative error, SLAM is usually by encountering the scene that has been reached, that is, after the motion trajectory forms a closed loop, by comparing the current posture (that is, the end point of the closed loop) in the world coordinate system with the posture (that is, the starting point of the closed loop) when passing last time, after confirming the formation of the closed loop, the cumulative error is corrected by the way of back-end optimization to the impact brought to positioning and mapping. Therefore, particularly for infrared SLAM with large noise and less texture, the success rate (also called recall rate) of loop detection has greatly affected the accuracy of positioning and mapping. Therefore, for the effect of the present invention, also the recall rate is used as one of the criteria for judging, and the recall rate test method is to test 100 times for the loop data set that will occur, and record the number of times of successful loop detection.

[0109] Extraction method RMSE(m) Recall rate (%) ORB-SLAM 0.05435 0％ OUR-SLAM 0.02040 98％

[0110] From the results in the table above, it can be seen that the SLAM recall rate is low when using the traditional ORB method to extract infrared features, which leads to large positioning errors in complex scenes or complex movements, and because loop detection cannot be triggered, there is no backend optimization. The method of the present invention is used for SLAM positioning and mapping, which ensures the recall rate and improves the positioning accuracy.

[0111] In summary, this embodiment provides a feature extraction method suitable for infrared SLAM. For the image to be processed, the line features of the image to be processed are first extracted, and then the feature points and the descriptors of the feature points are determined based on the line features. The line features are the edge information in the image. In the infrared image, the edge information of objects at different distances can be well preserved in the infrared image. Using line features for edge extraction can filter out the interference of nearby textures and reduce the false matching rate of infrared SLAM.

[0112] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present specification are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0113] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0114] Embodiment 2

[0115] Based on the above embodiments, the present invention also provides a feature extraction device suitable for infrared SLAM, such as Figure 7 As shown, the feature extraction device suitable for infrared SLAM includes:

[0116] A line feature extraction module, the line feature extraction module is used to obtain the image to be processed and extract the line features of the image to be processed, as described in the first embodiment;

[0117] A feature point determination module, the feature point determination module is used to determine the feature points of the image to be processed according to the line features of the image to be processed, as described in the first embodiment;

[0118] A feature point description module is used to determine the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed, as described in the first embodiment.

[0119] Embodiment 3

[0120] Based on the above embodiments, the present invention also provides a terminal, such as Figure 8 As shown, the terminal includes a processor 10 and a memory 20. Figure 8Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0121] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a feature extraction program 30 suitable for infrared SLAM is stored on the memory 20, and the feature extraction program 30 suitable for infrared SLAM can be executed by the processor 10, thereby realizing the feature extraction method suitable for infrared SLAM in the present application.

[0122] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other chip, used to run the program code or process data stored in the memory 20, such as executing the feature extraction method suitable for infrared SLAM.

[0123] In one embodiment, when the processor 10 executes the feature extraction program 30 suitable for infrared SLAM in the memory 20, the following steps are implemented:

[0124] Acquire an image to be processed, and extract line features of the image to be processed;

[0125] Determining feature points of the image to be processed according to line features of the image to be processed;

[0126] The descriptor corresponding to each feature point is determined according to the surrounding pixels of each feature point of the image to be processed.

[0127] Wherein, before extracting the line features of the image to be processed, the method includes:

[0128] Perform grayscale transformation processing on the image to be processed.

[0129] The feature extraction method applicable to infrared SLAM, wherein the grayscale transformation processing of the image to be processed includes:

[0130] A gamma transformation method is used to perform grayscale transformation processing on the image to be processed.

[0131] Wherein, extracting line features of the image to be processed includes:

[0132] Constructing a scale pyramid according to the image to be processed, wherein the scale pyramid includes multiple layers of images, each layer of images is obtained by scaling the next layer of images by a preset ratio, and the bottom layer of the scale pyramid is the image to be processed;

[0133] Line features are extracted from each layer of the image in the scale pyramid.

[0134] Wherein, constructing a scale pyramid according to the image to be processed includes:

[0135] Scaling the image of the nth layer in the scale pyramid by a preset ratio to obtain an intermediate image;

[0136] The intermediate image is smoothed to obtain the image of the n+1th layer in the scale pyramid.

[0137] The step of extracting line features from each layer of the image in the scale pyramid includes:

[0138] The LSD algorithm is used to extract line features in each layer of the image in the scale pyramid.

[0139] The feature extraction method applicable to infrared SLAM, wherein determining the feature points of the image to be processed according to the line features of the image to be processed comprises:

[0140] The start and end points of each line feature are determined as feature points.

[0141] The step of determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed includes:

[0142] Determining a target direction of the target feature point according to a gradient direction of the target feature point;

[0143] Determine at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point;

[0144] The descriptor corresponding to the target feature point is determined according to the pixel value size relationship between each pixel point in the at least one target pixel point and the target feature point.

[0145] Wherein, determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point comprises:

[0146] The target direction of the target feature point is rotated clockwise or counterclockwise, a point selection direction is determined at every preset angle, and at least one pixel point is selected in each point selection direction as the target pixel point.

[0147] Wherein, determining the direction of the target feature point according to the line feature where the target feature point is located includes:

[0148] When the target feature point is the starting point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a first preset angle in a first preset clockwise direction;

[0149] When the target feature point is the end point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a second preset angle in a second preset clockwise direction;

[0150] The first preset clockwise direction is opposite to the second preset clockwise direction, and the sum of the first preset angle and the second preset angle is not equal to 360°.

[0151] Embodiment 4

[0152] The present invention also provides a computer-readable storage medium, in which one or more programs are stored. The one or more programs can be executed by one or more processors to implement the steps of the feature extraction method applicable to infrared SLAM as described above.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature extraction method suitable for infrared SLAM, characterized in that: The method comprises: Acquire an image to be processed, and extract line features of the image to be processed; Determining feature points of the image to be processed according to line features of the image to be processed; Determine a descriptor corresponding to each feature point according to surrounding pixels of each feature point of the image to be processed; The extracting line features of the image to be processed comprises: Constructing a scale pyramid according to the image to be processed, wherein the scale pyramid includes multiple layers of images, each layer of images is obtained by scaling the next layer of images by a preset ratio, and the bottom layer of the scale pyramid is the image to be processed; Extracting line features in each layer of the image in the scale pyramid; The extracting line features in each layer of the image in the scale pyramid includes: The LSD algorithm is used to extract line features in each layer of the image in the scale pyramid; The step of determining the feature points of the image to be processed according to the line features of the image to be processed comprises: Determine the starting point and end point of each line feature as feature points; The step of determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed comprises: Determining a target direction of the target feature point according to a gradient direction of the target feature point; Determine at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point; Determine a descriptor corresponding to the target feature point according to a pixel value relationship between each pixel in the at least one target pixel and the target feature point; The step of determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point comprises: The target direction of the target feature point is rotated clockwise or counterclockwise, a point selection direction is determined at every preset angle, and at least one pixel point is selected in each point selection direction as the target pixel point.

2. The feature extraction method applicable to infrared SLAM according to claim 1, characterized in that, Before extracting the line features of the image to be processed, the method includes: Perform grayscale transformation processing on the image to be processed.

3. The feature extraction method applicable to infrared SLAM according to claim 2, characterized in that, The grayscale transformation processing of the image to be processed includes: A gamma transformation method is used to perform grayscale transformation processing on the image to be processed.

4. The feature extraction method applicable to infrared SLAM according to claim 1, characterized in that, The step of constructing a scale pyramid according to the image to be processed includes: Scaling the image of the nth layer in the scale pyramid by a preset ratio to obtain an intermediate image; The intermediate image is smoothed to obtain the image of the n+1th layer in the scale pyramid.

5. The feature extraction method applicable to infrared SLAM according to claim 1, characterized in that, The determining the direction of the target feature point according to the line feature where the target feature point is located comprises: When the target feature point is the starting point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a first preset angle in a first preset clockwise direction; When the target feature point is the end point of the line feature, the target direction of the target feature point is the gradient direction of the target feature point rotated by a second preset angle in a second preset clockwise direction; The first preset clockwise direction is opposite to the second preset clockwise direction, and the sum of the first preset angle and the second preset angle is not equal to 360°.

6. A feature extraction device suitable for infrared SLAM, characterized in that: include: A line feature extraction module, the line feature extraction module is used to obtain an image to be processed and extract line features of the image to be processed; The extracting line features of the image to be processed comprises: Constructing a scale pyramid according to the image to be processed, wherein the scale pyramid includes multiple layers of images, each layer of images is obtained by scaling the next layer of images by a preset ratio, and the bottom layer of the scale pyramid is the image to be processed; Extracting line features in each layer of the image in the scale pyramid; The extracting line features in each layer of the image in the scale pyramid includes: The LSD algorithm is used to extract line features in each layer of the image in the scale pyramid; A feature point determination module, the feature point determination module is used to determine the feature points of the image to be processed according to the line features of the image to be processed; The step of determining the feature points of the image to be processed according to the line features of the image to be processed comprises: Determine the starting point and end point of each line feature as feature points; A feature point description module, the feature point description module is used to determine a descriptor corresponding to each feature point according to surrounding pixels of each feature point of the image to be processed; The step of determining the descriptor corresponding to each feature point according to the surrounding pixels of each feature point of the image to be processed comprises: Determining a target direction of the target feature point according to a gradient direction of the target feature point; Determine at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point; Determine a descriptor corresponding to the target feature point according to a pixel value relationship between each pixel in the at least one target pixel and the target feature point; The step of determining at least one target pixel point corresponding to the target feature point within a preset range around the target feature point in a preset order according to the target direction of the target feature point comprises: The target direction of the target feature point is rotated clockwise or counterclockwise, a point selection direction is determined at every preset angle, and at least one pixel point is selected in each point selection direction as the target pixel point.

7. A terminal, characterized in that: The terminal includes: a processor, a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the computer-readable storage medium to execute the steps of the feature extraction method applicable to infrared SLAM as described in any one of claims 1 to 5 above.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the feature extraction method applicable to infrared SLAM as described in any one of claims 1-5.

Images