Method and related device for extracting visual feature points of image

Abstract

The present application provides a method for extracting visual feature points of an image and a related device, wherein the method includes: dividing a visual positioning image to obtain multiple blocks; there are overlapping areas in the multiple blocks; extracting visual features from each block to obtain a feature point set corresponding to each block; merging feature points with the same coordinate parameters in the overlapping areas, and determining a total feature point set corresponding to the visual positioning image based on multiple feature point sets corresponding to the multiple blocks. Since feature points with the same coordinate parameters are repeatedly extracted in the overlapping areas, and there is always a feature point that is not close to the block boundary among the feature points with the same coordinate parameters, by merging the feature points with the same coordinate parameters, it is possible to correctly extract features within a preset range near the feature points, and then correctly extract the visual feature points near the dividing line, so the present application can improve the accuracy of visual feature point extraction.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for extracting visual feature points of an image and a related device. Background Art

[0002] In the visual localization task, the visual feature points of the image are usually extracted through a visual feature extraction module to complete the visual localization task. Among them, the effectiveness of visual feature point extraction plays a key role in the accuracy of localization.

[0003] At present, before extracting visual feature points from an image, the image is usually preprocessed (for example, dividing the image into a grid), and then the preprocessed image is extracted through extraction techniques (such as SIFT (Scale-invariant feature transform), ORB (Oriented FAST and Rotated BRIEF, fast feature point extraction and description algorithm), etc.). Since the existing image preprocessing method is very simple, the visual feature points near the dividing line of the image often have mismatches or failed matches, which greatly reduces the accuracy of visual positioning or even causes failure.

[0004] Therefore, how to improve the accuracy of visual feature point extraction is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a method and a related device for extracting visual feature points of an image, which can improve the accuracy of visual feature point extraction.

[0006] In a first aspect, an embodiment of the present application provides a method for extracting visual feature points of an image, comprising: dividing a visual positioning image to obtain a plurality of blocks; there are overlapping areas in the plurality of blocks; performing visual feature extraction on each of the blocks to obtain a feature point set corresponding to each of the blocks; merging feature points with the same coordinate parameters in the overlapping areas, and determining a total feature point set corresponding to the visual positioning image based on the plurality of feature point sets corresponding to the plurality of blocks.

[0007] In an optional embodiment, the step of performing visual feature extraction on each of the blocks to obtain a feature point set corresponding to each of the blocks includes: performing visual feature extraction on each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks; the number of feature points in the feature point set corresponding to each of the blocks is within a preset numerical range.

[0008] In an optional implementation, the step of dividing the visual positioning image to obtain a plurality of blocks includes: dividing the visual positioning image to obtain a plurality of blocks, wherein there is an overlapping area between any two adjacent blocks.

[0009] In an optional implementation, the size of each of the blocks is M×N, the size of the overlapping area is m×n, and m×n=γM×γN, where γ is a preset coefficient less than 1 and greater than 0.

[0010] In an optional embodiment, the step of performing visual feature extraction on each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks includes: performing visual feature extraction on each of the blocks in parallel to obtain an extracted feature point set corresponding to each of the blocks; the number of feature points in the extracted feature point set is greater than or equal to a preset value; and the feature point set is determined from the extracted feature point set, and the number of feature points in the feature point set is within a preset value range.

[0011] In an optional embodiment, the step of determining the feature point set from the extracted feature point set, wherein the number of feature points in the feature point set is within a preset numerical range, includes: determining q feature points from the extracted feature point set as the feature point set based on feature parameters of the feature points; q is within a preset numerical range, and the feature parameters include a response value and at least one parameter of a scale parameter.

[0012] In an optional embodiment, each feature point corresponds to a scale parameter. When there are multiple feature points with the same coordinate parameters in the multiple feature point sets, the step of merging the feature points with the same coordinate parameters in the overlapping area and determining the total feature point set corresponding to the visual positioning image based on the multiple feature point sets corresponding to the multiple blocks includes: deleting feature points with smaller scale parameters among the multiple feature points with the same coordinate parameters; and merging the multiple feature point sets after deletion into the total feature point set.

[0013] In a second aspect, an embodiment of the present application provides a device for extracting visual feature points of an image, comprising: a division module, used to divide a visual positioning image into multiple blocks; there are overlapping areas in the multiple blocks; an extraction module, used to extract visual features from each of the blocks, and obtain a feature point set corresponding to each of the blocks; the extraction module is also used to merge feature points with the same coordinate parameters in the overlapping areas, and determine the total feature point set corresponding to the visual positioning image based on the multiple feature point sets corresponding to the multiple blocks.

[0014] In an optional implementation, the extraction module is used to perform visual feature extraction on each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks; the number of feature points in the feature point set corresponding to each of the blocks is within a preset value range.

[0015] In an optional implementation, the segmentation module is used to segment the visual positioning image to obtain a plurality of blocks, wherein there is an overlapping area between any two adjacent blocks.

[0016] In an optional implementation, the size of each of the blocks is M×N, the size of the overlapping area is m×n, and m×n=γM×γN, where γ is a preset coefficient less than 1 and greater than 0.

[0017] In an optional embodiment, the extraction module is used to perform visual feature extraction on each of the blocks in parallel to obtain an extracted feature point set corresponding to each of the blocks; the number of feature points in the extracted feature point set is greater than or equal to a preset value; the extraction module is also used to determine the feature point set from the extracted feature point set, and the number of feature points in the feature point set is within a preset value range.

[0018] In an optional embodiment, the extraction module is used to determine q feature points from the extracted feature point set as the feature point set according to feature parameters of the feature points; q is within a preset numerical range, and the feature parameters include at least one of a response value and a scale parameter.

[0019] In an optional embodiment, each feature point corresponds to a scale parameter. When there are multiple feature points with the same coordinate parameters in the multiple feature point sets, the extraction module is used to delete the feature points with smaller scale parameters among the multiple feature points with the same coordinate parameters; the extraction module is also used to merge the multiple feature point sets after deletion into the total feature point set.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the method described in any of the aforementioned implementations.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the aforementioned implementation modes are executed.

[0022] Based on any of the above aspects, since there are overlapping areas in multiple blocks, feature points with the same coordinate parameters will be repeatedly extracted in the overlapping areas of two overlapping blocks, and among the feature points with the same coordinate parameters, there is always a feature point that is not close to the block boundary. By merging the feature points with the same coordinate parameters in the overlapping areas, it is possible to correctly extract the features within a preset range near the feature points, and then achieve the correct extraction of the visual feature points near the dividing line of the two overlapping blocks. Therefore, the present application can improve the accuracy of visual feature point extraction.

[0023] In addition, in some embodiments, since there is an overlapping area between any two adjacent blocks, there will always be a feature point with the same coordinate parameters that is not close to the block boundary for any feature point close to or located at the block boundary. This can completely avoid the situation where the visual feature points near the image dividing line are mismatched or failed to match due to the failure to extract features within a preset range near the point, thereby further improving the accuracy of visual feature point extraction.

[0024] In addition, in some embodiments, by performing visual feature extraction on each block in parallel, the present application can also improve the speed of extracting visual feature points of the image, and thus the present application can improve the extraction efficiency of visual feature point extraction. Since the number of feature points of the feature point set corresponding to each block is ensured to be within a preset numerical range, that is, the number of feature points of the feature point set in each block is similar, the feature points of the entire visual positioning image are distributed more evenly, avoiding the concentrated distribution of the extracted visual feature points in invalid feature areas such as pedestrians and dynamic objects. Therefore, the total feature point set extracted by the present application can represent the overall characteristics of the image, and can further improve the accuracy of visual feature point extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of an existing method for extracting visual feature points from an image is shown;

[0027] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present application is shown;

[0028] Figure 3 A schematic diagram showing a flow chart of a method for extracting visual feature points of an image provided by an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a user interface of a mobile phone in the process of extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0030] Figure 5 A schematic diagram of another user interface of a mobile phone in the process of extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0031] Figure 6 A schematic diagram showing the method provided in an embodiment of the present application for extracting visual feature points from an image;

[0032] Figure 7 A schematic diagram of another user interface of a mobile phone in the process of extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0033] Figure 8 A flowchart showing a specific method for extracting visual feature points of an image in the method for extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0034] Fig. 9 Another schematic diagram showing the method provided in the embodiment of the present application for extracting visual feature points from an image;

[0035] Fig.10 Another schematic diagram showing the method provided in the embodiment of the present application for extracting visual feature points from an image;

[0036] Fig.11 A flowchart showing a specific method for extracting visual feature points of an image in the method for extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0037] Fig.12 A flowchart showing a specific method for extracting visual feature points of an image in the method for extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0038] Fig.13 Another schematic diagram showing the method provided in the embodiment of the present application for extracting visual feature points from an image;

[0039] Fig.14 A schematic diagram of the structure of a device for extracting visual feature points of an image provided by an embodiment of the present application is shown;

[0040] Fig.15 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0042] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0043] In order to enable those skilled in the art to use the contents of this application, the following implementation method is given in combination with the specific application scenario of "users performing AR (Augmented Reality) navigation through electronic devices such as mobile phones". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around users performing AR navigation through electronic devices such as mobile phones, it should be understood that this is only an exemplary embodiment.

[0044] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0045] In the process of implementing the technical solution of the embodiment of the present application, the inventor of the present application found that:

[0046] At present, before extracting visual feature points from an image, the image is usually preprocessed (such as Figure 1 As shown, the image shown in ① is divided into 2×2 grids to obtain the image shown in ②), and then the pre-processed image is extracted by extraction technology (such as SIFT, ORB, etc.) to extract visual feature points (such as Figure 1As shown, the image shown in ② is subjected to visual feature point extraction to obtain the image shown in ③). Since the existing image preprocessing method is very simple, and the visual feature points need to be extracted based on the features within a preset range near the point, the visual feature points near the dividing line of the image often have mismatches or failed matches because the features within the preset range near the point are not extracted (such as the visual feature points near the dividing line in the image shown in ③).

[0047] Therefore, in the process of completing the visual positioning task by extracting visual feature points of images through existing extraction technology, since the existing image preprocessing method is very simple, the visual feature points near the dividing line of the image often have mismatches or failed matches, which causes a significant decrease in the accuracy of visual positioning or even failure.

[0048] Therefore, in order to improve the above defects, the embodiments of the present application propose a method for extracting visual feature points of an image and a related device, which can improve the accuracy of visual feature point extraction. It should be noted that the defects existing in the solutions in the above prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application for the above problems below should all be the contributions made by the inventor to the present application during the application process.

[0049] First, a method for extracting visual feature points of an image provided in an embodiment of the present application can be applied to electronic devices, and the device type of the electronic device can be a server, a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable electronic device, a virtual reality device, a car center console and other electronic devices, and the embodiment of the present application does not impose any restrictions on this.

[0050] It should be understood that the method for extracting visual feature points of an image provided by the present application can be applied to any electronic device that has the function of acquiring and processing data (eg, a server, a mobile phone, a tablet computer, etc.).

[0051] For ease of understanding, take the device type of the electronic device as a server as an example, please refer to Figure 2, the server 100 can be communicatively connected with the mobile phone 200, and the server 100 can include a processor. The processor can process information and / or data related to the extraction of visual feature points of the image to perform one or more functions described in this application. For example, the processor can obtain data related to the visual positioning image from the mobile phone, and extract visual feature points from the visual positioning image, thereby implementing the method for extracting visual feature points of the image provided in this application. In some embodiments, the processor may include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), or a microprocessor, or any combination thereof.

[0052] For ease of understanding, the following examples of this application will be Figure 2 Taking the server and mobile phone shown as an example, combined with the accompanying drawings and application scenarios (users perform AR navigation through mobile phones and other electronic devices), the method for extracting visual feature points of the image provided in the embodiment of the present application is specifically explained.

[0053] Please refer to Figure 3 , Figure 3 A schematic diagram of a flow chart of extracting visual feature points of an image provided in an embodiment of the present application. The method can be applied to the above-mentioned electronic device (such as a server) and may include the following steps:

[0054] S100, dividing the visual positioning image into multiple blocks; there are overlapping areas in the multiple blocks.

[0055] In some possible embodiments, based on Figure 2The mobile phone and server shown in the figure are combined with the application scenario of "users use mobile phones and other electronic devices to perform AR navigation", such as Figure 4 As shown, the screen display system of the mobile phone displays the currently output interface content 201, and the interface content 201 is the main interface of the mobile phone. The main interface displays multiple applications, such as "Contacts", "Positioning", "Device", "Notes", "Update", "Information", "Call", "Settings" and "Navigation". It should be understood that the interface content 201 can also include other applications, which is not limited in this application.

[0056] When the user needs to inquire about the route, AR navigation can be performed through the mobile phone (that is, the user can realize AR navigation by operating the mobile phone). At this time, the mobile phone can respond to the user operation. When the mobile phone detects that the user clicks the icon 202 of the "Navigation" application on the main interface, the "Navigation" application can be started. After starting the "Navigation" application, Figure 5 As shown, the interface content 201 of the mobile phone can present the content of the interface 203. At this time, the mobile phone can turn on the camera, take the image of the environment that the camera is facing, and display the image of the environment through the display screen.

[0057] After the mobile phone obtains the image of the environment, it uses the image as the visual positioning image and sends the visual positioning image to the server through the network. The server divides the visual positioning image into multiple blocks. Figure 6 As shown, after the server obtains the visual positioning image sent by the mobile phone, it can first divide the visual positioning image according to the block method shown in ① to obtain the multiple blocks (i.e., multiple image blocks) shown in ②, wherein the block method shown in ① includes four blocks A, B, C, and D, the A block is an area surrounded by solid lines, and the B, C, and D blocks are areas surrounded by dotted lines of different densities. The overlapping relationship of the four blocks is: there is an overlapping part between the A block and the B block, there is an overlapping part between the B block and the A and D blocks, there is an overlapping part between the D block and the B and C blocks, and there is an overlapping part between the C block and the D and B blocks. Through the four blocks, the server can obtain the corresponding four blocks (A`, B`, C`, D`) after dividing the visual positioning image. The overlapping relationship of the four blocks is consistent with the overlapping relationship of the above-mentioned four blocks, that is, the server divides the visual positioning image, and there are overlapping areas in the multiple blocks obtained.

[0058] It should be noted that the server can Figure 6The division method shown is used to divide the visual positioning image, and the visual positioning image can also be divided differently according to the actual application scenario, as long as it is ensured that there are overlapping areas in multiple blocks. Therefore, this application does not limit the way the server divides the visual positioning image. In addition, the existence of overlapping areas in the above-mentioned multiple blocks can be understood as: there is at least one overlapping area in the multiple blocks. In other words, among the multiple blocks, two blocks may not overlap.

[0059] It should be added that, after acquiring the image of the environment, the mobile phone can directly use the image as a visual positioning image and extract visual feature points of the visual positioning image. It can be understood that this application does not limit the specific device type of the electronic device to which the method provided by this application is applied. This embodiment only exemplifies the application of the method provided by this application on a server in a specific scenario.

[0060] It should be understood that before S100, the method provided in the present application may also include: obtaining a visual positioning image, wherein the visual positioning image may be data representing an image, or may be data representing a video (i.e., data of multiple images), and the present application does not limit the specific form of the visual positioning image.

[0061] S110, extract visual features from each block to obtain a feature point set corresponding to each block.

[0062] For some possible embodiments, please refer to Figure 6After the server divides the visual positioning image into multiple blocks (i.e., after obtaining the multiple blocks shown in ②), the server can also extract visual features for each block to obtain the visual feature points shown in ③, where the common visual feature points a to m can be understood as feature points with the same coordinate parameters. Specifically, A`, B`, C`, D`, the feature point set corresponding to the A` block includes 7 feature points, of which feature points a to f are the common feature points in the overlapping area of ​​the A` block and the B` block; the feature point set corresponding to the B` block includes 24 feature points, of which feature points a to f are the common feature points in the overlapping area of ​​the A` block and the B` block, g to j are the common feature points in the overlapping area of ​​the B` block and the C` block, and j to m are the common feature points in the overlapping area of ​​the B` block and the D` block. The feature point set corresponding to the C` block includes 5 feature points, of which feature points g to j are common feature points in the overlapping area of ​​the C` block and the B` block, and feature point j is a common feature point in the overlapping area of ​​the C` block and the D` block; the feature point set corresponding to the D` block includes 5 feature points, of which feature points j to m are common feature points in the overlapping area of ​​the D` block and the B` block, and feature point j is a common feature point in the overlapping area of ​​the D` block and the C` block. It can be understood that the above-mentioned common feature points are actually repeatedly extracted feature points.

[0063] It should be noted that the extraction method for extracting visual features from each block may be an ORB extraction method, a SIFT extraction method, a siftGPU extraction method, or the like.

[0064] It should be understood that due to the existence of overlapping areas in multiple blocks, feature points with the same coordinate parameters will be repeatedly extracted in the overlapping areas of two overlapping blocks. When a feature point exists in the overlapping area of ​​one block of the two overlapping blocks and is located at or close to the block boundary, the feature point is not actually located at or close to the block boundary in the other of the two overlapping blocks. That is, among the feature points with the same coordinate parameters, there is always one feature point that is not close to the block boundary.

[0065] S120, merging feature points with the same coordinate parameters in the overlapping area, and determining a total feature point set corresponding to the visual positioning image according to multiple feature point sets corresponding to multiple blocks.

[0066] In some possible embodiments, please refer to Figure 6Since there are repeatedly extracted feature points in the overlapping area (for example, the feature point sets corresponding to the A` block and the B` block both include feature points a to f), after obtaining the feature point set corresponding to each block, the server can also merge the feature points with the same coordinate parameters in the overlapping area. Taking the merging of the feature points with the same coordinate parameters in the overlapping area of ​​the A` block and the B` block (that is, the common feature points) as an example, the server can randomly retain one of the feature points a in the feature point set corresponding to the A` block and the feature point a in the feature point set corresponding to the B` block, that is, randomly retain one of the feature points with the same coordinate parameters, thereby merging the feature points with the same coordinate parameters in the overlapping area.

[0067] After merging the feature points with the same coordinate parameters in the overlapping area, the server can merge the four feature point sets corresponding to blocks A` to D` into one feature point set as the total feature point set corresponding to the visual positioning image, so as to determine the total feature point set corresponding to the visual positioning image based on multiple feature point sets corresponding to multiple blocks.

[0068] In some possible embodiments, after the server determines the total feature point set, it can determine the visual positioning point of the image based on the total feature point set, and determine the position identification data corresponding to the user's needs based on the visual positioning point, thereby effectively completing the visual positioning task. The position identification data is then sent to the mobile phone to support the mobile phone to perform AR navigation, and then the mobile phone can display the corresponding path indication on the display screen according to the position identification data (such as Figure 7 As shown, the mobile phone can display "Navigation information: Go straight for 30 meters to the destination." on the display screen according to the direction identification data to meet the user's need to inquire about the path, and the user can realize AR navigation by operating the mobile phone.

[0069] It should be understood that due to the existence of overlapping areas in multiple blocks, feature points with the same coordinate parameters will be repeatedly extracted in the overlapping areas of two overlapping blocks, and among the feature points with the same coordinate parameters, there is always a feature point that is not close to the block boundary. Therefore, by merging the feature points with the same coordinate parameters in the overlapping areas, it is possible to correctly extract features within a preset range near the feature point, and the existence of overlapping areas in multiple blocks can ensure that the visual feature points near the dividing line of the two overlapping blocks can be correctly extracted, so the present application can improve the accuracy of visual feature point extraction.

[0070] Further, in Figure 3 Based on this, a possible implementation of a complete solution is given below, please refer to Figure 8 , Figure 8 Another flow chart of the method for processing conversation data of multiple rounds of conversations provided in an embodiment of the present application is shown.

[0071] Regarding how to divide the visual positioning image to further improve the accuracy of visual feature point extraction, S100 may include:

[0072] S101, dividing the visual positioning image into multiple blocks, wherein there is an overlapping area between any two adjacent blocks.

[0073] In some possible embodiments, based on Figure 5 For the visual positioning image in the application scenario shown, please refer to Fig. 9 , the server can divide the visual positioning image into the division blocks (1 to 4) shown in ①, ②, ③, and ④ respectively, and obtain the following Fig.10 The multiple blocks (1 to 4) shown in the figure, wherein block 1 is adjacent to blocks 2 and 3, and there is an overlapping area between block 1 and blocks 2 and 3; block 2 is adjacent to blocks 1 and 4, and there is an overlapping area between block 2 and blocks 1 and 4; block 3 is adjacent to blocks 1 and 4, and there is an overlapping area between block 3 and blocks 1 and 4; block 4 is adjacent to blocks 2 and 3, and there is an overlapping area between block 4 and blocks 2 and 3. That is, Fig.10 Among the multiple blocks shown, there is an overlapping area between any two adjacent blocks.

[0074] Since there is an overlapping area between any two adjacent blocks, there will always be a feature point with the same coordinate parameters that is not close to the block boundary for any feature point close to or located at the block boundary. This can completely avoid the situation where the visual feature points near the image dividing line are mismatched or failed to match due to the failure to extract features within the preset range near the point, thereby further improving the accuracy of visual feature point extraction.

[0075] Further, in order to reasonably divide the visual positioning image, the size of each block obtained by the server dividing the visual positioning image may be M×N, the size of the overlapping area may be m×n, and m×n=γM×γN, where γ is a preset coefficient less than 1 and greater than 0. Optionally, the value range of γ may be 1 / 5 to 1 / 2, i.e., 0.2 to 0.5.

[0076] It should also be added that the number of the above-mentioned multiple blocks can be N1×N2, wherein N1 and N2 can range from 3 to 8, and N1 and N2 can be unequal.

[0077] It should be understood that the settings of the above parameters are all reasonable values ​​obtained by the inventor through multiple experiments. By setting the above parameters, the visual positioning image can be reasonably divided and the accuracy and efficiency of visual feature point extraction can be improved.

[0078] In addition, in the process of implementing the technical solution of the embodiment of the present application, the inventor of the present application also found that:

[0079] In actual application scenarios, there are usually a large number of invalid feature areas in the image (such as pedestrians, dynamic objects, etc.). When the texture of these areas is relatively rich, the visual feature points extracted by the existing extraction technology will be concentrated in these areas. Figure 1 As shown, when visual feature point extraction algorithms such as SIFT or ORB are used to extract visual feature points from the image shown in ②, the extracted visual feature points are usually distributed as shown in ③. The visual feature points in the image shown in ③ are concentrated on dynamic objects such as pedestrians, and the overall characteristics of the image are not well represented.

[0080] Therefore, in the process of completing the visual positioning task by using the existing extraction technology to extract the visual feature points of the image, the existing extraction technology often concentrates the visual feature points on the image in invalid feature areas such as pedestrians and dynamic objects, which will lead to a large number of mismatches or failed matches in the visual positioning, causing the accuracy of the visual positioning to drop significantly or even fail.

[0081] Therefore, in order to improve the above defects, the method provided in the embodiment of the present application can further improve the accuracy of visual feature point extraction. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application for the above problems below should all be the contributions made by the inventor to the present application during the application process.

[0082] exist Figure 3 Based on this, in order to solve the technical problem that the existing extraction technology often concentrates the visual feature points of the image in invalid feature areas such as pedestrians and dynamic objects, please refer to Fig.11 , S110 may include: performing visual feature extraction on each block in parallel to obtain a feature point set corresponding to each block; the number of feature points in the feature point set corresponding to each block is within a preset value range.

[0083] In some possible embodiments, in order to improve the speed of extracting visual feature points of an image, after dividing the visual positioning image into multiple blocks, the server can also extract visual features from each block in parallel to obtain a feature point set corresponding to each block, wherein the number of feature points in the feature point set corresponding to each block is within a preset numerical range.

[0084] It should be understood that by performing visual feature extraction on each block in parallel, the present application can also improve the speed of extracting visual feature points of the image, and thus the present application can improve the extraction efficiency of visual feature point extraction. Since the number of feature points of the feature point set corresponding to each block is ensured to be within a preset numerical range, that is, the number of feature points of the feature point set in each block is similar, the feature points of the entire visual positioning image are distributed more evenly, avoiding the extracted visual feature points from being concentrated in invalid feature areas such as pedestrians and dynamic objects. Therefore, the total feature point set extracted by the present application can represent the overall characteristics of the image, and can further improve the accuracy of visual feature point extraction.

[0085] Further, in Fig.11 Based on Fig.12 Regarding how to extract visual features from each block in parallel to obtain a feature point set corresponding to each block, S111 may include:

[0086] S111-1, extract visual features from each block in parallel to obtain an extracted feature point set corresponding to each block; the number of feature points in the extracted feature point set is greater than or equal to a preset value.

[0087] In some possible embodiments, assuming that the preset value is 250, the server may extract at least 250 feature points from each block as a feature point set corresponding to the block when performing visual feature extraction on each block in parallel.

[0088] S111 - 2 , determining a feature point set from the extracted feature point set, wherein the number of feature points in the feature point set is within a preset value range.

[0089] In some possible embodiments, assuming that the preset numerical range is 125±5, after obtaining the extracted feature point set corresponding to each block, taking the determination of the feature point set in the extracted feature point set corresponding to a block as an example, the server can extract 125 feature points from the extracted feature point set as the feature point set.

[0090] It should be noted that the above-mentioned preset numerical range can be simplified to one preset value.

[0091] Furthermore, each feature point corresponds to a feature parameter. As for how to determine the feature point set from the extracted feature point set, it is possible to: determine q feature points from the extracted feature point set as the feature point set according to the feature parameters of the feature points; the feature parameters include at least one parameter of the response value and the scale parameter.

[0092] When the feature parameters include response values, how to determine q feature points from the extracted feature point set as the feature point set according to the feature parameters of the feature points, in some possible embodiments, it is possible to: determine the set of the first q feature points in the extracted feature point set as the feature point set in descending order of the response values. For example, assuming that the preset numerical range is 125±5, after the extracted feature point set extracted from block A includes 250 feature points, the 250 feature points can be arranged in sequence in descending order of the response values ​​of the feature points, and then the first 125 feature points are taken as the feature point set.

[0093] When the feature parameters include a scale parameter, how to determine q feature points from the extracted feature point set as the feature point set according to the feature parameters of the feature points, in some possible embodiments, it is possible to: determine the set of the first q feature points in the extracted feature point set as the feature point set in descending order of the scale parameter. For example, assuming that the preset numerical range is 125±5, after the extracted feature point set extracted from block A includes 250 feature points, the 250 feature points can be arranged in sequence in descending order of the scale parameter of the feature point, and then the first 125 feature points are taken as the feature point set.

[0094] When the feature parameters include response values ​​and scale parameters, how to determine q feature points from the extracted feature point set as the feature point set according to the feature parameters of the feature points, in some possible embodiments, it is possible to: extract the set of the first q1 feature points in the feature point set in descending order of response values, and then determine the set of q2 feature points in the set of the first q1 feature points in descending order of scale parameters as the feature point set, wherein q1 is greater than q2, and q2 is within a preset numerical range. For example, assuming that the preset numerical range is 125±5, after the extracted feature point set extracted from block A includes 250 feature points, the 250 feature points can be arranged in sequence in descending order of the response values ​​of the feature points, and then the first 200 feature points are taken, and then the 200 feature points are arranged in sequence in descending order of the scale parameters of the feature points, and the first 125 feature points are taken as the feature point set.

[0095] Based on the above possible embodiments, it can be understood that the present application does not limit the specific form of determining q feature points as a feature point set from a set of extracted feature points according to the feature parameters of the feature points. In actual application scenarios, different implementation methods can be adopted according to actual needs to achieve determining q feature points as a feature point set from a set of extracted feature points according to the feature parameters of the feature points.

[0096] Optionally, assuming that the number of the above-mentioned multiple blocks is N1×N2, and the number of visual feature points to be extracted from a visual positioning image is Q, the number of visual feature points to be extracted from each block may be Q / (N1×N2), and the above-mentioned preset value may be α×Q / (N1×N2), wherein α ranges from 1.0 to 2.0, and Q ranges from 500 to 4000. It should be understood that the settings of the above-mentioned parameters are all reasonable values ​​obtained by the inventor through multiple experiments. By setting the above-mentioned parameters, the visual positioning image can be reasonably divided, and the accuracy and efficiency of visual feature point extraction can be improved.

[0097] Regarding how to merge feature points with the same coordinate parameters in the overlapping area, and determine the total feature point set corresponding to the visual positioning image according to the multiple feature point sets corresponding to the multiple blocks, each feature point corresponds to a scale parameter. When there are multiple feature points with the same coordinate parameters in the multiple feature point sets, S120 may include:

[0098] S121, deleting feature points with smaller scale parameters among multiple feature points with the same coordinate parameters.

[0099] In some possible embodiments, please refer to Fig.13 , the block shown in ① includes feature points a to h (common visual feature points), the block shown in ② includes feature points b to j, the block shown in ③ includes feature points a to e and k, and the block shown in ④ includes feature points b to e as well as i and j. That is to say, when the server extracts visual features for each block, the multiple feature points obtained have feature points with the same coordinate parameters. For example, the feature point a included in the block shown in ① and the feature point a also included in the block shown in ③. It should be understood that the feature point a included in the block shown in ① and the feature point a also included in the block shown in ③ only have the same coordinate parameters.

[0100] Taking deleting the feature point a included in the block shown in ① and the feature point a with a smaller scale parameter also included in the block shown in ③ as an example, the feature point with the smaller scale parameter can be deleted and the feature point with the larger scale parameter can be retained.

[0101] S122, merging the deleted feature point sets into a total feature point set.

[0102] In some possible embodiments, after deleting feature points with smaller scale parameters among multiple feature points with the same coordinate parameters, it can be understood that at this time, there is only one feature point with the same coordinate parameter in the multiple feature point sets, and then all feature points in the multiple feature point sets can be directly used as the total feature point set.

[0103] It should be understood that the present application obtains the extracted feature point set corresponding to each block by performing visual feature extraction on each block in parallel, and the number of feature points in the feature point set in each block is similar, and the feature points of the entire visual positioning image are distributed relatively evenly, so as to avoid the extracted visual feature points being concentrated in invalid feature areas such as pedestrians and dynamic objects. And the present application can correctly extract the features within a preset range near the feature point by merging the feature points with the same coordinate parameters in the overlapping area, and then the visual feature points near the dividing line of the two overlapping blocks can be correctly extracted, and then the present application can greatly improve the accuracy and efficiency of visual feature point extraction.

[0104] In order to execute the corresponding steps in the above embodiments and various possible methods, the following is an implementation method of a device for extracting visual feature points of an image. Fig.14 , Fig.14 A functional module diagram of the device for extracting visual feature points of an image provided in an embodiment of the present application is shown. It should be noted that the basic principle and technical effect of the device for extracting visual feature points of an image 300 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above-mentioned embodiments. The device for extracting visual feature points of an image 300 includes: a division module 310 and an extraction module 320.

[0105] Optionally, the above modules can be stored in the memory in the form of software or firmware or fixed in the operating system (OS) of the electronic device, and can be executed by the processor in the electronic device. At the same time, the data and program codes required to execute the above modules can be stored in the memory.

[0106] The partitioning module 310 may be used to support the electronic device in executing the above-mentioned S100, S101, etc., and / or other processes for the technology described herein.

[0107] The extraction module 320 may be used to support the electronic device in executing the above-mentioned S110, S120, S111, S111-1, S111-2, S121, S122, etc., and / or other processes for the technology described herein.

[0108] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0109] Based on the above method embodiment, the present application embodiment further provides an electronic device 400, such as Fig.154 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present application, including: a processor 410, a memory 420, and a bus 430. The memory 420 can be used to store one or more computer programs, the one or more computer programs including instructions (for example, Fig.14 When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430, and the processor 410 can execute the above instructions stored in the memory 420, so that the electronic device 400 can perform the extraction of visual feature points of the image provided in some embodiments of the present application, as well as various functional applications and data processing.

[0110] Based on the above method embodiment, the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method for extracting visual feature points of an image are executed.

[0111] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, it can execute the method for extracting visual feature points of the above-mentioned image, thereby solving how to improve the accuracy of visual feature point extraction, which is a technical problem that needs to be solved urgently, and thereby achieving the effect of improving the accuracy of visual feature point extraction.

[0112] In summary, the embodiments of the present application provide a method and a related device for extracting visual feature points of an image. The method includes: dividing a visual positioning image to obtain a plurality of blocks; there are overlapping areas in the plurality of blocks; performing visual feature extraction on each block to obtain a feature point set corresponding to each block; merging feature points with the same coordinate parameters in the overlapping areas, and determining a total feature point set corresponding to the visual positioning image based on the plurality of feature point sets corresponding to the plurality of blocks. Since there are overlapping areas in the plurality of blocks, feature points with the same coordinate parameters will be repeatedly extracted in the overlapping areas of the two overlapping blocks, and among the feature points with the same coordinate parameters, there is always a feature point that is not close to the block boundary. By merging the feature points with the same coordinate parameters in the overlapping areas, it is possible to correctly extract features within a preset range near the feature points, and thereby correctly extract the visual feature points near the dividing line of the two overlapping blocks. Therefore, the present application can improve the accuracy of visual feature point extraction.

[0113] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0114] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0117] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for extracting visual feature points of an image, characterized in that: include: Perform image segmentation on the visual positioning image to obtain multiple blocks; There are overlapping areas in the multiple blocks and the sizes of the multiple blocks are different; Performing visual feature extraction on each of the blocks to obtain a feature point set corresponding to each of the blocks, wherein each feature point in the feature point set corresponds to a scale parameter; and the number of feature points in the feature point set of each block is similar; When there are multiple feature points with the same coordinate parameters in the multiple feature point sets, delete the feature points with smaller scale parameters among the multiple feature points with the same coordinate parameters; The deleted multiple feature point sets are merged into a total feature point set.

2. The method according to claim 1, characterized in that The step of extracting visual features from each of the blocks to obtain a feature point set corresponding to each of the blocks comprises: Visual feature extraction is performed on each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks; the number of feature points in the feature point set corresponding to each of the blocks is within a preset value range.

3. The method according to claim 1, characterized in that The step of dividing the visual positioning image to obtain a plurality of blocks comprises: The visual positioning image is divided into multiple blocks, wherein there is an overlapping area between any two adjacent blocks.

4. The method according to claim 3, characterized in that The size of each of the blocks is M×N, the size of the overlapping area is m×n, and m×n=γM×γN, where γ is a preset coefficient less than 1 and greater than 0.

5. The method according to claim 2, characterized in that: The step of extracting visual features from each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks comprises: Performing visual feature extraction on each of the blocks in parallel to obtain an extracted feature point set corresponding to each of the blocks; the number of feature points in the extracted feature point set is greater than or equal to a preset value; The feature point set is determined from the extracted feature point set, and the number of feature points in the feature point set is within a preset value range.

6. The method according to claim 5, characterized in that The step of determining the feature point set from the extracted feature point set, wherein the number of feature points in the feature point set is within a preset value range, comprises: According to the characteristic parameters of the characteristic points, q characteristic points are determined from the extracted characteristic point set as the characteristic point set; q is within a preset numerical range, and the characteristic parameters include at least one parameter of a response value and a scale parameter.

7. A device for extracting visual feature points of an image, characterized in that: include: A division module, used for dividing the visual positioning image into multiple blocks; There are overlapping areas in the multiple blocks and the sizes of the multiple blocks are different; An extraction module, used for performing visual feature extraction on each of the blocks to obtain a feature point set corresponding to each of the blocks, wherein each feature point in the feature point set corresponds to a scale parameter; The extraction module is further used to delete feature points with smaller scale parameters among the multiple feature points with the same coordinate parameters when there are multiple feature points with the same coordinate parameters in the multiple feature point sets; and merge the multiple feature point sets after deletion into a total feature point set.

8. The device according to claim 7, characterized in that The extraction module is used to extract visual features from each of the blocks in parallel to obtain a feature point set corresponding to each of the blocks; the number of feature points in the feature point set corresponding to each of the blocks is within a preset value range.

9. The device according to claim 7, characterized in that The division module is used to divide the visual positioning image into multiple blocks, wherein there is an overlapping area between any two adjacent blocks.

10. The device according to claim 9, characterized in that The size of each of the blocks is M×N, the size of the overlapping area is m×n, and m×n=γM×γN, where γ is a preset coefficient less than 1 and greater than 0.

11. The device according to claim 8, characterized in that The extraction module is used to extract visual features from each of the blocks in parallel to obtain an extracted feature point set corresponding to each of the blocks; the number of feature points in the extracted feature point set is greater than or equal to a preset value; The extraction module is further configured to determine the feature point set from the extracted feature point set, wherein the number of feature points in the feature point set is within a preset value range.

12. The device according to claim 11, characterized in that The extraction module is used to determine q feature points from the extracted feature point set as the feature point set according to feature parameters of the feature points; q is within a preset numerical range, and the feature parameters include at least one parameter of a response value and a scale parameter.

13. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images